Status codes, redirects and soft 404s

Video lecture

Status codes, redirects and soft 404s

13 chapters · about 8 min · full transcript

Coming soon

Chapter 1 of 13

Status codes, redirects and soft 404s

What each code tells Google
Choosing the right redirect
Chains, loops and soft 404s

The narrated lecture is in production

Every chapter is scripted and ready. Browse the chapters and read the full transcript now — the video will appear here when it’s published.

Chapters

Transcript of the narration, chapter by chapter.

0:00 Status codes, redirects and soft 404s

Before a search engine reads a single word on your page, it reads a three-digit number. The HTTP status code. Get it right and you tell Google exactly what to do with the URL and its signals. Get it wrong and you create soft four-oh-fours, wasted crawl and lost link equity. In this lecture you'll learn what each status code tells Google, how to choose the right redirect, how to find chains and loops with a short script, and how to fix soft four-oh-fours at scale.

0:37 The status map

Here's the quick map. Two hundred means OK, eligible for indexing. Three-oh-one and three-oh-eight are permanent moves and strong signals that the target should be canonical. Three-oh-two and three-oh-seven are temporary, weaker canonical signals, though if they persist Google may treat them as permanent. Three-oh-four means not modified. Four-oh-four and four-ten mean gone; the URL drops out over time, and there's no penalty for having them. Four-twenty-nine and five hundreds slow crawling, and prolonged errors can drop URLs.

1:11 404s are normal

Here's the key insight people miss. Four-oh-fours are normal and healthy for content that no longer exists. You don't need to redirect every dead URL. The problem is four-oh-fours on URLs that should exist, or dead URLs that have valuable backlinks or traffic. Those deserve a redirect to the most relevant equivalent.

1:34 Choosing the redirect

Choosing the redirect. For a permanent move, like a URL change, HTTPS, a domain move or merged content, use three-oh-one or three-oh-eight. For something genuinely temporary, like a short promotion or a test where the original should stay canonical, use three-oh-two or three-oh-seven. For planned downtime, use five-oh-three with a Retry-After header, not a redirect to a maintenance page that returns two hundred. And avoid meta refresh and JavaScript redirects when a server redirect is possible, because they're slower and weaker signals.

2:10 Chains and loops

Chains and loops. A chain is A to B to C. Google documents following up to ten hops, but every hop costs crawl resources, adds latency for users, and adds risk. Best practice: redirect every legacy URL directly to its final destination in one hop. Update internal links, canonicals, hreflang and sitemaps to point at final URLs so users and bots rarely hit redirects at all. And when you add a new redirect, rewrite older rules that pointed at the now-redirected URL.

2:46 Hands-on: the redirect tracer

Hands-on. The lesson text has a Python script that traces redirects hop by hop for a list of URLs. It uses HEAD requests, falls back to GET if the server rejects HEAD, follows Location headers, detects loops, and prints the final URL, its status, the number of hops, the redirect types and a problem flag: chain, loop, temporary or error. Run it on legacy URLs, on internal links that point at redirects, and on your top backlink targets.

3:20 Soft 404s

Now soft four-oh-fours. A soft four-oh-four is a URL that returns two hundred but looks like an error or an empty page. No products found. This item is unavailable. A thin template with no main content. Or a redirect to something irrelevant. Search Console reports them in Page indexing. Fixes depend on the cause. Deleted product: return four-oh-four or four-ten, or redirect to a close replacement. Empty filter results: four-oh-four. Out of stock but returning: keep two hundred with useful content like alternatives and a restock date. Single-page app error views: a real server four-oh-four.

4:01 Example 1: 12,000 discontinued products (illustrative)

Worked example one, with illustrative details. A UK retailer's audit finds twelve thousand discontinued product URLs returning two hundred with sorry, this product is no longer available. Some have backlinks. The plan: pull backlink data and Search Console clicks per URL; redirect those with links or traffic to the closest current product or the parent category, only where genuinely relevant; return four-ten for the rest; remove them from sitemaps and internal links; and watch the soft four-oh-four count trend down in Search Console.

4:37 Redirect mapping principles

A quick word on redirect mapping principles, because this is where most redirect work goes wrong. Map each old URL to the most relevant equivalent, not the home page. Mass-redirecting unrelated URLs to the home page is typically treated as a soft four-oh-four, so you gain nothing. If there's genuinely no equivalent, a four-oh-four or four-ten is honest and perfectly acceptable. And keep redirects in place for the long term. Google's migration guidance suggests at least a year, and many practitioners keep them indefinitely, because external links and bookmarks persist for years.

5:17 Example 2: maintenance done wrong (illustrative)

Worked example two, also illustrative. A Doha electronics retailer takes its store offline for a four-hour upgrade and redirects every URL with a three-oh-two to a maintenance page that returns two hundred. Afterwards, Search Console shows spikes in soft four-oh-fours, and some product results temporarily show the maintenance page. Next time, the right pattern is a five-oh-three with Retry-After on the original URLs, a short window, and never a two hundred we'll be back page in place of real content.

5:52 Watch me do it: top 100 backlink targets

Watch me do it. I'll clean up redirects for a retailer's top backlink targets. Step one: I export the top one hundred linked URLs from a backlink tool and save them to urls dot txt. Step two: I run the redirect tracer. The summary shows sixty-two resolving directly, eighteen chains, four loops, nine temporary redirects on permanent moves, and seven errors. Step three: I open a chain. Old HTML page, to a new slug, to a market folder, to a trailing-slash version. Four hops. The final URL is right, so the fix is simply pointing the first URL straight at the final one. Step four: the loops. Two URLs redirect to each other because of a trailing-slash rule fighting a CMS redirect. I note both rules. Step five: the temporary redirects. They're three-oh-twos created by a plugin default. They should be three-oh-ones. Step six: the errors. Five return four-oh-four; I check each against Search Console clicks and find relevant replacements for four; the fifth has no equivalent, so a four-ten is honest. Step seven: I write one ticket with a CSV attached: old URL, correct final URL, correct status. After deploy, I run the tracer again. One hundred of one hundred resolve in one hop or less.

7:22 Mistakes and measures

Common mistakes. Using three-oh-twos for permanent moves because it's the platform default, so check your CMS or plugin. Redirecting everything to the home page, which Google typically treats as a soft four-oh-four. Custom error pages that return two hundred. And letting redirect rules pile up for years without flattening chains. How do you measure success? Soft four-oh-fours trending down, zero chains on priority URLs, and legacy URLs with backlinks resolving in one hop to relevant two-hundred pages.

7:55 Recap and try this now

Recap. Status codes are instructions. Use permanent redirects for permanent moves, five-oh-three for downtime, and honest four-oh-fours for gone content. Flatten chains to one hop and fix soft four-oh-fours by cause. Try this now. Export your top one hundred backlink target URLs, run them through the redirect tracer, and ticket every chain, loop, temporary redirect and error you find.

Status codes are instructions

Search engines read the HTTP status code before anything else. Getting it right is one of the highest-leverage technical fixes because it tells Google what to do with the URL and its signals.

Code	Meaning	What Google does
200	OK	Content eligible for indexing
301	Moved permanently	Strong signal that the target should be canonical
308	Permanent redirect (method preserved)	Treated like 301
302 / 307	Temporary redirect	Weak canonical signal; source may stay indexed. If it persists, Google may treat it as permanent
304	Not modified	Reuses the previously fetched content
404	Not found	URL dropped from index over time; no penalty for having 404s
410	Gone	Similar to 404; may be processed slightly faster
429	Too many requests	Treated like a server error; slows crawling
500 / 503	Server error / unavailable	Crawling slows; prolonged errors can drop URLs

Key insight: 404s are normal and healthy for content that no longer exists. The problem is 404s on URLs that should exist, or with valuable backlinks pointing at them.

Choosing the right redirect

Permanent move (URL change, HTTPS, domain move, merged content): 301 or 308.
Temporary (short promotion, maintenance on a specific page, geo or A/B test where the original should stay canonical): 302 or 307.
Planned downtime: 503 with a Retry-After header, not a redirect to a maintenance page returning 200.
Avoid: meta refresh and JavaScript redirects when a server redirect is possible. Google can follow them, but they are slower and weaker signals.

# Nginx: single-hop redirects
server {
  listen 80;
  server_name example.com www.example.com;
  return 301 https://www.example.com$request_uri;
}
location = /old-guide { return 301 /guides/new-guide; }

# Apache .htaccess
RewriteEngine On
RewriteCond %{HTTPS} off [OR]
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteRule ^(.*)$ https://www.example.com/$1 [L,R=301]
Redirect 301 /old-guide /guides/new-guide

Redirect chains and loops

A chain is A → B → C. Google documents following up to 10 hops, but every hop costs crawl resources, adds latency for users, and adds risk. Best practice:

Redirect every legacy URL directly to its final destination (one hop).
Update internal links, canonicals, hreflang and sitemaps to point at final URLs, so users and bots rarely hit redirects at all.
When you add new redirects, rewrite older ones that point at the now-redirected URL.

A quick chain check:

curl -sIL https://example.com/old-page | grep -Ei "^(HTTP|location)"

Redirect mapping principles

Map to the most relevant equivalent, not the home page. Mass-redirecting unrelated URLs to the home page is typically treated as a soft 404.
If there is no equivalent, a 404/410 is honest and acceptable.
Keep redirects in place long term — Google's migration guidance suggests at least a year, and many practitioners keep them indefinitely because external links persist.

Soft 404s

A soft 404 is a URL that returns 200 but looks like an error or empty page: "No products found", "This item is unavailable", a thin template with no main content, or an irrelevant redirect. Search Console reports these in Page indexing.

Fixes by cause:

Cause	Fix
Deleted product showing "not found" with 200	Return 404/410, or 301 to a close replacement
Empty category/filter results	Return 404, or noindex and remove internal links to it
Out-of-stock product	Keep 200 if it will return, with useful content (alternatives, restock date); avoid thin "unavailable" pages
SPA error view	Server 404 or JavaScript-injected noindex
Redirect to irrelevant page	Redirect to a relevant page or return 404

Worked example 1: 12,000 discontinued products

A UK retailer's audit finds 12,000 discontinued product URLs returning 200 with "Sorry, this product is no longer available". Some have backlinks. The plan: pull backlink data and Search Console clicks for each URL; 301 those with links or traffic to the closest current product or the parent category (only where genuinely relevant); return 410 for the rest; remove them from sitemaps and internal links; monitor the soft 404 count trending down in Search Console.

Hands-on: a redirect chain and status checker

# pip install requests
import csv, sys, requests

def trace(url, max_hops=10):
    hops, current = [], url
    for _ in range(max_hops):
        r = requests.head(current, allow_redirects=False, timeout=15,
                          headers={"User-Agent": "Mozilla/5.0 (compatible; audit-bot)"})
        if r.status_code in (405, 501):          # some servers reject HEAD
            r = requests.get(current, allow_redirects=False, timeout=15, stream=True)
        hops.append((current, r.status_code))
        loc = r.headers.get("Location")
        if r.status_code in (301, 302, 303, 307, 308) and loc:
            current = requests.compat.urljoin(current, loc)
            if current in [h[0] for h in hops]:
                hops.append((current, "LOOP")); break
        else:
            break
    return hops

w = csv.writer(sys.stdout)
w.writerow(["start", "final", "final_status", "hops", "types", "problem"])
for url in (l.strip() for l in open("urls.txt") if l.strip()):
    h = trace(url)
    types = [str(s) for _, s in h[:-1]]
    final, status = h[-1]
    problem = ("loop" if status == "LOOP" else "chain" if len(h) > 2 else
               "temporary" if any(t in ("302", "307") for t in types) else
               "error" if isinstance(status, int) and status >= 400 else "")
    w.writerow([url, final, status, len(h) - 1, "|".join(types), problem])

Run it on legacy URLs, internal links pointing at redirects, and backlink targets. Anything flagged chain, loop, temporary (for a permanent move) or error goes into a ticket with the correct single-hop destination.

Worked example 2: a Gulf retailer's maintenance page

A Doha electronics retailer (illustrative) takes its store offline for a four-hour platform upgrade and redirects every URL with a 302 to /maintenance, which returns 200. Afterwards, Search Console shows spikes in soft 404s and some product URLs temporarily replaced by the maintenance page in results. The better pattern for next time: return 503 with a Retry-After header on the original URLs during maintenance, keep the window short, and never serve a 200 "we'll be back" page in place of real content.

Common mistakes

Using 302s for permanent moves because it is the platform default (check your CMS or plugin).
Redirecting everything to the home page.
Custom error pages that return 200.
Letting redirect rules pile up for years without flattening chains.

Use 301/308 for permanent moves, 302/307 for temporary, 503 plus Retry-After for maintenance.
Redirect legacy URLs in a single hop to the most relevant equivalent, then update internal links to the final URL.
404 and 410 are healthy for genuinely gone content; mass home-page redirects become soft 404s.
Soft 404s are 200 responses that look empty or erroneous — fix the status code or the content.

Run a crawl, export all 3xx URLs with their redirect chains, and produce a flattened one-hop redirect map for the worst ten chains.

Enrol for free to save your progress

Reading is always free. Enrol to keep your place, take the final assessment and earn a verifiable certificate.

Status codes, redirects and soft 404s