Technical SEO Audit WorkshopRunning the crawl and gathering evidence · Lesson 4 of 14

Search Console, logs and performance evidence

Article · 13 min · 9 min lecture

Video lecture

Search Console, logs and performance evidence

13 chapters · about 9 min · full transcript

Coming soon

Chapter 1 of 13

Search Console, logs and performance evidence

  • Four sources, one story
  • Segment by market and template
  • An evidence log for every claim

The narrated lecture is in production

Every chapter is scripted and ready. Browse the chapters and read the full transcript now — the video will appear here when it’s published.

Chapters

Crawl data shows what could happen; other sources show what did

A crawler simulates. Search Console shows what Google did with your URLs, logs show what Googlebot actually requested, and field performance data shows what real users experienced. A strong audit joins all four.

Search Console: questions and where to answer them

QuestionReportKiran Home finding (illustrative)
When did the decline start and where?Performance, compare periods, filter by page segment via regexDrop starts the week of relaunch; concentrated in UAE and Pakistan traffic to product pages
Which URLs are not indexed and why?Page indexing, filtered by sitemapMany product URLs under "Alternate page with proper canonical tag" and "Duplicate, Google chose different canonical"
What canonical did Google pick?URL Inspection on samplesGoogle-selected canonical = UK product for UAE pages
Are rich results intact?Enhancements (Products, Breadcrumbs)Product items fell after relaunch; missing offers in JSON-LD on some templates
Is the server healthy for Googlebot?Crawl statsResponse time increased after relaunch; some 5xx spikes

Useful filters in the Performance report — the built-in branded queries filter (where available for the property) or your own regex, plus custom annotations to mark the relaunch date on charts:

Page contains regex:  /en-ae/products/|/ar-ae/products/
Query does not match regex: kiran ?home|kiranhome

Compare 28 days before relaunch with 28 days after, and also year-on-year to control for seasonality (for example, if the relaunch coincided with a Ramadan period one year and not the other).

Logs: what Googlebot actually did

Cloudflare logs for four weeks, filtered to verified Googlebot, then classified by segment:

SegmentShare of Googlebot requestsObservation
Facet parametersLarge majorityCrawl trapped in filter combinations
Internal searchNoticeable share/search?q= URLs linked from "popular searches" widget
ProductsSmall shareMany products not requested at all in four weeks
Old URLs (redirects/404s)ModerateGooglebot still requests pre-relaunch URLs heavily

Shares described qualitatively; your real audit would quantify them.

Two conclusions for the report: Googlebot's attention is being spent on URLs with no value, and legacy URLs are still important to Google, so redirect gaps matter.

A simple join makes the point unarguable: products with zero Googlebot hits in four weeks cross-referenced with revenue from the pre-relaunch period. If many former best-sellers have not been crawled, that is a headline finding for the stakeholder report.

Performance evidence

Run PageSpeed Insights for each template and read the field data (CrUX) first:

Template (mobile)LCPINPCLSStatus
HomeNeeds improvementGoodGood—
CategoryPoorNeeds improvementGoodHero carousel image lazy-loaded
ProductNeeds improvementPoorPoorReviews widget shifts layout and blocks main thread
BlogGoodGoodGood—

Illustrative statuses. Where CrUX has insufficient data for a URL, use the origin-level data and the Search Console Core Web Vitals report groups. Then use a lab trace to identify the cause.

Rendering evidence

For three products, use URL Inspection's live test and view the rendered HTML. Search for a unique sentence from the product description and the price. On Kiran Home, the description renders, but "Customers also bought" links are loaded after interaction and do not appear — so they provide no discovery value.

Building the evidence log

Every claim in the final report must trace back to evidence. Keep an evidence log:

E-07 | Source: Cloudflare logs 1–28 Mar | Filter: verified Googlebot | Finding: facet URLs dominate requests | File: logs_segment_pivot.xlsx
E-08 | Source: GSC Page indexing, sitemap products-en-ae.xml | Finding: most URLs 'Alternate page with proper canonical tag' | Screenshot: e08.png

This protects you when a stakeholder challenges a recommendation, and it makes verification later straightforward.

Newer evidence sources (2026)

SourceWhat it adds for this audit
Search Console generative AI reportWhether Kiran Home pages appear in AI Overviews/AI Mode, by page and country (impressions only) — useful to see if AI exposure also fell after relaunch
Bing Webmaster Tools (Site Explorer, AI Performance)Whether Bing indexing and Copilot citations dropped too — a second engine's view helps separate site problems from Google-specific changes
CDN firewall/bot eventsWhether real crawlers (Googlebot, Bingbot, AI search bots) are being challenged since the relaunch (hypothesis H5)

Hands-on: before/after by segment with the Search Console API

# Reuses the authenticated `gsc` client from the Technical SEO Mastery course (Lesson 7.3)
import pandas as pd
def pull(start, end):
    body = {"startDate": start, "endDate": end, "dimensions": ["page", "country"],
            "rowLimit": 25000, "dataState": "final"}
    rows = gsc.searchanalytics().query(siteUrl="sc-domain:kiranhome.example", body=body).execute().get("rows", [])
    return pd.DataFrame([{"page": r["keys"][0], "country": r["keys"][1], "clicks": r["clicks"]} for r in rows])

before, after = pull("2026-05-04", "2026-05-31"), pull("2026-06-08", "2026-07-05")
for d, label in ((before, "before"), (after, "after")):
    d["segment"] = d.page.str.extract(r"/(products|collections|journal)/")[0].fillna("other")
    d["period"] = label
both = pd.concat([before, after])
print(both.pivot_table(index=["segment", "country"], columns="period", values="clicks", aggfunc="sum").fillna(0)
          .assign(change=lambda t: (t.after - t.before) / t.before.replace(0, pd.NA)).sort_values("before", ascending=False).head(15))

(Dates are illustrative; use 28-day windows either side of the relaunch plus the same windows a year earlier. Paginate with startRow for larger sites.)

Worked example 2: when the evidence exonerates the relaunch

For a different client, a UK garden-furniture retailer (illustrative), the before/after segmentation shows the decline spread evenly across all templates and markets, starting two weeks before the relaunch and matching the end of the spring buying season in last year's data too. Logs show healthy crawling; indexing is stable. The evidence points to seasonality, not a technical regression — and the audit says so, saving the client from an unnecessary emergency rebuild.

Common mistakes

  • Reading Performance data without regex segmentation, so a market-specific drop is diluted.
  • Comparing with the previous 28 days only, ignoring seasonality.
  • Using unverified "Googlebot" log lines.
  • Treating Lighthouse lab scores as the verdict on Core Web Vitals.

Key takeaways

  • Join crawl data with Search Console, logs and field performance data for evidence-backed findings.
  • Segment Search Console Performance with regex and compare both period-over-period and year-over-year.
  • Log analysis shows where Googlebot's attention actually goes; join it with revenue to show business impact.
  • Keep an evidence log so every recommendation traces to a source.

Check your understanding

Quick questions to lock in the lesson. They don’t count towards your certificate.

  1. Which source tells you what Googlebot actually requested?
  2. Why compare year-over-year as well as the 28 days before a relaunch?
  3. A product page's 'Customers also bought' links only load after a click. What does this mean for SEO?

Put it into practice

Create an evidence log template and fill in five entries from a real site's Search Console, logs or PageSpeed Insights.

Enrol for free to save your progress

Reading is always free. Enrol to keep your place, take the final assessment and earn a verifiable certificate.