Search Console skill

Google Search Console

Workflow

1. Property Setup

Verify ownership via DNS TXT record (most reliable):

google-site-verification=XXXXXXXXXXXXXXXX

Alternatives: HTML file upload, HTML meta tag, Google Analytics, Google Tag Manager.

Prefer a Domain property (example.com, verified by DNS TXT). It aggregates every protocol and subdomain (http, https, www, m., blog.) into one dataset, so you see the whole site without managing many properties. Add a URL-prefix property (https://example.com/) only when you need narrower scope — e.g. delegating reporting/ownership for one subfolder to a team, isolating a staging host, or because a tool requires a URL-prefix siteUrl. URL-prefix properties match the exact protocol + host you enter and do not include subdomains.

The siteUrl you use in the APIs must match the property type exactly: Domain → sc-domain:example.com; URL-prefix → https://example.com/ (with trailing slash). Mismatches return 403 PERMISSION_DENIED even when you own the site.

2. Page Indexing Audit

The old Coverage report ("Valid / Valid with warnings / Excluded / Error") is gone. Current GSC has a Page indexing report that splits URLs into two buckets — Indexed and Not indexed — and lists a reason per URL. There is no "Valid with warnings" tier; warnings now live under structured-data / enhancement reports, not page indexing.

Reading the report:

• Open Indexing → Pages. The top chart shows the indexed vs not-indexed split over time.
• Each reason row links to a sample of affected URLs (the table is a sample, capped at ~1,000 rows — see API/BigQuery below to get the full list).
• A reason being non-zero is not automatically a bug: most healthy sites permanently carry "Alternate page with proper canonical", "Page with redirect", and "Excluded by 'noindex' tag" rows. Triage by intent, not by zeroing every bucket.

Not-indexed reasons — likely cause and fix:

Validation workflow (replaces the old "Valid" state): after fixing a reason, click into the reason and press Validate Fix. GSC re-crawls the sample over days/weeks; status moves Started → Passed (or Failed if some URLs still trip the rule). Don't re-press Validate while one is running — it restarts the clock.

Canonical check (do this before "fixing" a not-indexed page): in URL Inspection, compare User-declared canonical vs Google-selected canonical.

• They match → your signal won; nothing to do.
• They differ → Google is clustering this URL with another. Common causes: contradictory signals (canonical says A, sitemap/internal links/hreflang point at B), near-duplicate content, or A being a weaker variant (less linked, slower, parameterized). Align all signals (canonical tag, internal links, sitemap entry, hreflang return tags) on the single preferred URL — a lone canonical tag is a hint, not a command.

3. Performance Analysis

Key metrics: impressions, clicks, CTR, average position. The default Performance → Search results report covers the web property; Discover and Google News are separate tabs that only appear once you have eligible traffic and have their own rules (see below).

Analysis by query cluster:

1. Export performance data (Queries tab, up to 16 months in the UI).
2. Group queries by intent/topic.
3. Compare cluster CTR against rough position benchmarks:

Treat these as a sanity-check ceiling, not a target — actual CTR varies hugely by query type (branded, navigational, local pack, shopping).

AI Overviews / AI Mode caveat (critical for 2026): GSC counts an impression when your link appears in an AI Overview, but the user often gets their answer without clicking, so CTR at a given position can look "broken" even when the snippet is fine. GSC does not break out AI-surface impressions separately, and the position reported is the link's position, not the AI block's. Before rewriting titles on a low-CTR query, check whether it's an informational query that now triggers an AI Overview (search it). Don't "fix" a snippet that's losing clicks to a zero-click AI answer.

• If actual CTR < expected (and no AI Overview): title/description likely needs work, or a competitor has a richer snippet.
• If actual CTR > expected: strong snippet — protect this content; note what's working and reuse it.

Comparing across the AI-Overview rollout: sudden CTR drops with flat/rising impressions on informational clusters usually mean AI Overviews ate the clicks, not a ranking loss. Segment by intent before drawing conclusions.

Quick wins — filter for:

• Position 5-15 with high impressions → optimize to push into top 5.
• High impressions, low CTR, no AI Overview → rewrite title tags and meta descriptions.
• Position 1-3 with declining impressions → content freshness, or query volume / AI-surface shift.

Regex filters (UI): in the Queries/Pages filter, switch the match type to Custom (regex) for slicing the UI without exporting. Regex uses RE2 syntax and is case-sensitive by default — prefix (?i) to ignore case.

4. Sitemap Management

Submit at Sitemaps → Add a new sitemap:

https://example.com/sitemap.xml

Sitemap audit checklist:

• All indexable pages included; only canonical, 200-status, indexable URLs (no noindex, no redirects, no canonicalized-away duplicates).
• <lastmod> reflects the actual last meaningful content change. Do NOT stamp every URL with today's date on each deploy — Google learns to distrust <lastmod> and may ignore it sitewide, hurting recrawl of genuinely updated pages. Update it only when the page content actually changed.
• Response is HTTP 200 with valid XML; URLs are absolute and properly entity-escaped (& → &).
• ≤ 50,000 URLs and ≤ 50 MB uncompressed per file; split larger sites into multiple sitemaps behind a sitemap index. Gzip is fine.
• Submitted in GSC and referenced via Sitemap: in robots.txt.

Sitemap index triage: submit the index file in GSC, not each child. The Sitemaps report then shows per-child "Discovered URLs" and status. When a sitemap shows "Couldn't fetch": confirm it isn't blocked by robots.txt, returns 200 (not 3xx/4xx), and is valid XML — re-fetch with curl -sI and validate. Resubmitting won't help until the underlying fetch succeeds. The "Discovered URLs" count is how many URLs Google read from the file, not how many it indexed — cross-check indexing in the Page indexing report.

5. Core Web Vitals

Check Page Experience → Core Web Vitals:

Debugging workflow:

1. Identify failing URL groups in GSC
2. Test specific URLs with PageSpeed Insights
3. Fix the highest-impact issue first (usually LCP)
4. Validate fix in GSC (takes 28 days for field data)

Common fixes:

• LCP: Optimize hero image (WebP, proper sizing, preload), eliminate render-blocking resources
• INP: Reduce JavaScript execution time, break long tasks, use requestIdleCallback
• CLS: Set explicit width/height on images/video, avoid dynamic content injection above the fold

6. URL Inspection

Use the URL Inspection tool to:

• Check whether a specific URL is indexed and view the user-declared vs Google-selected canonical.
• See how Googlebot renders the page ("View crawled page" + "Test live URL" for the current state).
• Request indexing for a single new/updated page (rate-limited; see safety note).
• Debug discovery/crawl/index status for one URL at a time.

API access (URL Inspection API). Read-only; returns the same index status the UI shows. It does not request indexing (no public indexing endpoint for normal web pages — the Indexing API is only for JobPosting/BroadcastEvent markup).

OAuth scope: https://www.googleapis.com/auth/webmasters.readonly. Quota (as of Jun 2026; verify at https://developers.google.com/webmaster-tools/limits): ~2,000 inspections/day and ~600/min per property. The siteUrl must exactly match a property you own, in its native form.

# pip install google-api-python-client google-auth
from google.oauth2 import service_account
from googleapiclient.discovery import build

SCOPES = ["https://www.googleapis.com/auth/webmasters.readonly"]
SITE_URL = "sc-domain:example.com"        # Domain property; URL-prefix → "https://example.com/"

# Service account must be added as a user on the property in GSC Settings → Users and permissions.
creds = service_account.Credentials.from_service_account_file(
    "service-account.json", scopes=SCOPES
)
service = build("searchconsole", "v1", credentials=creds, cache_discovery=False)

def inspect(url: str) -> dict:
    body = {"inspectionUrl": url, "siteUrl": SITE_URL}
    res = service.urlInspection().index().inspect(body=body).execute()
    r = res["inspectionResult"]["indexStatusResult"]
    return {
        "verdict": r.get("verdict"),                     # PASS / NEUTRAL / FAIL
        "coverage": r.get("coverageState"),              # e.g. "Submitted and indexed"
        "user_canonical": r.get("userCanonical"),
        "google_canonical": r.get("googleCanonical"),    # differs => Google overrode you
        "robots": r.get("robotsTxtState"),               # ALLOWED / DISALLOWED
        "last_crawl": r.get("lastCrawlTime"),
    }

print(inspect("https://example.com/page"))

Use OAuth user credentials (google-auth-oauthlib, InstalledAppFlow) instead of a service account if you can't add a service account to the property; the scope and API calls are identical.

7. Search Analytics API & BigQuery Bulk Export

The UI caps every Performance table at 1,000 rows per view and aggregates away long-tail queries. To get the full dataset, use the Search Analytics API or the Bulk Data Export to BigQuery.

Search Analytics API (searchanalytics.query) — same webmasters.readonly scope and service object as §6. Key constraints:

• Returns up to 25,000 rows per request; page with startRow (multiples of 25,000) until you get fewer than 25,000 back.
• Same 16-month retention window as the UI; request startDate/endDate in YYYY-MM-DD.
• dimensions can combine query, page, country, device, searchAppearance, date. Query data is anonymized: rare queries are dropped for privacy, so summed clicks/impressions will be lower than the totals row — expected, not a bug.
• type selects the surface: web (default), image, video, news, discover, googleNews. Discover/News have no query dimension.
• dataState: "all" includes the most recent (still-incomplete) days; default "final" excludes them. Don't compare a "fresh" pull against a "final" one.

# reuse `service` and SITE_URL from the §6 snippet
def export_queries(start: str, end: str) -> list[dict]:
    rows, start_row = [], 0
    while True:
        body = {
            "startDate": start, "endDate": end,
            "dimensions": ["query", "page"],
            "type": "web", "dataState": "final",
            "rowLimit": 25000, "startRow": start_row,
        }
        resp = service.searchanalytics().query(siteUrl=SITE_URL, body=body).execute()
        batch = resp.get("rows", [])
        rows += batch
        if len(batch) < 25000:
            return rows           # last page
        start_row += 25000        # next page

data = export_queries("2026-01-01", "2026-06-01")
print(len(data), "rows")          # far beyond the UI's 1,000-row cap

Bulk Data Export → BigQuery (the enterprise path for sites that blow past 25k rows/day or want unsampled history). Configure in GSC → Settings → Bulk data export: pick a Google Cloud project, grant the GSC export service account BigQuery Job User + BigQuery Data Editor, and GSC writes a daily dump into a searchconsole dataset. Tables you get:

• searchdata_site_impression — aggregated by property (one row per query/date, no URL).
• searchdata_url_impression — aggregated by URL (query × page × date); this is where the long tail lives.
• exportLog — which dates have landed (export is daily, ~2-day lag; backfill is not retroactive — you only get data from the day you enable it forward).

Query it with standard SQL — no row caps, full history from enablement:

-- top pages by clicks, last 28 days, from the URL-level export
SELECT url, SUM(clicks) AS clicks, SUM(impressions) AS impressions,
       SAFE_DIVIDE(SUM(clicks), SUM(impressions)) AS ctr
FROM `your-project.searchconsole.searchdata_url_impression`
WHERE data_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 28 DAY)
GROUP BY url
ORDER BY clicks DESC
LIMIT 100;

BigQuery storage/query is billed by Google Cloud, not GSC. Partition-prune on data_date (the tables are date-partitioned) to keep scan costs down. Note the same anonymized-query caveat applies — is_anonymized_query = TRUE rows have a null query.

8. Structured Data & Rich Results

GSC surfaces structured-data issues as per-type Enhancement reports that appear only when Google detects that markup on your site. Two important deprecations — do not add these to chase rich results:

• HowTo rich results were deprecated and removed (2023). The HowTo enhancement report is gone; HowTo markup no longer produces any special SERP treatment.
• FAQ rich results were restricted to authoritative government/health sites (Aug 2023) and are effectively unavailable for general sites by 2026. Don't expect FAQ accordions in search from FAQPage markup. (You can still keep FAQPage markup for non-SERP machine understanding, but it won't earn a rich result.)

Currently worth marking up (eligible types, mid-2026 — confirm at https://developers.google.com/search/docs/appearance/structured-data/search-gallery):

• Product / Merchant listings — price, availability, aggregateRating; the path to free Shopping/product listings.
• Review snippet — only on supported entity types (Product, Recipe, Book, etc.); self-serving "reviews of your own business" on a LocalBusiness page are not eligible.
• Breadcrumb — almost always worth it; replaces the URL line in the SERP.
• Article / NewsArticle — eligibility for Top stories and rich presentation.
• Organization — logo, name, contact, sameAs; feeds the knowledge panel / entity understanding.
• LocalBusiness — NAP, hours, geo for local features.
• Video (VideoObject) — key moments, video thumbnails, Video tab.
• Event, Recipe, JobPosting, Dataset, Q&A (forum), Profile/Discussion — where they fit the content.

Beyond rich results, valid schema (especially Organization, Article, Product, BreadcrumbList) helps machine/LLM understanding of the page — increasingly relevant as AI surfaces summarize content.

Validation workflow:

1. Test markup with the Rich Results Test (https://search.google.com/test/rich-results) — it shows eligibility, which the generic Schema.org validator does not.
2. Fix errors in the GSC enhancement report for that type.
3. Click Validate Fix; GSC re-crawls and moves the issue Started → Passed.

Common schema errors:

• Missing required fields (e.g. Product offers without price/priceCurrency; aggregateRating without ratingValue/reviewCount).
• Invalid dates — use ISO 8601 (2026-06-07 or full 2026-06-07T09:00:00+02:00).
• Markup describing content not visible on the page (against Google's policy → can trigger a structured-data manual action).
• Structured-data URL not matching the page's canonical.

9. Search Appearance Optimization

There is no single title formula. Write to the dominant query intent for the page, keep titles unique across the site, and lead with the most distinctive/relevant words. Brand placement is a judgment call: lead with the brand only for branded/navigational queries; otherwise put it at the end. Google frequently rewrites titles (using H1s, anchor text, etc.) — chasing a rigid pattern is wasted effort if Google replaces it. Check the live SERP to see what Google actually displays before "fixing" a title.

Practical title/description guidance:

• Front-load the term users actually search; avoid boilerplate prefixes/suffixes that get truncated or stripped.
• Watch truncation: titles render ~50-60 chars (pixel-based, not a hard char count); descriptions ~120-160 chars. Test in the Rich Results Test or a SERP-preview tool rather than counting characters.
• Make meta descriptions genuinely descriptive of the page — Google rewrites ~60%+ of them, so treat them as a suggested snippet, not a guaranteed one.
• Don't mass-rewrite titles to a template across a whole site; you risk replacing well-performing ones. Change titles where data shows a problem.

Test changes:

1. Identify pages with CTR below benchmark and no AI Overview eating the clicks (see §3).
2. Rewrite title + description for one cohort.
3. Track CTR change over 2-4 weeks in GSC (compare to the same pages' prior period, not the sitewide average).

Weekly Audit Checklist

• Check index coverage for new errors
• Review performance trends (7d vs previous 7d)
• Monitor Core Web Vitals for regressions
• Check sitemap processing status
• Review manual actions (should always be empty)
• Check security issues
• Flag pages losing >20% impressions week-over-week