Free bounded technical site crawl

Audit the site—not just the homepage.

What the crawl records

Eight evidence layers with explicit boundaries.

The crawl combines bounded public source requests, robots and sitemap parsing, an internal-link graph, cross-page aggregation, and optional Google performance evidence. Every result names what was observed; no pass claims that Google indexed, selected, or ranked a page.

01

Coverage and provenance

The report states URLs discovered, pages attempted, pages analyzed, links checked, crawl limits, elapsed time, warnings, and the exact stop reason.

02

Responses and redirects

Every fetched URL records its final HTTP status, content type, bytes read, duration, and redirect path under bounded response limits.

03

Indexability

The crawler records robots.txt policy, robots meta directives, X-Robots-Tag headers, sitemap conflicts, and eligibility for every analyzed page.

04

Canonical and metadata

Each page records resolved canonicals, title, description, headings, exact duplicates, language, and conflicts across the crawled set.

05

Internal-link graph

The audit counts internal inlinks and outlinks, checks additional destinations, identifies broken and redirecting links, and labels potential orphans as sample-based review items.

06

Sitemap reconciliation

Robots-declared and conventional sitemap locations are parsed, sitemap indexes are expanded within limits, and sitemap-only versus crawl-only URLs are reported.

07

Schema, images, and tags

JSON-LD syntax and declared types, missing image alt attributes, common analytics patterns, and common advertising patterns are recorded as source observations.

08

Field and lab performance

Public CrUX 75th-percentile values are shown when available, while a single mobile Lighthouse run and its findings remain clearly labeled as lab evidence.

What it does not prove

Technical eligibility is not search visibility.

This evidence report does not report keyword positions, Search Console impressions, local map-pack placement, backlinks, review strength, entity consistency, or whether an AI system cited the site. Those require different evidence and often authenticated first-party data.

It answers bounded technical questions: which public URLs were discovered, what each response and source declared, how crawled pages link together, where sitemap and indexability signals conflict, and which cross-page patterns need correction. It does not claim that Google rendered, indexed, selected, or ranked the reported URLs.

Compare against real Los Angeles sites

One audit result is more useful with a public baseline.

Spec collected a frozen 100-site Los Angeles service-business sample across ten categories. Its internal aggregate QA checks passed with 97 technical observations and a deterministic 20-site manual review. Unknown evidence remains outside every denominator. External editorial outreach is on hold until the selection ledger and independent manual-review gaps are closed.

83.2%

Declared a canonical

79 of 95 known observations.

82.1%

Exposed an H1

78 of 95 known observations.

45.3%

Used recognized local schema

43 of 95 known observations.

100%

Allowed OAI-SearchBot in robots

96 of 96 known robots checks; access still does not guarantee citation.

Technical SEO evidence questions.

Does this audit show my Google rankings?

No. It does not query a rank-tracking database or Search Console. Ranking evidence should come from Search Console, a documented rank tracker, or a reproducible search observation.

Is this a complete crawl of every website?

Only when every same-origin candidate discovered within the configured depth fits inside the 25-page limit. Larger sites receive a bounded sample, and the report states that it is partial, how many URLs were discovered, and why the crawl stopped.

Does the crawl render JavaScript?

No. It inspects fetched source HTML and labels that limitation in the report. Client-rendered metadata, links, directives, or schema require a separate rendered-DOM comparison.

Does this audit show what ChatGPT or an AI Overview says about my business?

No. The tool checks technical eligibility signals such as indexability, structured data, and OAI-SearchBot access. It does not ask an assistant to generate an answer and does not claim citation or answer visibility.

What does OAI-SearchBot access mean?

OpenAI documents OAI-SearchBot as the crawler used to surface websites in ChatGPT search. Allowing it is an eligibility control, not a promise that a page will be crawled, selected, cited, or ranked.

Do passing technical checks mean the page will rank?

No. Technical checks establish whether the page is accessible and declares key signals. Rankings also depend on intent satisfaction, content quality, internal and external links, entity evidence, local signals, competition, historical performance, and whether Google indexes the intended canonical page.

Why can a check show 'Not verified'?

A site can time out, block the audit request, return an unexpected response, or omit a conventional sitemap location. The tool labels that state unknown instead of turning a failed verification into a false pass or failure.

Can I audit a URL with query parameters?

No. Submitted query strings are rejected because preview links, sessions, and signed parameters can contain private values. Enter the public canonical page or site origin instead; discovered tracking and query variants are excluded from the HTML page queue.

Does Spec receive the URL I enter?

The public URL, without query parameters, is sent to Spec's same-origin audit endpoint so the server can inspect robots first, crawl a bounded set of permitted same-origin pages, parse sitemap documents, check internal destinations, and request optional PageSpeed evidence. Email is requested only after the audit and only if you choose to receive the report.

Found the technical gate. Now fix the commercial system.

Bring the results to a discovery call. Spec will separate what the automated scan proved, what still needs first-party evidence, and the order in which the work should ship.