Tools from the research.

Instruments I built for my own audit and tracking work. The crawl checker is free for anyone: no signup, no email gate. The tracker is the private instrument behind client retainers.

Free, live now

Common Crawl Visibility Checker

Most AI training data starts at Common Crawl. This checker reads Common Crawl's own files and shows how it sees any domain: how many pages each monthly crawl captured, when an AI-crawler block first appeared in robots.txt and who wrote it, what CCBot actually archived vs the live page, whether a firewall challenges bots, sitemap coverage, and the domain's crawl priority in the web graph.

Seven checks, one input, plain-language verdicts, and copy-paste fixes for anything it finds.

Open the checker

Checks: Captures per crawl, robots.txt block dating, block attribution, stored-copy drift, live CCBot probe, sitemap coverage, crawl priority
Data: Common Crawl static archives and the full 118M-domain web graph, refreshed monthly
Cost: Free, no signup, no email
Export: JSON and sitemap-gap CSV

Private, for tracking clients

AI Visibility Tracker

The instrument behind my tracking retainers. Every night it asks ChatGPT and Google's AI Overview a client's real buyer questions, from the countries they sell in, and records every source the AI cites. Day by day it charts whether the brand is in the answer, at what rank, who else gets cited, and who is buying the sponsored slots around those queries.

Each client gets a private dashboard: citation trends, source presence day by day, competitor share of voice, and a placement-target list their PR team can act on directly.

Client login

Tracks: ChatGPT and Google AI Overview citations, collected nightly, every cited source logged
Coverage: Per query, per surface, per country. AI answers change by market, so each is tracked separately
Reports: Citation rank trends, competitor share, sponsored slots, placement-target CSV
Access: Private. Included with the tracking retainer, client login by email

More tools ship as the research produces them. Each one automates a check I run in paid audits: the methodology is published in the research, the tool just saves you the manual work.

Want the full picture?

The checker covers one layer: Common Crawl. The full audit scores seven dimensions across your site, your bot logs, and 30+ live citation queries against competitors.

See the audit