More and more of your potential visitors never see a results page: they ask ChatGPT, Claude, or Perplexity, and the assistant answers with what it can crawl and cite. If your robots.txt blocks the AI crawlers, deliberately or by accident, you're invisible to that entire channel. This tool reads your robots.txt and tells you exactly who can see you.
Step 1: open yoursite.com/robots.txt in another tab and copy everything you see. Step 2: paste it here. (Browsers don't let this page fetch other sites directly, so the copy-paste is on you.)
Why this matters now
Search referrals are flattening while assistant citations grow, and the assistants can only cite what their crawlers can read. Blocking GPTBot or ClaudeBot was a popular defiant gesture in 2023; in 2026 it mostly means your competitor's page gets cited instead of yours. There's a legitimate philosophical case for blocking training crawlers, and this tool won't argue you out of it, but make it a decision, not an accident inherited from an old robots.txt.
A few nuances the table glosses over: Google-Extended controls whether Google may use your content for its AI models, not whether you appear in regular Search (that's Googlebot). The "-User" agents (ChatGPT-User, Claude-User, Perplexity-User) fetch your page live when a human asks about it, which is about as close to a warm referral as the AI era offers. And robots.txt is a request, not a lock; it governs the polite crawlers, which the major ones are.
While you're tuning for machines, make sure the humans get something worth citing.
The useful website is making a comeback