FREE TOOLAI Crawler Checker. No sign-up.No sign-up.

Which AI crawlers can reach your site?

Enter a domain or page. The checker reads robots.txt, tells you bot by bot which AI crawlers are allowed, and shows what they can read without JavaScript.

✓ Free, no sign-up✓ Results in seconds✓ A fix for every problem

30 free checks a day. We read the public page and do not store it.

What it checks

What it checks.

AI Crawler Checker

18 crawlers: Google, Bing, ChatGPT, Claude, Perplexity, Apple, DuckDuckGo, Mistral and training-only bots
Whether robots.txt loads, and what an error there does to Google
What a crawler gets without JavaScript: title, description, headings, links and how much text
noindex and nosnippet on the page itself

Quick answers

Frequently asked questions.

Should I block AI crawlers?

Block training-only bots such as GPTBot, ClaudeBot or Google-Extended if you do not want your content used to train models; that does not stop AI search from citing you. Blocking search and answer bots such as OAI-SearchBot or PerplexityBot keeps you out of those assistants' answers.

Does Google-Extended affect my Google rankings?

No. Google says Google-Extended only controls use of your content for training Gemini models. AI Overviews use Googlebot, so blocking Google-Extended does not remove you from them.

Why does a bot show as allowed when my firewall blocks it?

The checker reads robots.txt, which is a request bots choose to follow. Blocks set at a CDN or firewall, such as Cloudflare's AI bot setting, do not appear in robots.txt.

Fix the gaps with Realword.

Article Studio writes researched articles that open with a direct answer and link every statistic to its source, with tables, an FAQ and schema included. Content Ideas finds the topics your site is missing.

Start free See Article Studio

You start with 500 free credits and no card.