FREE TOOLRobots.txt Tester. No sign-up.No sign-up.

Test your robots.txt rules

Enter a URL and test it against the live robots.txt, or paste new rules to try them before you publish.

✓ Free, no sign-up✓ Results in seconds✓ A fix for every problem

30 free tests a day. Nothing is stored.

What it checks

What it checks.

Robots.txt Tester

Allowed or blocked for 18 search, AI and training crawlers, plus any user agent you enter
The exact line that matches each crawler
Lines crawlers ignore, such as unknown fields or rules before any User-agent
Whether robots.txt loads and lists a sitemap

Quick answers

Frequently asked questions.

Which rule wins when several match?

Google uses the most specific rule, the one with the longest path. When an Allow and a Disallow are equally specific, Allow wins. The tester applies the same logic.

Can I test rules before publishing them?

Yes. Paste the new robots.txt into the box and the URL is tested against your pasted rules instead of the live file.

Does blocking a page in robots.txt remove it from Google?

No. A blocked URL can still be indexed from links, without its content. To keep a page out of search, allow crawling and use a noindex tag.

Fix the gaps with Realword.

Article Studio writes researched articles that open with a direct answer and link every statistic to its source, with tables, an FAQ and schema included. Content Ideas finds the topics your site is missing.

Start free See Article Studio

You start with 500 free credits and no card.