SEO Tools

Robots.txt Tester

Paste a robots.txt file, choose a user-agent, and enter a URL path to test whether it's allowed or disallowed.

Allowed

Matched: allow /private/public-page.html

Follows the longest-matching-rule precedence used by Google (and most crawlers): the most specific matching Allow or Disallow pattern wins, with Allow winning ties.

How to use this tool

  1. 1Paste your robots.txt content.
  2. 2Enter the user-agent to test (e.g. * or Googlebot).
  3. 3Enter the URL path to test.
  4. 4See whether it's allowed or disallowed and which rule matched.

Frequently asked questions

What happens when Allow and Disallow rules conflict?+

The most specific (longest) matching rule wins; if both rules are the same length, Allow takes precedence — this matches Google's documented robots.txt matching behavior.

Related tools