SEO Tools
Robots.txt Generator
Build a robots.txt file by adding Allow/Disallow rules, an optional crawl-delay, and your sitemap URL — copy the result straight into your site's root.
Rules
How to use this tool
- 1Add Allow or Disallow rules for the paths you want to control.
- 2Enter your sitemap URL and an optional crawl-delay.
- 3Copy the generated robots.txt into your site's root directory.
Frequently asked questions
Where does robots.txt need to go?+
At the root of your domain, e.g. https://example.com/robots.txt — it won't be respected anywhere else.
Does robots.txt guarantee pages won't be indexed?+
No — it only asks well-behaved crawlers not to fetch a path. Pages can still be indexed if linked from elsewhere; use a noindex meta tag to actually prevent indexing.