Robots.txt Generator
Create custom robots.txt files to control search engine crawling and indexing — 100% private, no server uploads.
Settings
Rules 0
Output
Frequently Asked Questions
Do wildcards and dollar anchors work?
Where do I list sitemaps?
What format is the output?
Are there any usage limits?
Does it work offline?
Which browsers are supported?
How to Use
Add Allow and Disallow Rules
Type paths per user-agent with wildcard and dollar-anchor support.
Set Sitemaps and Delays
Attach sitemap URLs and crawl-delay so polite bots pace themselves.
Copy robots.txt
Copy the file content to your site root and verify with Search Console.
How to Calculate
Rules group by user-agent with Allow and Disallow paths, where the longest matching rule wins, followed by a Sitemap line. The output validates directive spelling and warns on broad blocks like Disallow: /.
About the Robots Txt Generator
The Robots.txt Generator writes crawler rules with per-agent allow and disallow lines, wildcard and anchor support, sitemap directives and live syntax validation.
Preview the file, catch errors before upload and verify in Search Console after deploying to the site root. Fully offline in your browser with no account. Keep admin and staging paths disallowed in every environment.
Related Tools
Methodology & feedback
Figures run locally in your browser from the values you enter (page updated ). Estimates are educational only — verify finance, tax and health decisions with a qualified professional (see Disclaimer). Spotted an error? Report it via contact and we correct with a new update date.