{ }Free
robots.txt generator
Generate crawl rules and optional sitemap URL.
Generate crawl rules and optional sitemap URL. Write one user-agent group with disallowed paths and an optional absolute sitemap URL. Each supplied path becomes its own Disallow line for copying into a robots.txt file.
How to use
- Enter your text, values, or image.
- Adjust the options and run the tool.
- Copy or download your result.
Worked example
Example input
User agent: * Disallowed paths, one per line: /admin/ /private/ Sitemap URL: https://example.com/sitemap.xml
Example result
User-agent: * Disallow: /admin/ Disallow: /private/ Sitemap: https://example.com/sitemap.xml
Limitations and details
Robots rules request crawler behavior and do not protect private data or reliably remove indexed pages; no file is deployed automatically.
Inputs and options
- User agent
- Text
- Disallowed paths, one per line
- Text
- Sitemap URL
- Text
Frequently asked questions
Is this tool free and private?
Yes. All 300 tools are free in this phase. Processing happens in your browser; your input and files are not sent to our servers.