Robots.txt Generator
Build and export crawler rules in a clean, ready-to-use format.
Free SEO Tool
Settings
Configure crawler access
User-agent
All bots (*)
Googlebot
Bingbot
GPTBot
CCBot
Custom bot
Custom bot name
Allow paths
Enter one path per line.
Disallow paths
Enter one path per line.
Sitemap URL
Crawl-delay
Host directive
Block all crawling
Adds Disallow: /
Allow full crawling
Used when no blocked paths exist.
WordPress preset
Add common WordPress rules.
Add file comment
Add a short generator header.
Generate Robots.txt
Load Sample
Clear
Generated Output
Preview before download
Copy
Download robots.txt
Waiting for settings
0
User-agent Groups
0
Allow Rules
0
Disallow Rules
0
Total Lines