robots.txt Generator

Generate a robots.txt file for search engine crawlers

Sitemap URL

Disallow Paths (one per line)

Allow Paths (one per line)

User-Agent

Crawl Delay (seconds)

Result

robots.txt Content

User-agent: *
Disallow:

Rules

0 rules defined

Crawl Status

Open (all allowed)

Generate a sitemap.xml file

Generate HTML meta tags

Build canonical link tags