🗺️ Robots.txt & Sitemap Generator

Draft a valid robots.txt and XML sitemap from your own URL list. Add paths or full URLs, set crawl rules and download the result. This is a local generator, not a crawler: it never visits or uploads your site.

Canonical site URL

Sitemap changefreq

Sitemap priority

Pages — one path or full URL per line

Blank lines and lines starting # are ignored; duplicates are removed.

Disallow paths — one per line

Use robots directives only for crawl preference — they do not secure a private page.

Last modified Crawl-delay (optional) Add Allow: /

Generated file

Important: robots.txt is not access control

A disallow rule asks compliant crawlers not to fetch a URL. It cannot protect confidential content and it is not a guaranteed way to remove an already-indexed page. Protect private routes with authentication or remove them from the public web; use a sitemap only for canonical, indexable URLs.