seometatools

Robots.txt & Meta Tags Generator

Build crawler directives and social sharing tags with a visual form. Output is syntax-highlighted and ready to copy or download.

# robots.txt generated by seometatools
User-agent: *
Disallow: /admin/
Allow: /
 
Sitemap: https://example.com/sitemap.xml

How to use

  1. 1

    Pick a crawler

    Use “*” to apply rules to every bot, or target one such as Googlebot.

  2. 2

    Add Allow / Disallow rules

    Example: Disallow “/cart/” and “/search?” to save crawl budget on low-value pages.

  3. 3

    Add your sitemap

    Including the Sitemap line helps all search engines discover your URLs.

  4. 4

    Generate social tags

    Switch tabs, fill in title, description and a 1200×630 image, then paste the output into your page's <head>.

Key features

  • Visual robots.txt rule builder
  • Common user-agent presets
  • Open Graph and Twitter Card tags with HTML escaping
  • Syntax-highlighted output with copy and download

Limitations & best practices

  • Robots.txt is advisory — malicious bots may ignore it
  • Disallow does not guarantee de-indexing; use noindex instead
  • Googlebot ignores Crawl-delay
  • Test your file in Google Search Console after uploading

Frequently asked questions