Skip to content
RHRAZU

Resources

Free tools to help you get found.

The guides, checklists and templates I use with clients, plus the free tools I rely on every day.

SEO expert or developer? Jump to 7 free tools: SERP preview, schema and hreflang generators, redirect maps and AI crawler controls.

For SEO experts, developers & technical SEOs

The free SEO & AI visibility toolkit

Free, in-browser tools for the jobs SEOs do every week: control AI crawlers, write schema, preview snippets, set up hreflang and map redirects for migrations. Nothing is sent anywhere, no sign-up.

  • GPTBot

    OpenAI · Model training

  • OAI-SearchBot

    OpenAI · AI search index

  • ChatGPT-User

    OpenAI · Live user fetch

  • ClaudeBot

    Anthropic · Model training

  • Claude-SearchBot

    Anthropic · AI search index

  • Claude-User

    Anthropic · Live user fetch

  • PerplexityBot

    Perplexity · AI search index

  • Perplexity-User

    Perplexity · Live user fetch

  • Google-Extended

    Google (Gemini) · Model training

  • Applebot-Extended

    Apple · Model training

  • DuckAssistBot

    DuckDuckGo · AI search index

  • MistralAI-User

    Mistral · Live user fetch

  • Amazonbot

    Amazon · AI search index

  • meta-externalagent

    Meta · Model training

  • CCBot

    Common Crawl · Model training

  • Bytespider

    ByteDance · Model training

robots.txt
# AI crawlers, generated at rhrazu.com/resources

User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: DuckAssistBot
User-agent: MistralAI-User
User-agent: Amazonbot
Allow: /

User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: meta-externalagent
User-agent: CCBot
User-agent: Bytespider
Disallow: /

# Everyone else (incl. Googlebot and Bingbot)
User-agent: *
Allow: /

Sitemap: https://www.example.com/sitemap.xml

Tip: blocking training bots doesn't stop you appearing in AI answers. The search and user bots are what fetch your pages when someone asks ChatGPT, Claude or Perplexity a question.

Now booking new projects

Want me to apply these for you?

Skip the learning curve. I'll audit your visibility on Google and in AI answers.

99% Client Satisfaction Replies within 1 business day
Rakibul Hasan Razu
20K → 200K+monthly organic traffic