How to get recommended by Claude
Claude searches the web when a question needs it and cites its sources. For it to cite you, its crawlers have to be able to get in and your text has to be in the HTML.
How it decides who to recommend
Anthropic splits its crawling in three: Claude-SearchBot indexes the web for Claude's searches, Claude-User visits when someone asks Claude to look at a page, and ClaudeBot collects text for training.
Blocking ClaudeBot takes you out of training, not out of answers. To appear in what Claude searches and cites, Claude-SearchBot and Claude-User are the ones that count.
The crawlers that need to get in
If robots.txt or your firewall turns them away, they can't read you. The free analysis checks each one.
What to do, step by step
Let Claude-SearchBot and Claude-User in
It's the first thing and the one that fails most: a robots.txt copied from another site, a security plugin or a firewall that blocks "unusual bots" shut Claude out without warning. Check that robots.txt doesn't name them with Disallow and that your server doesn't answer them with a 403.
Put your text in the HTML
AI crawlers almost never run JavaScript. If your site paints its content in the browser (very common with Lovable, Bolt and other AI-built sites), Claude receives a blank page. Whatever isn't in the HTML your server sends doesn't exist for it.
Say clearly who you are and what you offer
A title and description that say what you do and where, a clear H1 and structured data (Organization or LocalBusiness, Product, FAQPage). AI doesn't guess: if you don't say it, it doesn't know.
Answer what your customer asks
Claude recommends whoever best answers a specific question: "best bike shop in Valencia", "how much does… cost". Prices, area, opening hours, comparisons and FAQs as text, not in images or PDFs.
Write to be quoted
Claude quotes specific sentences. Paragraphs that make sense on their own, facts with their source, headings that say what's underneath: what can be copied straight into an answer is what ends up cited.
Frequently asked questions
If I block ClaudeBot, do I disappear from Claude?
No. ClaudeBot is the training crawler. Searches go through Claude-SearchBot and pages someone asks for, through Claude-User.
Does Claude respect robots.txt?
Anthropic says its crawlers respect it, and that each one can be allowed or blocked separately.
How do I know if Claude cites me?
Ask it what a customer would ask, without naming your brand. With Tarsier's Watch we do it for you every week and let you know if it changes.