Trains and answers
GrokBot: what it is and how to allow or block it
The same name decides two things: whether they train on your text and whether they use you to answer. Blocking it closes both doors at once.
What it is
Grok (xAI). For Grok, xAI's AI built into X: it trains their models and answers with what it reads. Note: part of its traffic arrives as a regular browser and ignores robots.txt.
Like most AI crawlers, it reads the HTML exactly as your server sends it: whatever your site paints with JavaScript may never reach it.
How it identifies itself
Mozilla/5.0 (compatible; GrokBot/1.0; +https://x.ai)
This is what it sends in the User-Agent header. In robots.txt you only use the short name: GrokBot.
The lines for your robots.txt
Allow it
User-agent: GrokBot Allow: /
Block it
User-agent: GrokBot Disallow: /