Bot Monitor
Two views of AI crawler activity: what your robots.txt allows, and which bots actually visit your site — measured by a lightweight tracker.
robots.txt analysis
Bot Monitor reads your live robots.txt and reports, per bot, whether it's allowed or blocked: GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Google-Extended (Gemini), and BingBot. If key bots are blocked, you get a copyable robots.txt fix snippet.
Blocking bots doesn't affect Upranke's data. Upranke measures your AI visibility through direct API calls to the engines, not by crawling through these bots — so your robots.txt choices change how AI systems see your site, never how accurately Upranke measures you.
Real visit telemetry
The robots.txt view tells you what's allowed; the tracker tells you what actually happens. Install the lightweight tracking snippet (Settings → AI Crawler Tracking, with step-by-step Cloudflare Worker instructions) and Upranke records real crawler visits — ChatGPT, Claude, Perplexity, Gemini, and Bing — with totals over 24 hours, 7 days, 30 days, and all time.
The tracking state is explicit: inactive (tracker not installed), waiting (installed, no visits recorded yet), or active (visits flowing). Detailed per-visit records are kept for 7 days, then rolled into daily statistics — so long-range totals stay accurate without storing every hit forever.
Reading the two together
- Blocked + no visits — unblock first; nothing else matters while crawlers can't get in.
- Allowed + no visits — access isn't your problem; content and authority are. AI has no reason to fetch you yet.
- Allowed + regular visits — AI systems are actively reading you; now the question is whether that turns into citations, which is what Visibility Tracking measures.