AI Crawler Log Analyzer
100% client-side, paste Common / Combined log lines, instantly see AI bot activity
Access Log
What this tool does
Parses Apache/Nginx Common or Combined access log lines in your browser (no data sent to server) and classifies every hit as: AI training crawler, AI search crawler, classic search engine bot, social preview bot, or likely human. Produces hit counts, top paths, and insights.
Who it's for: Site operators and SEO pros who want to know which AI crawlers are actually hitting their site and how often.
When to use
- •Measure GPTBot / ClaudeBot / PerplexityBot visits over the last week.
- •Detect which AI training crawlers respect vs ignore your robots.txt block.
- •Compare classic search bot traffic vs AI crawler traffic.
- •Identify high-traffic paths that AI bots favor for context.
How it works
- 1Paste log lines (Common or Combined format) into the textarea.
- 2Browser-side parser matches User-Agents against a 35-bot registry.
- 3Hit counts, top paths, and heuristic insights render instantly.
- 4Nothing is uploaded - analysis stays in your browser.
Tips
- •Use tail -n 10000 access.log to paste a recent sample.
- •AI bot hits rising while human traffic stays flat often signals AI summarization cannibalization.
- •Combine with AI Bot Access Check to confirm whether blocks are working.