Loading...
Loading...
Paste your server access log and see exactly which AI bots crawl you — GPTBot, ClaudeBot, PerplexityBot, CCBot and 20+ more. Runs 100% in your browser; nothing is uploaded.
Your logs never leave your browser — parsing runs 100% client-side.
No upload, no server call, no storage. Files are read in-browser with FileReader and discarded when you close the tab.
Common or combined log format (Apache / Nginx). 20+ AI crawlers detected.
Paste your server access log above, or load the sample to see how it works.
We'll show which AI bots crawled you, how often, and exactly which pages they read.
Export a slice of your Apache or Nginx access log (e.g. /var/log/nginx/access.log). A few days to a few weeks of traffic gives the clearest picture of AI crawl activity.
Paste the lines into the box, or click "Upload .log / .txt" to read a file in-browser. Nothing is uploaded — parsing runs entirely on your machine. No log handy? Hit "Load sample log".
See total AI-bot hits, the % of traffic that is AI, every distinct crawler with its vendor and category, a crawl-over-time chart, and the top 10 pages AI bots read.
Missing GPTBot or ClaudeBot? Your content is not being discovered for AI citations. Export the CSV, then fix crawlability with robots.txt, llms.txt, and an AI-bot policy on your CDN.
It matches 20+ known AI crawler user-agents: GPTBot, ChatGPT-User, and OAI-SearchBot (OpenAI); ClaudeBot, anthropic-ai, and Claude-Web (Anthropic); PerplexityBot and Perplexity-User (Perplexity); Google-Extended and GoogleOther (Google); Applebot-Extended (Apple); meta-externalagent and FacebookBot (Meta); Bytespider (ByteDance); CCBot (Common Crawl); Amazonbot (Amazon); cohere-ai (Cohere); YouBot (You.com); Diffbot; ImagesiftBot; and Timpibot. Each bot is tagged as training, search, or both, so you can tell apart crawlers that feed model training from ones that power live AI-search citations.
No. Parsing runs 100% client-side in your browser. When you paste text or upload a .log/.txt file, the file is read in-browser with the FileReader API and analyzed in JavaScript on your own machine — there is no server call, no storage, and nothing leaves your computer. Close the tab and the data is gone. That makes it safe to run on production access logs that may contain visitor IP addresses.
The common and combined log formats produced by Apache and Nginx by default — lines like `203.0.113.7 - - [08/Oct/2026:03:11:42 +0000] "GET /page HTTP/1.1" 200 5312 "-" "user-agent"`. The analyzer extracts the user-agent from the last quoted field, the request path from the request quote, and the date from the bracketed timestamp. Junk or non-matching lines are counted but skipped, so a messy log will not break the report.
AI assistants can only cite pages they have crawled. If GPTBot, ClaudeBot, and PerplexityBot are not in your logs, you are invisible inside ChatGPT, Claude, and Perplexity answers — even if you rank #1 in Google. Seeing the crawl pattern tells you whether AI engines have discovered your content, which pages they prioritise, and whether an aggressive crawler like Bytespider is hammering your origin. It is the ground-truth complement to robots.txt and llms.txt: those declare your policy; the logs show what bots actually did.
The daily bar chart only renders when the analyzer can parse dates from your log lines. It reads the bracketed timestamp in `[10/Oct/2026:03:11:42 +0000]` format and maps the three-letter month to a number. If your log uses a non-standard date format or strips timestamps, the panel hides itself gracefully and the rest of the report (per-bot table, top paths, summary) still works.
Yes. The per-bot table has an Export CSV button that builds the file in-browser via a Blob and downloads it directly — bot name, vendor, category, hit count, and share of AI traffic, one row per crawler. Like everything else in the tool, the CSV is generated on your machine and never touches a server.
Reading the logs is diagnosis. We do the fix: AI-bot allow / block rules at the edge (Cloudflare, Fastly, Vercel), rate limits on aggressive crawlers like Bytespider, plus the robots.txt, llms.txt, and schema plumbing that gets you cited in ChatGPT and Perplexity. The same playbook we run on our in-house products PenLeap and TalkDrill.
Configure my CDNIf AI crawlers are missing from your logs, our SEO team makes your content discoverable and citable — technical SEO, llms.txt curation, schema markup, and the structured-data work that wins both Google and AI-search engines.
Talk to our SEO teamWe audit your logs, configure AI-bot rules at the edge, and ship the robots.txt + llms.txt + schema markup that gets your pages cited in ChatGPT, Claude, and Perplexity answers.