AI Crawler Activity
What this measures
A fetch, not a citation. This counts requests that arrived carrying the user-agent of a known AI crawler or retrieval bot. It says nothing about whether an assistant then quoted the page in an answer — that is a different instrument with its own page, the engine benchmark.
How it is collected
This page was built around a Cloudflare Pages middleware tap that would sit in front of every request and matches the User-Agent against a list of known AI bots — training crawlers and retrieval bots both. Matches are logged to Workers Analytics Engine, and a daily cron rolls twenty-four hours into the file this page renders. The agents the site explicitly welcomes are named in robots.txt, which is the shorter and more easily checked half of the same story.
Why publish it
If you sell to anyone an AI assistant might recommend, which engines fetch your pages is worth knowing. Citation engineering is one of the lanes this firm sells, so the measurement runs on our own domain first — including, as now, when it has nothing to show. An instrument you only see when its numbers flatter you is not an instrument.
Open data. /assets/data/crawler-public.json — CC0-1.0, no auth, no rate limit. The schema is the file’s own keys, and the file is short enough to read in full.