A public record for the agentic web

AI agents are becoming real commerce traffic, and the question of who lets them in — and who locks them out — is being decided quietly, one robots.txt line and one WAF rule at a time. Nobody was writing it down. ARC Report exists to be that record.

Every day at 02:00 UTC we scan 1,015 e-commerce brands for 9 AI agents: robots.txt policy, live HTTP access tests, structured data, platform and WAF detection, and protocol files like llms.txt. The results are published in full — browsable pages, bulk downloads, a public API, and an MCP server — under CC BY 4.0.

Independent

ARC Report is not affiliated with any AI lab, e-commerce platform, or bot-management vendor. No brand pays to be listed, removed, or re-scored. The quiet Pro tier (history depth and rate limits) funds the infrastructure; the dataset itself stays free.

Daily

Web policy is volatile. A snapshot from last quarter is trivia; a daily series is evidence. Every page shows its last-scanned timestamp, and the changelog records each confirmed change with before/after values.

Verified

We separate what sites declare (robots.txt) from what they enforce (WAF behaviour), require inferred changes to appear in two consecutive scans before publishing, and run a public corrections log and dispute process. The full method is on /methodology — anyone can reproduce a scan with curl.

Who builds it

ARC Report is built and run by Andy Bryn, an independent developer. It started as a tool to answer one question — "can an AI agent actually buy from this store?" — and turned into the dataset this site publishes. Questions, corrections, research collaborations: hello@arcreport.ai.