# ARC Report — Methodology

ARC Report scans 1,015 e-commerce brands daily (02:00 UTC) with an HTTP-only scanner (~25 requests per brand, no browser automation).

## What is tested

1. robots.txt parsing — explicit Allow/Disallow rules for 9 agents: GPTBot, ChatGPT-User, ClaudeBot, Claude-Web, PerplexityBot, Google-Extended, Amazonbot, Bingbot, CCBot.
2. Live HTTP access tests — requests with each agent's User-Agent string against the homepage and a product page; detects WAF blocks (403s, challenge pages) and content stripping (<25% of a Chrome baseline).
3. Structured data — JSON-LD, Schema.org Product, Open Graph, sitemap.xml, product feeds.
4. Protocol files — llms.txt (size, link count), agents.txt variants, UCP endpoints.
5. Infrastructure — e-commerce platform, CDN, and WAF fingerprinting.

## Two-scan confirmation rule

robots.txt rule changes publish immediately (text-file diffs). Inference-based signals (HTTP verdicts, CDN/WAF, structured data) must appear in two consecutive scans before publishing. Timeouts and rate-limit responses are never published as changes.

## ARC Score v1.0

0–100 = agent access breadth (50) + structured data quality (25) + protocol files (15) + scan stability (10).
- Agent access: mean over agents of allowed/no_rule=1, inconclusive=0.5, restricted=0.25, blocked=0, × 50.
- Structured data: JSON-LD 7, Schema.org Product 7, Open Graph 4, sitemap 4, product feed 3.
- Protocol files: llms.txt 6 (+3 if it contains links), agents.txt 3, UCP 3.
- Scan stability: share of conclusive per-agent verdicts × 10.
Score changes only with a versioned methodology update.

## Known limitations

- robots.txt is a policy declaration; enforcement may differ (we test both, and label them separately).
- Structured-data detection reads server HTML; JS-rendered markup can be missed.
- WAF behaviour can vary by region and time; the two-scan rule reduces, but cannot eliminate, flicker.
- UA-string tests approximate agent traffic; they do not replicate full agent behaviour (no JS execution).
---

Source: https://www.arcreport.ai/methodology
Last updated: 2026-06-20T06:31:59.173Z
License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) — Data: ARC Report (www.arcreport.ai), licensed CC BY 4.0