Method changelog

Current method version: arc-method-2026.10. Every row in the open dataset carries it. Changes to how ARC measures are logged here so a method change is never reported as a trend.

Machine-readable: JSON.

2026-10-08 · arc-method-2026.10 · Checkout WebMCP
Free rails checks now inspect Shopify checkout HTML using GET-only cart permalinks with an isolated cookie jar and available variants. Shop Pay redirects are read for the same-store ur_back_url only. Checkout markers and get_checkout / update_checkout / complete_checkout names are saved separately from the storefront adapter in evidence JSON. No JavaScript, checkout tools, buyer data or payments are executed. Tokens and query strings are never saved.
Effect: Shopify-only rates separate measured no, unknown failures and not checked. Completed stores missing this field join the existing daily backfill, never-checked first. HTML declarations do not establish a completed purchase; buyer approval is required.
2026-10-08 · arc-method-2026.10 · Sample
The curated directory grew from 3,014 to 3,314 stores: 300 stores from the 7 Oct validated reserve, re-checked over plain HTTP on 8 Oct (live HTTPS or bot protection, brand on page, store signals; obvious mismatches dropped). The pick was weighted toward everyday buys (grocery, pantry and drinks in Food & drink; household essentials and kitchenware in Home; drugstores) and toward stores not on Shopify: 121 Food & drink, 72 Home, 107 across other categories; 205 not detected as Shopify (146 behind bot protection), 95 Shopify.
Effect: Food & drink and Home rates will shift as these stores get their first checks (about 80 per hour). Bot-protected stores are recorded as blocked or unmeasured, never as no. No model calls or new spend.
2026-10-08 · arc-method-2026.10 · Market labels
The 12 Index stores outside the curated directory now carry the same curated market label (11 US or Canada brands and Princess Polly, Australia), so the research pages no longer show them as Not labelled.
Effect: Labels only; no measurement changes. Market is a curated label, not a measurement.
2026-10-07 · arc-method-2026.10 · Research pages
Added /insights: agent readiness by commerce platform, by market and by category, and return and shipping policy clarity. Each page is generated hourly from the same representative checks as the adoption tracker (latest completed check per listed host; policies and product data from the latest attempt), shows n and its data-as-of time, gives every metric its own measured denominator, hides percentages below 10 checked stores, and offers JSON and CSV downloads under CC-BY 4.0. The platform table on the adoption tracker and the research pages now states its coverage: platform measured for X of Y checked stores.
Effect: New views of existing saved checks; no new requests, model calls or spend. Platform rates describe only stores with measured platform evidence; blocked homepages and checks saved before platform detection are excluded and counted in the coverage line.
2026-10-07 · arc-method-2026.10 · Dataset versions
Dataset snapshots are now immutable. The first export on a date is published as /data/YYYY-MM-DD/, any later export that day as YYYY-MM-DD.r2, .r3 and so on, and the exporter refuses to overwrite a published version. Each manifest records snapshot_version, revision, generated_at (UTC) and the version it supersedes; /data/versions.json lists them all. latest/ is a moving copy of the newest version.
Effect: Before this fix the 2026-10-07 folder was silently regenerated four times on the day (rails_checks 607, then 647, then 1,044 rows). All five published versions are restored byte-for-byte from git as 2026-10-07 (r1, 08:08 UTC) to 2026-10-07.r5 (19:17 UTC); latest equals r5. Cite a version folder, not latest/.
2026-10-07 · arc-method-2026.10 · Sample
The curated directory grew from 1,383 to 3,014 stores (1,631 new). Every new host answered over HTTPS (or with a bot-protection 403/429) when added, and was dropped if parked, closed, password-protected, showing no store signals, not naming the brand, or redirecting to a different brand; brand redirects to the brand's own new domain use the final host. New stores were balanced to about 275 per category and across markets (US, UK, Ireland, rest of Europe, Canada, Australia), and every validated store on a detected non-Shopify platform (Salesforce Commerce Cloud, Adobe Commerce/Magento, BigCommerce, WooCommerce and others) was included. Each store now carries an ARC-curated market label: the storefront's country (country-code domain or country subdomain), otherwise where the brand is based.
Effect: Rates will shift as the new stores receive their first checks (about 80 per hour, all within about a day); compare like with like using the dated insights pages and rails_checks. Market is a curated label, not a measurement. No model calls or new spend: rails checks are plain HTTP.
2026-10-07 · arc-method-2026.10 · Open agent leaderboard
Added ARC-100 v1, a frozen public test set of 100 real stores (11 categories, 9 per category plus one in Beauty; balanced for market from the domain and platform; Index stores first, then directory stores with a measured product page). Added /leaderboard: cart success per agent path (ARC browser shoppers A to E, UCP path, storefront MCP path) over ARC-100, with n, dates, evidence labels and Wilson 95% intervals. ARC-100 stores outside the Index join the existing capped weekly Index scan queue, only in slots the Index does not need. Outside builders can submit runs (arc-leaderboard-submission/v1); they stay unverified until ARC re-runs a sample.
Effect: New view of existing saved results plus extra scans under the same weekly spend cap. Index scores, ranks and the dataset's Index rows are unchanged: benchmark-only scans are kept out of Index pages.
2026-10-07 · arc-method-2026.10 · Commerce platforms
Deterministic platform fingerprints use existing homepage headers and HTML, with Shopify storefront catalog tools as corroboration. Blocked homepages are unavailable; Not detected includes headless/custom stores. Adoption now groups measured checks by platform with separate metric denominators.
Effect: New evidence only; no extra requests, model calls, verdict or score changes. ARC’s directory sample is not the whole web.
2026-10-07 · arc-method-2026.10 · Protocol Matrix: MCP
MCP Usable now counts any tool or catalog answering on /api/mcp (on Shopify today, the policy and FAQ tool), the same definition as the adoption tracker's storefront MCP signal. MCP Transacts is measured only for stores whose /api/mcp lists a cart tool, and is Not measured while none do, because Shopify moved cart and catalog tools to UCP (/api/ucp/mcp), where they are counted.
Effect: MCP Usable moved from 0 to the number of stores whose MCP server answers; MCP Transacts moved from 0 Measured to Not measured.
2026-10-07 · arc-method-2026.10 · One host set
The directory, adoption tracker, Protocol Matrix and dataset count the same stores: the curated directory, Index stores and accepted submissions, each represented by its latest completed check (www-insensitive). Index stores are now listed in the directory. Test and rejected hosts are excluded.
Effect: Directory and tracker WebMCP and checkout counts now match; n drops by the excluded test and rejected hosts.
2026-10-07 · arc-method-2026.10 · Robots agent access
A refused robots.txt request (HTTP 401, 403, 407 or 429, usually a bot firewall answering ARC) is no longer counted as 'no robots.txt'; it is unmeasured. The HTTP status is now saved. Earlier no-robots rows without a status are not measured until rechecked (daily backfill).
Effect: The 'no robots.txt' count fell: 8 of the earlier 10 were firewall 403s.
2026-10-07 · arc-method-2026.10 · Index labels
The all-shoppers result label 'Finished the purchase' is now 'Added to cart (stopped before payment)' in the UI, API and stores.csv. ARC never pays; the label now says so.
Effect: Wording only; scores unchanged.
2026-10-07 · arc-method-2026.10 · Store hubs
The Agent Reachability Matrix rows are ARC browser shoppers A to E plus the UCP path, not named consumer agents. A press-reported block of a named agent appears in its own row with the source.
Effect: Labels only; cells unchanged.
2026-10-07 · arc-method-2026.10 · Robots agent access
Free rails checks now parse robots.txt for the shared answer, search and training agent roster, on the homepage and tested product path. Saved evidence is available on store cards, JSON and MCP.
Effect: Adoption rates use only measured evidence in the latest completed check per host. HTTP 4xx means no robots.txt; network and 5xx failures are unavailable. Older rows are not measured and receive a daily backfill. Robots rules do not establish actual fetch access.
2026-10-07 · arc-method-2026.10 · Brand claims
Store cards can be claimed free by proving domain control using DNS TXT or an HTTPS file. Verified brands may publish a labelled statement and contact URL, and prioritise a free directory re-check.
Effect: Claims never change measured scores, ranks, verdicts or dataset measurements.
2026-10-07 · arc-method-2026.10 · Sample
Anyone can add stores (single check, up to 50 per bulk submission, API or MCP). New hosts are validated (live HTTPS or bot protection, not parked or closed, no redirect to another brand) and join the directory with source=submitted.
Effect: The sample grows beyond ARC's curated list. Rates can shift as submitted stores are checked; filter rails_checks by source to compare like with like.
2026-10-07 · arc-method-2026.10 · Index personas
The five browser cart shoppers are labelled ARC browser shoppers A to E in the UI, API and dataset. The old ids (muse, instinct, grok, openai-dots, openclaw) remain only as a documented alias field.
Effect: Labels only. No scores change. Dataset columns agent_* are now persona_a to persona_e.
2026-10-07 · arc-method-2026.10 · Dataset: adoption_history
Weekly Index metrics now use the latest completed scan per store in the week and only published Index stores, the same definition as stores.csv. Rails metrics now count each host once per week (latest attempt), not every check.
Effect: W41 index metrics now match stores.csv. Earlier snapshots counted the first scan per store per week, and rails hosts_checked counted repeated checks.
2026-10-07 · arc-method-2026.10 · WebMCP
Rails checks saved before WebMCP detection existed (before 18:00 UTC on 6 Oct 2026) are treated as not measured for WebMCP instead of 'no'.
Effect: The jump from 39.8% (6 Oct) to 54% (7 Oct) on the adoption tracker was a detector start, not rollout. All 54 recorded 'no to yes' flips paired a pre-detector check with a post-detector check; no post-detector pair changed.
2026-10-07 · arc-method-2026.10 · Store hubs
Read-back headlines are model-neutral: 'One AI model's read-back named X instead', with the model disclosed in the note.
Effect: Wording only.
2026-10-07 · arc-method-2026.10 · Protocol Matrix · change #66
Added /protocols: Declared, Valid, Usable, Transacts per protocol with Measured / Inferred / Not measured labels.
Effect: New view of existing saved checks.
2026-10-07 · arc-method-2026.10 · Policies and product data · change #61
Added return and shipping policy, support contact and product structured-data checks to rails checks.
Effect: New measured fields; older rows show not measured.
2026-10-06 · arc-method-2026.10 · WebMCP · change #41
Added Shopify WebMCP adapter detection to HTTP rails checks (server HTML markers).
Effect: WebMCP is measured only from this point.
2026-10-06 · arc-method-2026.10 · Rails checks · change #33
First live HTTP check of store agent rails: storefront MCP, UCP profile, UCP MCP, catalog tool and cart link.
Effect: Start of rails history.

Cite: ARC method arc-method-2026.10, arcreport.ai/methodology/changelog. CC-BY-4.0. Citation file: CITATION.cff.