ARC · OPEN BENCHMARK
Open agent leaderboard
How far do AI agents get toward checkout? Every agent path ARC measures, ranked by cart success on the same fixed test set: ARC-100 v1, 100 real stores across 11 categories, frozen 7 Oct 2026.
Free, neutral and open (CC-BY-4.0). ARC takes no payment from any model, protocol or platform company. Anyone can run their agent on ARC-100 and submit the run.
JSON → · CSV → · ARC-100 test set (CSV) · (JSON) · Submission spec → · Protocol Matrix →
Cart success on ARC-100 v1
Cart success = the path put the test product in a cart, or got a checkout link back, and stopped before payment. Ranked by rate; rows with fewer than 10 results are not ranked. Live from saved results, refreshed hourly.
| # | Agent path | Cart success | 95% range | n | Not counted | Results dated (UTC) |
|---|---|---|---|---|---|---|
| 1 | ARC browser shopper D MeasuredReal browser session on the store's product page (task-oriented checkout). Legacy id openai-dots. | 55.6%50 of 90 | 45.3–65.4% | 90 | 1 run broke9 no result yet | 5 Oct 2026 to 8 Oct 2026 |
| 2 | ARC browser shopper C MeasuredReal browser session on the store's product page (fast, decides from the page). Legacy id grok. | 52.2%47 of 90 | 42–62.2% | 90 | 1 run broke9 no result yet | 5 Oct 2026 to 8 Oct 2026 |
| 3 | UCP path (e.g. Google AI Mode / Gemini) MeasuredARC's HTTP client reads /.well-known/ucp, searches the catalog and calls create_cart on the store's UCP endpoint. Stores with no UCP endpoint count as no cart for this path. | 52%52 of 100 | 42.3–61.5% | 100 | 0 | 6 Oct 2026 to 11 Oct 2026 |
| 4 | ARC browser shopper B MeasuredReal browser session on the store's product page (price and preference). Legacy id instinct. | 51.1%46 of 90 | 41–61.2% | 90 | 1 run broke9 no result yet | 5 Oct 2026 to 8 Oct 2026 |
| 5 | ARC browser shopper A MeasuredReal browser session on the store's product page (social and visual). Legacy id muse. | 50.6%45 of 89 | 40.4–60.7% | 89 | 2 run broke9 no result yet | 5 Oct 2026 to 8 Oct 2026 |
| 6 | ARC browser shopper E MeasuredReal browser session on the store's product page (scripted crawler, weak on js-heavy ui). Legacy id openclaw. | 40.7%37 of 91 | 31.1–50.9% | 91 | 9 no result yet | 5 Oct 2026 to 8 Oct 2026 |
| — | Storefront MCP path (/api/mcp) MeasuredARC's HTTP client lists tools on the store's /api/mcp server and calls a cart tool when one is listed.None of the 100 measured ARC-100 stores lists a cart tool on /api/mcp, so this path cannot add to cart on them today. Shopify moved cart tools to UCP, which the UCP row counts. Not ranked. | —0 of 100 list a cart tool | — | 100 | 0 | 6 Oct 2026 to 11 Oct 2026 |
Measured: ARC observed it directly. Ranges are Wilson 95% confidence intervals; overlapping ranges mean the gap may be chance. Rows can have different n until every ARC-100 store has a result on every path. 9 ARC-100 stores have no published browser result yet. Stores outside the Index are waiting in ARC's capped weekly scan queue, so browser n will grow over the coming days.
Real-agent spot checks
Rows for named consumer agents appear here only when ARC has real saved runs of that agent on ARC-100. There are none yet, so this table is empty on purpose. ARC does not estimate or stand in for them.
Submitted runs unverified
Outside agent builders can submit runs against ARC-100 (spec). Submissions are self-reported and stay unverified until ARC re-runs a sample of the claimed stores. They are never mixed into the ranked table above.
No submissions yet. Be the first: POST /api/leaderboard/submit or the MCP tool submit_leaderboard_run at /api/ask.
The test set: ARC-100 v1
100 real stores picked from ARC's Index and open directory, balanced across categories, markets and platforms, then frozen. The set does not change inside a version; a new version gets a new name. CSV · JSON · How the stores were picked
Category
- Beauty10
- Apparel9
- Footwear9
- Supplements & wellness9
- Home9
- Food & drink9
- Outdoor9
- Jewelry9
- Kids9
- Pets9
- Electronics & accessories9
Market (from domain)
- Generic domain (.com etc.)67
- GB14
- IE9
- AU4
- DE3
- FR1
- EU1
- DK1
Platform
- Shopify63
- Not detected33
- Salesforce Commerce Cloud2
- Adobe Commerce / Magento1
- WooCommerce1
Market comes from the store's domain only (.co.uk = GB); a generic domain like .com says nothing about where a store sells. Platform is the homepage fingerprint when measured, Shopify when the store answers a Shopify agent endpoint (inferred), else not detected.
Per-store results (100 rows)
| ID | Store | Category | Market | A | B | C | D | E | UCP | MCP |
|---|---|---|---|---|---|---|---|---|---|---|
| 001 | apc.fr | Apparel | FR | … | … | … | … | … | ✗ | – |
| 002 | chubbiesshorts.com | Apparel | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 003 | crewclothing.co.uk | Apparel | GB | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 004 | everlane.com | Apparel | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 005 | girlfriend.com | Apparel | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | – |
| 006 | gymshark.com | Apparel | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | – |
| 007 | patagonia.com | Apparel | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 008 | phase-eight.com | Apparel | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 009 | vuoriclothing.com | Apparel | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 010 | allbirds.com | Footwear | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 011 | athleticpropulsionlabs.com | Footwear | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | – |
| 012 | birdies.com | Footwear | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | – |
| 013 | cheaney.co.uk | Footwear | GB | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 014 | nisolo.com | Footwear | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 015 | office.co.uk | Footwear | GB | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 016 | rmwilliams.com | Footwear | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 017 | rothys.com | Footwear | INTL | … | … | … | … | … | ✓ | – |
| 018 | thursdayboots.com | Footwear | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 019 | alyaskin.com.au | Beauty | AU | ✗ | ✗ | ✗ | ✗ | ✓ | ✓ | – |
| 020 | aureliacellular.co.uk | Beauty | GB | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | – |
| 021 | byredo.com | Beauty | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 022 | drunkelephant.com | Beauty | INTL | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | – |
| 023 | ellaandjo.ie | Beauty | IE | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 024 | functionofbeauty.com | Beauty | INTL | … | … | … | … | … | ✓ | – |
| 025 | rarebeauty.com | Beauty | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 026 | rhodeskin.com | Beauty | INTL | … | … | … | … | … | ✓ | – |
| 027 | saiehello.com | Beauty | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 028 | theordinary.com | Beauty | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | – |
| 029 | bulknutrients.com.au | Supplements & wellness | AU | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 030 | drinklmnt.com | Supplements & wellness | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 031 | edubily.de | Supplements & wellness | DE | · | ✗ | ✗ | ✗ | ✓ | ✓ | – |
| 032 | gruns.co | Supplements & wellness | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 033 | huel.com | Supplements & wellness | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 034 | mudwtr.com | Supplements & wellness | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | – |
| 035 | naturesbest.co.uk | Supplements & wellness | GB | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | – |
| 036 | seed.com | Supplements & wellness | INTL | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | – |
| 037 | us.foursigmatic.com | Supplements & wellness | INTL | ✗ | ✗ | · | · | ✗ | ✓ | – |
| 038 | awaytravel.com | Home | INTL | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | – |
| 039 | brooklinen.com | Home | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | – |
| 040 | buffy.co | Home | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 041 | casper.com | Home | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | – |
| 042 | dunelm.com | Home | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | – |
| 043 | floydhome.com | Home | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 044 | lights4fun.co.uk | Home | GB | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 045 | liveouter.com | Home | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 046 | mocka.com.au | Home | AU | … | … | … | … | … | ✓ | – |
| 047 | chamberlaincoffee.com | Food & drink | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | – |
| 048 | daily-harvest.com | Food & drink | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 049 | dovesfarm.co.uk | Food & drink | GB | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | – |
| 050 | drinktrade.com | Food & drink | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 051 | hukitchen.com | Food & drink | INTL | … | … | … | … | … | ✓ | – |
| 052 | imbibe.ie | Food & drink | IE | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | – |
| 053 | magicspoon.com | Food & drink | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 054 | palaisdesthes.com | Food & drink | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 055 | partakefoods.com | Food & drink | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | – |
| 056 | 53degreesnorth.ie | Outdoor | IE | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 057 | bergans.com | Outdoor | INTL | … | … | … | … | … | ✗ | – |
| 058 | berghaus.com | Outdoor | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 059 | cotswoldoutdoor.com | Outdoor | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 060 | durstongear.com | Outdoor | INTL | ✗ | ✗ | ✗ | ✗ | ✓ | ✓ | – |
| 061 | greatoutdoors.ie | Outdoor | IE | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 062 | hardloop.co.uk | Outdoor | GB | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 063 | nordisk.eu | Outdoor | EU | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | – |
| 064 | paramo-clothing.com | Outdoor | INTL | … | … | … | … | … | ✗ | – |
| 065 | angara.com | Jewelry | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 066 | dowerandhall.com | Jewelry | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | – |
| 067 | enamelcopenhagen.dk | Jewelry | DK | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 068 | georgjensen.com | Jewelry | INTL | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | – |
| 069 | jared.com | Jewelry | INTL | · | · | ✗ | ✓ | ✗ | ✗ | – |
| 070 | mimco.com.au | Jewelry | AU | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 071 | momuse.ie | Jewelry | IE | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | – |
| 072 | queensmith.co.uk | Jewelry | GB | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 073 | vitalydesign.com | Jewelry | INTL | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | – |
| 074 | bigjigstoys.co.uk | Kids | GB | … | … | … | … | … | ✓ | – |
| 075 | bobandblossom.co.uk | Kids | GB | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | – |
| 076 | bugaboo.com | Kids | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 077 | caramel-shop.co.uk | Kids | GB | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ | – |
| 078 | childrensplace.com | Kids | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 079 | cybex-online.com | Kids | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 080 | nameit.com | Kids | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 081 | naturebaby.com | Kids | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | – |
| 082 | rachelriley.co.uk | Kids | GB | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | – |
| 083 | edgardcooper.com | Pets | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 084 | fressnapf.de | Pets | DE | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | – |
| 085 | k9ballistics.com | Pets | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 086 | littlebeast.co | Pets | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 087 | maxizoo.ie | Pets | IE | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 088 | miacara.com | Pets | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 089 | nonstopdogwear.com | Pets | INTL | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | – |
| 090 | petworld.ie | Pets | IE | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ | – |
| 091 | poochandmutt.co.uk | Pets | GB | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | – |
| 092 | alternate.de | Electronics & accessories | DE | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 093 | audeze.com | Electronics & accessories | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 094 | beyerdynamic.com | Electronics & accessories | INTL | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | – |
| 095 | did.ie | Electronics & accessories | IE | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | – |
| 096 | gradolabs.com | Electronics & accessories | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 097 | ifi-audio.com | Electronics & accessories | INTL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| 098 | jlab.com | Electronics & accessories | INTL | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | – |
| 099 | marshall.com | Electronics & accessories | INTL | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
| 100 | powercity.ie | Electronics & accessories | IE | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – |
✓ cart · ✗ no cart · – no cart tool on this path · · ARC's run broke (not counted) · … queued. A–E are ARC browser shoppers A–E.
Method
Cart success means the path put the test product in a cart, or got a checkout link back, and stopped before payment. No order is placed and nothing is paid.
n is the number of ARC-100 stores where that path has a result. Stores where ARC's own run broke (not the store's fault) are left out of n and listed as not run.
Ranges are Wilson 95% confidence intervals. When two ranges overlap, the gap between those rows may be chance.
Paths are measured on different stores until every ARC-100 store has a result for every path, so compare rows with their n in mind.
ARC browser shoppers A to E are ARC personas run on ARC-chosen models. They are not the consumer agents their legacy ids once named.
The UCP and storefront MCP rows are ARC's own HTTP client calling each store's public agent endpoints. They show what the protocol path allows, not how a particular consumer agent behaves.
Browser results come from ARC's existing weekly Index scans, which run under a fixed weekly spend cap; ARC-100 stores outside the Index use the same capped queue. UCP and MCP results come from ARC's free HTTP checks. Method arc-method-2026.10: /methodology · changelog. Agents can read this through the free MCP tool agent_leaderboard at /api/ask.
Cite this
ARC Agent Leaderboard (ARC-100 v1), arcreport.ai/leaderboard, accessed 2026-10-11. Licensed CC-BY-4.0.
Raw rows: /data. Citation file: CITATION.cff. Corrections: /contact.