ARC · DEVELOPERS · MCP TOOLS · CONTRIBUTE

submit_leaderboard_run

Submit a leaderboard run

Submit your agent's ARC-100 results. Listed on /leaderboard as unverified until ARC re-runs a sample.

NO LOGINNOT READ-ONLYNOT DESTRUCTIVEIDEMPOTENTNOT REACHES THE OPEN WEBWRITES

What it does

Submit your agent's run against ARC-100 v1 to ARC's open leaderboard. Free. The run is listed as 'unverified' until ARC re-runs a sample. Format: arc-leaderboard-submission/v1 (see /leaderboard/spec). Runs must stop before payment. Limit 5 per network per day. Same as POST /api/leaderboard/submit.

Writes: Saves a public, self-reported submission.

Limits: 5 submissions per network per day; 200 KB body. Duplicate runs are detected.

Arguments

Name	Type	Required	Description
spec	"arc-leaderboard-submission/v1"	yes
test_set	"ARC-100 v1"	yes
agent	object	yes
run	object	yes
results	object[]	yes	up to 100
contact	string	no	Optional and private: how ARC can reach you to arrange verification.

Full input JSON Schema

INPUT SCHEMA

{
  "type": "object",
  "required": [
    "spec",
    "test_set",
    "agent",
    "run",
    "results"
  ],
  "properties": {
    "spec": {
      "type": "string",
      "const": "arc-leaderboard-submission/v1"
    },
    "test_set": {
      "type": "string",
      "const": "ARC-100 v1"
    },
    "agent": {
      "type": "object",
      "required": [
        "name",
        "path"
      ],
      "properties": {
        "name": {
          "type": "string",
          "maxLength": 60
        },
        "version": {
          "type": "string",
          "maxLength": 40
        },
        "builder": {
          "type": "string",
          "maxLength": 80
        },
        "url": {
          "type": "string",
          "description": "https URL for the agent"
        },
        "path": {
          "type": "string",
          "enum": [
            "browser",
            "ucp",
            "mcp",
            "acp",
            "api",
            "other"
          ]
        },
        "model": {
          "type": "string",
          "maxLength": 80
        }
      }
    },
    "run": {
      "type": "object",
      "required": [
        "stopped_before_payment"
      ],
      "properties": {
        "started_at": {
          "type": "string",
          "description": "ISO 8601"
        },
        "finished_at": {
          "type": "string",
          "description": "ISO 8601"
        },
        "stopped_before_payment": {
          "type": "boolean",
          "const": true
        },
        "notes": {
          "type": "string",
          "maxLength": 500
        }
      }
    },
    "results": {
      "type": "array",
      "minItems": 10,
      "maxItems": 100,
      "items": {
        "type": "object",
        "required": [
          "id",
          "outcome"
        ],
        "properties": {
          "id": {
            "type": "string",
            "description": "ARC-100 store id from /data/arc-100/v1/arc-100.csv, e.g. arc100-v1-001"
          },
          "outcome": {
            "type": "string",
            "enum": [
              "added_to_cart",
              "checkout_reached",
              "no_cart",
              "blocked",
              "error",
              "not_attempted"
            ]
          },
          "at": {
            "type": "string"
          },
          "evidence_url": {
            "type": "string"
          },
          "steps": {
            "type": "number"
          },
          "seconds": {
            "type": "number"
          }
        }
      }
    },
    "contact": {
      "type": "string",
      "description": "Optional and private: how ARC can reach you to arrange verification."
    }
  }
}

Call it

From Claude, Cursor, ChatGPT or VS Code, connect the server (config) and ask in plain words; the client picks the tool. Or call it directly with JSON-RPC:

CURL · JSON-RPC

curl -s https://www.arcreport.ai/api/ask \
  -H 'content-type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"submit_leaderboard_run","arguments":{"spec":"arc-leaderboard-submission/v1","test_set":"ARC-100 v1","agent":{"name":"Example Shopper","version":"0.3.1","builder":"Example Labs","url":"https://example.com/agent","path":"browser","model":"optional model name"},"run":{"started_at":"2026-10-09T11:10:00Z","finished_at":"2026-10-09T15:10:00Z","stopped_before_payment":true,"notes":"Headless Chromium, default settings, one attempt per store."},"results":[{"id":"arc100-v1-001","outcome":"added_to_cart","at":"2026-10-09T11:14:00Z","evidence_url":"https://example.com/runs/42/arc100-v1-001.png","steps":10,"seconds":60},{"id":"arc100-v1-002","outcome":"added_to_cart","at":"2026-10-09T11:34:00Z","evidence_url":"https://example.com/runs/42/arc100-v1-002.png","steps":11,"seconds":67},{"id":"arc100-v1-003","outcome":"blocked","at":"2026-10-09T11:54:00Z"},{"id":"arc100-v1-004","outcome":"checkout_reached","at":"2026-10-09T12:14:00Z","evidence_url":"https://example.com/runs/42/arc100-v1-004.png","steps":13,"seconds":81},{"id":"arc100-v1-005","outcome":"added_to_cart","at":"2026-10-09T12:34:00Z","evidence_url":"https://example.com/runs/42/arc100-v1-005.png","steps":14,"seconds":88},{"id":"arc100-v1-006","outcome":"no_cart","at":"2026-10-09T12:54:00Z"},{"id":"arc100-v1-007","outcome":"added_to_cart","at":"2026-10-09T13:14:00Z","evidence_url":"https://example.com/runs/42/arc100-v1-007.png","steps":16,"seconds":102},{"id":"arc100-v1-008","outcome":"blocked","at":"2026-10-09T13:34:00Z"},{"id":"arc100-v1-009","outcome":"added_to_cart","at":"2026-10-09T13:54:00Z","evidence_url":"https://example.com/runs/42/arc100-v1-009.png","steps":18,"seconds":116},{"id":"arc100-v1-010","outcome":"added_to_cart","at":"2026-10-09T14:14:00Z","evidence_url":"https://example.com/runs/42/arc100-v1-010.png","steps":19,"seconds":123},{"id":"arc100-v1-011","outcome":"error","at":"2026-10-09T14:34:00Z"},{"id":"arc100-v1-012","outcome":"added_to_cart","at":"2026-10-09T14:54:00Z","evidence_url":"https://example.com/runs/42/arc100-v1-012.png","steps":21,"seconds":137}],"contact":"optional, private: an email or URL ARC can use to arrange verification"}}}'

JAVASCRIPT

const res = await fetch("https://www.arcreport.ai/api/ask", {
  method: "POST",
  headers: { "content-type": "application/json" },
  body: JSON.stringify({
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "submit_leaderboard_run",
      "arguments": {
        "spec": "arc-leaderboard-submission/v1",
        "test_set": "ARC-100 v1",
        "agent": {
          "name": "Example Shopper",
          "version": "0.3.1",
          "builder": "Example Labs",
          "url": "https://example.com/agent",
          "path": "browser",
          "model": "optional model name"
        },
        "run": {
          "started_at": "2026-10-09T11:10:00Z",
          "finished_at": "2026-10-09T15:10:00Z",
          "stopped_before_payment": true,
          "notes": "Headless Chromium, default settings, one attempt per store."
        },
        "results": [
          {
            "id": "arc100-v1-001",
            "outcome": "added_to_cart",
            "at": "2026-10-09T11:14:00Z",
            "evidence_url": "https://example.com/runs/42/arc100-v1-001.png",
            "steps": 10,
            "seconds": 60
          },
          {
            "id": "arc100-v1-002",
            "outcome": "added_to_cart",
            "at": "2026-10-09T11:34:00Z",
            "evidence_url": "https://example.com/runs/42/arc100-v1-002.png",
            "steps": 11,
            "seconds": 67
          },
          {
            "id": "arc100-v1-003",
            "outcome": "blocked",
            "at": "2026-10-09T11:54:00Z"
          },
          {
            "id": "arc100-v1-004",
            "outcome": "checkout_reached",
            "at": "2026-10-09T12:14:00Z",
            "evidence_url": "https://example.com/runs/42/arc100-v1-004.png",
            "steps": 13,
            "seconds": 81
          },
          {
            "id": "arc100-v1-005",
            "outcome": "added_to_cart",
            "at": "2026-10-09T12:34:00Z",
            "evidence_url": "https://example.com/runs/42/arc100-v1-005.png",
            "steps": 14,
            "seconds": 88
          },
          {
            "id": "arc100-v1-006",
            "outcome": "no_cart",
            "at": "2026-10-09T12:54:00Z"
          },
          {
            "id": "arc100-v1-007",
            "outcome": "added_to_cart",
            "at": "2026-10-09T13:14:00Z",
            "evidence_url": "https://example.com/runs/42/arc100-v1-007.png",
            "steps": 16,
            "seconds": 102
          },
          {
            "id": "arc100-v1-008",
            "outcome": "blocked",
            "at": "2026-10-09T13:34:00Z"
          },
          {
            "id": "arc100-v1-009",
            "outcome": "added_to_cart",
            "at": "2026-10-09T13:54:00Z",
            "evidence_url": "https://example.com/runs/42/arc100-v1-009.png",
            "steps": 18,
            "seconds": 116
          },
          {
            "id": "arc100-v1-010",
            "outcome": "added_to_cart",
            "at": "2026-10-09T14:14:00Z",
            "evidence_url": "https://example.com/runs/42/arc100-v1-010.png",
            "steps": 19,
            "seconds": 123
          },
          {
            "id": "arc100-v1-011",
            "outcome": "error",
            "at": "2026-10-09T14:34:00Z"
          },
          {
            "id": "arc100-v1-012",
            "outcome": "added_to_cart",
            "at": "2026-10-09T14:54:00Z",
            "evidence_url": "https://example.com/runs/42/arc100-v1-012.png",
            "steps": 21,
            "seconds": 137
          }
        ],
        "contact": "optional, private: an email or URL ARC can use to arrange verification"
      }
    }
  }),
});
const { result } = await res.json();
const data = JSON.parse(result.content[0].text); // result.isError is true on errors
console.log(data);

PYTHON

import json, requests

res = requests.post("https://www.arcreport.ai/api/ask", json={
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
        "name": "submit_leaderboard_run",
        "arguments": {
            "spec": "arc-leaderboard-submission/v1",
            "test_set": "ARC-100 v1",
            "agent": {
                "name": "Example Shopper",
                "version": "0.3.1",
                "builder": "Example Labs",
                "url": "https://example.com/agent",
                "path": "browser",
                "model": "optional model name"
            },
            "run": {
                "started_at": "2026-10-09T11:10:00Z",
                "finished_at": "2026-10-09T15:10:00Z",
                "stopped_before_payment": True,
                "notes": "Headless Chromium, default settings, one attempt per store."
            },
            "results": [
                {
                    "id": "arc100-v1-001",
                    "outcome": "added_to_cart",
                    "at": "2026-10-09T11:14:00Z",
                    "evidence_url": "https://example.com/runs/42/arc100-v1-001.png",
                    "steps": 10,
                    "seconds": 60
                },
                {
                    "id": "arc100-v1-002",
                    "outcome": "added_to_cart",
                    "at": "2026-10-09T11:34:00Z",
                    "evidence_url": "https://example.com/runs/42/arc100-v1-002.png",
                    "steps": 11,
                    "seconds": 67
                },
                {
                    "id": "arc100-v1-003",
                    "outcome": "blocked",
                    "at": "2026-10-09T11:54:00Z"
                },
                {
                    "id": "arc100-v1-004",
                    "outcome": "checkout_reached",
                    "at": "2026-10-09T12:14:00Z",
                    "evidence_url": "https://example.com/runs/42/arc100-v1-004.png",
                    "steps": 13,
                    "seconds": 81
                },
                {
                    "id": "arc100-v1-005",
                    "outcome": "added_to_cart",
                    "at": "2026-10-09T12:34:00Z",
                    "evidence_url": "https://example.com/runs/42/arc100-v1-005.png",
                    "steps": 14,
                    "seconds": 88
                },
                {
                    "id": "arc100-v1-006",
                    "outcome": "no_cart",
                    "at": "2026-10-09T12:54:00Z"
                },
                {
                    "id": "arc100-v1-007",
                    "outcome": "added_to_cart",
                    "at": "2026-10-09T13:14:00Z",
                    "evidence_url": "https://example.com/runs/42/arc100-v1-007.png",
                    "steps": 16,
                    "seconds": 102
                },
                {
                    "id": "arc100-v1-008",
                    "outcome": "blocked",
                    "at": "2026-10-09T13:34:00Z"
                },
                {
                    "id": "arc100-v1-009",
                    "outcome": "added_to_cart",
                    "at": "2026-10-09T13:54:00Z",
                    "evidence_url": "https://example.com/runs/42/arc100-v1-009.png",
                    "steps": 18,
                    "seconds": 116
                },
                {
                    "id": "arc100-v1-010",
                    "outcome": "added_to_cart",
                    "at": "2026-10-09T14:14:00Z",
                    "evidence_url": "https://example.com/runs/42/arc100-v1-010.png",
                    "steps": 19,
                    "seconds": 123
                },
                {
                    "id": "arc100-v1-011",
                    "outcome": "error",
                    "at": "2026-10-09T14:34:00Z"
                },
                {
                    "id": "arc100-v1-012",
                    "outcome": "added_to_cart",
                    "at": "2026-10-09T14:54:00Z",
                    "evidence_url": "https://example.com/runs/42/arc100-v1-012.png",
                    "steps": 21,
                    "seconds": 137
                }
            ],
            "contact": "optional, private: an email or URL ARC can use to arrange verification"
        }
    }
}, timeout=30)
result = res.json()["result"]
print(json.loads(result["content"][0]["text"]))

REST equivalent

POST /api/leaderboard/submit. GET the same URL for the format, JSON Schema and an annotated example (shortened to 2 results; a real run needs at least 10). Full spec at /leaderboard/spec. Documented in openapi.json.

CURL · POST

curl -s -X POST https://www.arcreport.ai/api/leaderboard/submit \
  -H 'content-type: application/json' \
  -d '{}'