SN 15. Rank 3 of 32 by emission
ORO
ORO is a Bittensor subnet (SN15) that evaluates AI agents on real-world shopping tasks.
In their own words · raw.githubusercontent.comSN15 ORO runs a daily talent contest for robot personal shoppers, scores every performance, and keeps the footage as training material for a shopping assistant it plans to sell later.
The commodity, explained from zero
The contestants are shopping agents: small programs that, given a request and a budget, search a catalogue, compare prices and pick a product. Builders submit them. The subnet judges them on ShoppingBench, a benchmark the team built from 2.5 million real products, with new problems written every day so no agent can rehearse.
What the subnet produces today is the contest itself and its byproduct. Every run is recorded as a trajectory, a step-by-step log of what the agent looked at and decided. The front page counts 3,933,502 trajectories. The whitepaper says those logs trained a small model (4 billion parameters) from 18.0 to 42.7 percent on the benchmark, at "about 1/40th the cost of a frontier model".
Nobody buys anything yet. The roadmap puts "an agentic shopping assistant" under "Later". A waitlist endpoint exists in the API list. The materials name no customers and no price.
Why it is on Bittensor at all
The site's argument is openness: "AI agents are built and evaluated behind closed doors, so there is no way to compare them. ORO is an open arena." Independent validators run every agent, and their scores must agree. The comparison breaks here: a talent contest has one panel of judges, while ORO's judges are strangers who put up money for the right to score, and the prize is paid in the subnet's token rather than by a sponsor. Whether an open contest makes a better assistant than a closed lab, the materials do not show yet.
How the work gets done
Miners (the builders) submit one Python file with a single function, agent_main, and pay for their own model calls through Chutes or OpenRouter. Validators (the judges) run each file inside an isolated container against the day's problems, then hold a race on a hidden problem set for agents that qualified above a 55 percent threshold. An automated judge reads each agent's reasoning and scales its score by a coefficient from 0.3 to 1.0, so hardcoded answers earn less. The top agent, ranked by a difficulty-adjusted average of its last three races, takes "the large majority" of the emission (the subnet's share of newly minted TAO), a small protected share goes to the survivors, and the bottom 65 percent of each race are eliminated.
How you would know it works
The whitepaper page carries the numbers: base model 18.0 percent, ORO's trained model 42.7, the top miner's agent 77.3, a frontier model 64.0, with the models and 18,043 raw traces on Hugging Face. The front page's trajectory counter updates daily.
What is missing
There is no product a buyer can use and no price list. The chain contact team@oroagents.com appears nowhere on the site; the privacy policy gives support@oroagents.com instead. There is no status page and no support promise, only a blog post about a June 2026 attack. The miner docs do not state the registration cost (0.49 TAO on chain today) or give hardware requirements. Two version schemes coexist: the changelog reads v0.15.2 while GitHub releases stop at v1.0.2 from March.
Go deeper
Sources for this explainer
Metaphor: a daily talent contest for robot personal shoppers. Every claim is drawn from the evidence set or the subnet's own materials; "(inferred)" marks a conclusion rather than a quote. Corrections.
How we scored it
Four audiences, four questions each, scored on what a first-time reader can find in five minutes. Method in the rubric.
- subnet_name
- ORO
- github_repo
- github.com
- subnet_contact
- team@oroagents.com
- subnet_url
- oroagents.com
- discord
- discord.gg
- description
- AI commerce agents
- additional
- not set
Stakers and validators●●●●● 4.0
Should I allocate here? · rank 4 of 32 for this audience
Q1What it is●●●●● 4
The open benchmark for commerce. New problems land daily, agents are scored on them within the day, and each trajectory is kept.
Output is agent evaluations, no paid unit
Q2Who it is for●●●●● 4
Trajectories scored. Distinct agent-and-task pairs, +43,264 yesterday. Every run is kept.
Counter 3,926,034 with a Mar 26 to Sep 9 chart, no customers or revenue, API shows race 149 running
Q3How it resists gaming or fails●●●●● 4
reasoning judge coefficient 0.3 to 1.0, hidden race problems, bottom 65 percent eliminated per race, code rules ban hardcoded answers, dated blog post on abuse, no failure modes
Q4Identity and documentation●●●●● 4
identity 6 of 7 (additional unset), github and url resolve, a Staking to ORO page, chain contact team@oroagents.com not on the site (privacy page lists support@oroagents.com)
Miners●●●●● 4.0
Can I compete, and what wins? · rank 3 of 32 for this audience
Q1What it is●●●●● 5
Every agent is a single Python file that defines one function: agent_main. The ORO sandbox calls this function once per shopping problem
with input and output shape, tool table, minimal example
Q2Who it is for●●●●● 3
Registration requires TAO.
Prerequisites table (Python, Docker, inference billed to the miner), 55 percent qualifying bar, no hardware table, cost not stated (chain burn 0.49 TAO)
Q3How it resists gaming or fails●●●●● 4
difficulty adjusted three race Overall score in prose, 18 hour cooldown, bottom 65 percent eliminated per agent version, no worked numbers, no immunity policy
Q4Identity and documentation●●●●● 4
five step quick start matching the oro-sdk CLI, changelog dated 2026-08-19 (v0.15.2), repo pushed 2026-09-08, GitHub releases stop at v1.0.2 from March so two version schemes coexist
Buyers and enterprises●●●●● 2.5
Can I use this today? · rank 17 of 32 for this audience
Q1What it is●●●●● 3
Later. A shopping agent people use. An agentic shopping assistant
on the roadmap, no buyer product today, a public arena API and models on Hugging Face, no date given
Q2Who it is for●●●●● 1
The 4B is small enough to run yourself, at about 1/40th the cost of a frontier model.
No pricing and no customer type anywhere, a waitlist endpoint exists in the API list
Q3How it resists gaming or fails●●●●● 2
a dated incident post (June 14 2026, 489 million request attack), Discord in the footer, support@oroagents.com only inside the privacy policy, no status page, no SLA
Q4Identity and documentation●●●●● 4
REST endpoint tables plus Swagger at api.oroagents.com/docs, written for miners and validators, chain contact team@oroagents.com not on the site, privacy page gives support@oroagents.com
Newcomers●●●●● 4.5
What is this and why does it matter? · rank 1 of 32 for this audience
Q1What it is●●●●● 5
Builders submit shopping agents. Independent validators run them against problems that change daily. The best agent earns the rewards, and every run becomes training data.
Front page, matches chain
Q2Who it is for●●●●● 4
AI agents are built and evaluated behind closed doors, so there is no way to compare them. ORO is an open arena.
Openness against closed door evaluation, no cost or ownership comparison
Q3How it resists gaming or fails●●●●● 5
dated paper page (blog June 1 2026) with ShoppingBench scores 18.0 to 42.7 percent and top miner 77.3 percent, models on Hugging Face, live counter on the front page, leaderboard is JS rendered
Q4Identity and documentation●●●●● 4
subnet_name ORO matches the site, description AI commerce agents is a tag not a sentence, url resolves, What is ORO section and docs serve as about and learn pages
How this score came to be. Verified means the scorer fetched the page and the words are on it; inferred means concluded from code, absence, or a third party. The link badge is a separate automated check made before publication.
Something wrong? Corrections of fact are applied as they arrive during the window; score disputes are batched at its close. How to file one · GitHub issue · email.