Verifiable rewards (RLVR), explained

In short

Reinforcement learning with verifiable rewards (RLVR) trains a model with rewards computed by a program that checks the result, such as an answer key or a test suite, instead of a learned reward model. Ai2 named the approach in its Tülu 3 report in November 2024. The open problem is making rewards verifiable for work that has no answer key.

What makes a reward verifiable

A reward is verifiable when a deterministic check can decide it, without asking another model for an opinion. Common checks:

an answer key for questions with one correct answer;
a test suite or compiler for code;
a proof checker or feasibility certificate for mathematics and optimisation;
a simulator that settles the outcome by the domain’s rules;
the real event itself, for forecasts.

The alternatives are a reward model trained on human preferences (as in RLHF) or an LLM asked to judge the output. Both are useful, but both are learned functions, and a learned function has blind spots an optimiser can find.

Why verifiable rewards took off

The Tülu 3 report from the Allen Institute for AI (November 2024) introduced the name reinforcement learning with verifiable rewards, training on maths and instruction-following tasks where correctness can be checked.

The appeal is simple: a cheap, exact check beats a sophisticated but gameable one, at least in domains where an exact check exists.

The limit: most work has no answer key

Maths and code are unusual. Most economic work has no answer key: whether a price was right, a loan was sound or a strategy had real edge is known late, blurred by luck, and observed once. RLVR in its original form cannot reach that work.

The way forward is to build the check. Recreate the situation in an environment, let the agent act, and settle the result with the domain’s own rules. The reward is then verifiable because the mechanism is, not because a human wrote down the answer in advance.

Seven ways to make a reward verifiable

Held-out exact answers, kept by a third party so the grader never sees them. Machine-learning competitions work this way.
Executable tests, run in isolation, including hidden tests the agent cannot read.
Certificates: a feasible solution that can be checked, or a proof that none exists. Optimisation problems allow both.
Process invariants: rules that must hold at every step, such as a ledger that stays balanced and every entry carrying evidence. They catch agents that reach the right final state the wrong way.
Settlement by a mechanism: a matching engine settles trades; the reward is the change in equity. Nobody’s opinion is involved.
Real-world resolution: forecasts graded when the event happens, against a baseline sealed in advance.
Human-anchored thresholds: the best human result on the same task, measured the same way, as the bar a model must clear.

Where verifiable rewards still fail

A check can be exact and still incomplete. In March 2026, METR had maintainers of scikit-learn, Sphinx and pytest review 296 AI-generated pull requests that had passed the automated grader; merge rates ran 24.2 percentage points below the grader’s score. In February 2026, OpenAI reported that 59.4% of 138 difficult SWE-bench Verified tasks it reviewed had substantial flaws in their tests or descriptions.

Verifiers need verification too. The next article covers how agents exploit them.

A checklist for a verifiable reward

Deterministic and cheap enough to run on every rollout.
Checks the process, not only the final state.
Attacked before release with degenerate strategies: doing nothing, reusing stale evidence, exhausting the budget, submitting output that only looks correct.
Evaluated on tasks held out from anything published.
Versioned, with every bypass that was found logged next to the fix.

Key takeaways

RLVR uses programmatic checks instead of learned judges.
It works best where an exact check exists: maths, code, proofs.
For economic work, the check has to be built: a simulator that settles outcomes by real rules.
Exact checks can still be incomplete, so every verifier should be attacked before use.

Questions

Is RLVR the same as RLHF?

No. RLHF trains on a reward model learned from human preferences. RLVR computes the reward with a programmatic check, such as an answer key or a test suite.

Can an LLM judge be a verifiable reward?

Not in the strict sense. An LLM judge is a learned function whose output can be influenced by the text it grades. It can support a verifiable reward, for example by extracting an answer that is then checked exactly.

What is a rule-based reward?

A reward computed by fixed rules, such as whether the final answer matches the key and whether the output follows a required format.