In short
When you license an RL environment you are buying a reward signal as much as a simulator. The questions that matter most are whether the reward can be gamed, whether evaluation tasks are held out from anything public, how the environment compares with human performance, and who owns the trajectories it produces.
The market in one paragraph
RL environments became a distinct market in 2025 and 2026. A landscape published by Troveo in August 2026 groups vendors into large data companies expanding into RL (such as Scale AI, Surge AI, Mercor and Turing), environment-native startups (such as Mechanize, Fleet AI, HUD and Bespoke Labs), and open ecosystems and infrastructure (such as Prime Intellect and Modal). Wing Venture Capital argued in January 2026 that labs want partners who stress-test verification with them, not just vendors who build environments on request. SimReal is an environment-native builder focused on economic work graded by mechanisms.
Twelve questions to ask
The reward
- What exactly computes the reward? A mechanism, a test suite, a held-out answer or a model acting as judge?
- Has the verifier been attacked, and can we see the record? Ask for the strategies tried, which got through, and what changed.
- Does it grade the process or only the final state?
The tasks
- Are evaluation tasks held out from anything published?
- What happens when a split saturates? Is there a refresh or a live track?
- Is there a human baseline, and how was it measured?
Realism
- Whose rules drive the dynamics? Real exchange rules, accounting standards, a production codebase?
- Who defined correct? Practitioners with credentials you can check?
Engineering
- What is the interface? A Gym-style API, containers, throughput per episode, seeded replay?
- Is every episode logged in a form you can audit later?
Commercial
- Who owns the trajectories and outcome data, and is the licence exclusive?
- How is it priced? Per environment, per build, per run, or as continuous re-testing?
Red flags
- Scores produced only by an LLM judge, with no deterministic check behind it.
- No record of adversarial testing, or a claim that the reward “cannot be gamed” without evidence.
- Evaluation tasks that are all public.
- No human baseline, or one measured under different conditions from the model.
- Vague answers about who owns the data an environment generates.
Pricing models you will see
Environment pricing is rarely published. The common structures are a licence per standard environment, a fixed fee for a custom build, a premium for exclusivity, and recurring fees for re-testing new model versions. As one reference point, SimReal’s pricing starts at $20,000 per standard environment and $300,000 per complex simulation, with exclusive licences at four to five times the non-exclusive price.
Key takeaways
- You are buying a reward signal; audit it first.
- Ask for the adversarial record, the held-out policy and the human baseline.
- Check data ownership and exclusivity before signing.
- Expect per-environment, per-build, exclusive and recurring pricing structures.
Questions
How much does an RL environment cost?
It depends on complexity and exclusivity. Published prices are rare; SimReal lists standard environments from $20,000 each, complex simulations from $300,000 per build, and exclusive licences at four to five times the non-exclusive price.
Should we build environments in-house or buy them?
Build where the workflow is proprietary and you have domain experts; buy where an existing environment already has a tested verifier and human baselines. Many labs do both.
Can one environment serve training and evaluation?
Yes, if the evaluation split is held out and never used for training. The same world then gives a benchmark score and a training reward on the same terms.