Wing Venture Capital's analysis of the RL environment market names coding and computer use as the first markets to take off. Its reasons are that the work is economically valuable, the tasks are complex, and the results are verifiable. The last reason is the one buyers should care about most. In 2026, vendors sell "environments" as a product, labs hire engineers just to build them, and enterprises get pitched custom post-training for their own workflows. Many of those pitches leave out the main fact about the deal: the environment is the deliverable. The policy-gradient variant, the GPU hours and the base model are all interchangeable. The scoring function for your task is not.
That leads to a simple filter, and we'll defend it below. If you cannot write a programmatic checker for the task, you cannot buy an RL environment for it. If you can write the checker, you already have most of an eval suite, and an eval suite is usually the cheaper win.
What You Are Buying When You Buy an RL Environment
Han Chung Lee's taxonomy of RL environments for LLM agents is a good reference for the parts. From a buyer's side, an environment breaks into five components. Each one has different cost, risk and ownership questions.
| Component | What it is | Who usually builds it | What can go wrong |
|---|---|---|---|
| Task set | The prompts or goals the model attempts, with difficulty spread | Vendor, often seeded with buyer data | Tasks too easy, too narrow, or leaked into eval sets |
| Environment state | The sandbox: a repo, a browser session, a mock CRM, a database snapshot | Vendor | Drift from your real systems, so skills don't transfer |
| Action interface | The tools and APIs the model can call | Shared | Tool surface differs from production harness |
| Verifier / reward | Code that scores each attempt | Should be the buyer, often is the vendor | Reward hacking, false positives, silent partial credit |
| Episode harness | Reset, parallel rollout, logging, isolation | Vendor or training provider | Non-reproducible runs, cross-tenant data exposure |
SuperAnnotate's guide to building tasks and reward systems makes the same point from the builder's side. Most of the engineering effort goes into task design and reward logic, and very little goes into the training loop. That has a commercial consequence. A vendor that builds your verifier holds the most valuable artifact in the engagement. If the contract treats the verifier as vendor tooling and not as a deliverable, you'll rent your own definition of success.
Business readers can think of the environment as the job's test, the practice field and the grading rubric bundled together. Training teaches the model to pass that test. If the test measures the wrong thing, the model gets very good at the wrong thing, and it does so quickly.
Why Coding and Computer Use Moved First
Coding has a built-in checker: the test suite. A patch either makes the failing tests pass without breaking the others, or it doesn't. Computer-use tasks have a similar property, because you can inspect the end state of the application. Did the form get submitted? Does the spreadsheet cell hold the right value? Is the file in the right folder? These two domains combine high economic value with cheap, automatic scoring, which matches Wing's explanation.
The research trend points the same way. The Tülu 3 post-training report named the approach "reinforcement learning with verifiable rewards" (RLVR). In RLVR, the model is rewarded only when an outcome can be checked, such as a correct math answer or a satisfied instruction constraint. The DeepSeek-R1 paper relied on rule-based accuracy and format rewards and explicitly avoided neural reward models for its reasoning training, citing the risk of reward hacking. Much of the recent progress in post-training has come from environments where a program decides who won, and less from adding more pretraining data.
The vendor market has organized around that fact. Troveo's overview of RL environment companies in 2026 describes consolidation: environment specialists are raising significant capital, and human-data labeling firms are pivoting into environment construction. The pivot makes sense. Labeling vendors already have workforces that can write tasks and grade edge cases, and environments are where the budget has moved.
The category is still immature. A review titled RL Environment Platforms: Not One Buyable Product Yet argues that interfaces are converging but procurement is not. Buying "an RL environment" turns out to be a set of separate decisions: who builds the tasks, who hosts the sandbox, who runs training, who serves the resulting model. One vendor can't substitute for another across those decisions, and each one has its own line of responsibility. A breach in the sandbox, a bug in the verifier and a regression in the trained model would each fall on a different party.
The Checker Test: Can You Score the Task With Code?
Before taking a meeting about custom post-training, put the task through one question: can a program decide whether an attempt succeeded, without a human in the loop and without asking another model for its opinion?
The answer usually puts the task into one of three tiers.
| Tier | Example tasks | How success is checked | What to buy |
|---|---|---|---|
| 1. Deterministic | Code changes with test suites, SQL that must return a known result set, form-filling with a known correct end state, data extraction with ground-truth fields | Exact match, tests, state diff | RL is viable. Build the evals first anyway. |
| 2. Structured but partial | Contract clause extraction, ticket routing, claims coding against a rule book | Schema validation plus business rules, with some cases needing review | Evals plus harness work. Revisit RL once the checker covers most cases. |
| 3. Judgment-only | Negotiation emails, strategy memos, brand-voice writing, escalation tone | Human or model judgment | No RL environment can be built honestly. Consider SFT, retrieval, or human review. |
Tier 3 is where the most misleading pitches show up. A vendor can wrap a Tier 3 task in an "environment" by scoring it with an LLM judge. That technically produces a reward signal, but the model under training will optimize against the judge's blind spots. DeepSeek's reason for avoiding learned reward models applies here directly: a policy trained against a soft scorer learns to please the scorer.
Here is a hypothetical composite example. A mid-sized freight forwarder wants a model that drafts customs declarations from commercial invoices. Part of the task is deterministic: HS codes must be valid, declared values must match invoice totals, and required fields must be present and well-formed. Part of it is judgment: picking the right tariff classification for an ambiguous product description. The deterministic part could become an RL verifier. The judgment part cannot, unless the company has a large history of declarations that were audited and corrected, so that classification becomes a lookup against ground truth. Which tier the task lands in depends on the data the company keeps, as much as on the task itself.
The checker test also catches a quieter failure. Some tasks look deterministic only because the obvious check is shallow. "The JSON parses" is a check. "The JSON parses and every field matches the source document" is a verifier. If a vendor's reward only confirms that the format is right, you're paying to train formatting, which a schema in the harness enforces for free. Our post on output contracts and schema validation covers that cheaper layer.
Why the Checker Is Usually the Cheaper Win
Suppose your task passes the checker test. Before you fund training, consider what the checker already gives you.
A working verifier plus a few hundred representative tasks is an eval suite. You can run every current frontier model and open-weight model against it this week. You'll learn how far the best available model falls short on your task, and whether that shortfall is in reasoning, tool use, retrieval or formatting. That's the question workflow-specific evals exist to answer, and generic benchmarks can't answer it for you.
Here is the counter-intuitive part. The more verifiable a task is, the less you may need to train for it. A cheap, reliable checker lets you generate several candidates at inference time, verify each one, and return only the one that passes. You can retry on failure, route hard cases to a stronger model, or escalate to a human when nothing passes. Those are harness techniques. They need the checker and don't need a training run. Before paying to move pass@1, measure how pass@k and the variance across agent runs behave with a verifier in the loop.
The checker-first sequence looks like this:
- Write the verifier. Start with the deterministic checks and add business rules. Write down which cases it cannot score.
- Assemble the task set. Pull real cases from production logs and treat every production failure as a new eval case. Hold out a slice that no vendor ever sees.
- Benchmark the field. Run several models with your production harness against the suite. Record pass rate, cost per task and failure categories.
- Fix the harness. Tighten tool definitions, add retrieval, add generate-and-verify loops. Re-run the suite after each change, and use a harness A/B discipline so you know which change helped.
- Price the remaining gap. If a measured gap remains, task volume is high, and the per-task value justifies it, you now have a strong case for post-training, plus the verifier and held-out set to hold the vendor accountable.
Most teams stop at step 4, and that's a good outcome. The teams that reach step 5 negotiate from a much better position. They can give a vendor a verifier, a task distribution and a baseline, and ask a narrower question: how much can you improve this number, at what cost, and how will we confirm the gain on tasks you haven't seen? Our earlier comparison of SFT vs. RL for enterprises covers the next choice. RL fits when you can score outcomes but can't easily write down ideal trajectories. SFT fits when you have good examples to copy. Tools such as the Harbor framework for agent evals give the suite a home that can run the same tasks against any model or vendor.
Diligence Questions for RL Environment Vendors
If you get to procurement, keep in mind that this is a young category with a thin diligence record. The future-stack review counts 38 vendors, of which only 10 publicly disclose SOC 2 certification. Environments often contain copies of your code repositories, database snapshots and internal tool schemas. The security bar should be at least the one you'd set for any vendor holding production data, and arguably higher, because the sandbox is built to let an autonomous model take actions inside it.
| Question | Why it matters | Acceptable answer |
|---|---|---|
| Who owns the verifier, task set and environment code at the end? | Those are the reusable assets; the model checkpoint goes stale | Buyer owns them, with source delivered |
| How are held-out tasks kept apart from training tasks? | Without separation, reported gains measure memorization | Buyer-held test set the vendor never sees |
| What reward-hacking audits do you run? | Models find verifier gaps faster than people do | Sampled trajectory review with documented exploits and patches |
| How are our repos and data isolated during rollouts? | Thousands of parallel episodes multiply exposure | Per-tenant sandboxes, egress controls, current attestation |
| Can we reproduce a training run's rewards from logs? | Disputes over results need evidence | Logged episodes with seeds, versions and verifier outputs |
| What happens to our environment if you are acquired or pivot? | The market is consolidating fast | Escrow or full handover terms in the contract |
| Does the sandbox match our production tool surface? | Skills learned against mock APIs may not transfer | Shared tool schemas, with drift measured on held-out production cases |
Pay particular attention to the acquisition row. When labeling firms pivot into environments and specialists raise large rounds, some of today's vendors will be owned by someone else, or will be serving a frontier lab as their main customer, within a year. Environment code you own keeps its value either way, while a vendor-hosted environment you can only access through their platform may not.
Hiring is the other side of this. Labs now recruit dedicated RL environment engineers, and the skills overlap heavily with good eval engineering: domain modeling, adversarial test design and sandbox infrastructure. If you can't staff that internally, the verifier is the piece most worth paying an outside team to build, because it stays useful no matter which model or vendor you pick next.
Build the Checker Before You Sign the Training Contract
OpenNash works on the checker-first half of this sequence. That means mapping the task, writing the verifier and the held-out set, benchmarking current models in the client's own harness, and closing whatever gap harness changes can close. Clients own all of it. If the numbers then justify post-training, the client goes to an environment vendor with a defined task, a baseline and an acceptance test. If they don't, the client has a working eval suite and a better production agent, and has avoided a training contract.
Some buyers should go straight to a vendor. A team with a large coding or computer-use workload, an existing test harness and in-house ML staff can skip us and should. Teams with Tier 3 tasks should wait. No environment vendor can fix a task that has no checker, and OpenNash can't either.
If you have a post-training proposal on your desk, book a call and we'll run the checker test on the task in that proposal with you. You'll leave knowing which tier the task falls into and what the verifier would need to check.