Environments compound. Datasets deplete.
Why we sell pinned environments with deterministic verifiers instead of one-shot preference data, and what that means for teams buying post-training data.
Preference data dies with the checkpoint that consumed it. Train on a batch of comparisons and the next model release makes it archaeology: the failure modes it captured are gone, the behaviors it rewarded are baseline, and the budget it consumed buys nothing against the new frontier.
A pinned environment with a deterministic verifier works the other way. Every new model release is a reason to run it again. The environment does not care which checkpoint is inside the loop; the verifier grades the new model exactly as it graded the old one, and the delta between runs is itself the product.
We sell the instrument, not the delta
This is the core of how rlsupply prices and builds. An environment is an instrument: real software at a pinned version, seeded state, a reward function compiled from expert rubric weights, and verifiers grounded on the authoring expert’s own recorded run. Instruments get more valuable as models improve, because headroom is what researchers pay to measure.
It also changes the refresh economics for buyers. Instead of recommissioning a dataset from scratch each cycle, a licensed environment yields fresh variants, harder difficulty bands, and a new failure corpus against each release, from the same instrument your team already integrated.
What depletion looks like in practice
Teams that bought static expert datasets describe the same arc: meaningful gains against the current model, then a new release that blows past the tuned checkpoint and resets the shopping list. The dataset did its job once. It cannot do it again.
Environments hold their calibration across that boundary. The tasks are real work with an explicit notion of done, so a better model does not invalidate the instrument. It just scores higher, until it does not, and the places where it does not are exactly where the next training contract lives.
What to ask any environment vendor
Three questions separate instruments from screenshots:
- Does state reset deterministically? If two runs can pollute each other, the scores are noise.
- What grounds the grader? If the answer is a judge model’s opinion, ask what audits it. Our verifiers must reproduce the authoring expert’s own run before shipping.
- What happens at the next model release? If the honest answer is “buy another dataset,” you are renting depreciation.