PM POV on Post Training: Your Tasks Are Not Grounded in Economic Reality

You can have a solid RL environment and well-written rubrics and still end up training a model nobody cares about. If the tasks you're training on don't map to work that real people do and pay for, you're optimizing for a universe that doesn't exist. This is the product manager's half of post-training, and most teams skip it.

Know why the domain matters

If you're building tasks for a domain, you should be able to articulate why that domain matters. How many people do this job? What does a day in their life actually look like? Which high-value tasks eat their time and could realistically be handed to a model?

OpenAI's GDPval is this paragraph turned into a benchmark, and the recipe is copyable at startup scale. They took the nine sectors that each contribute more than 5% of US GDP, picked the five occupations per sector that contribute most to total wages using O*NET and BLS wage data, kept only occupations where at least 60% of the O*NET tasks are digital, and then had professionals averaging 14 years of experience write the tasks. A task's dollar value is expert hours multiplied by the occupation's median hourly wage; the average task on their gold set is worth $398.46.¹ Anthropic runs a version of the same arithmetic on live traffic and reports the average task value on Claude.ai in hourly-wage equivalents, $47.90 in the March 2026 report.²

The public data you need is free. The Occupational Employment and Wage Statistics tables give you employment and the median hourly wage for roughly 830 occupations.³ O*NET's task statements flag each task as Core (relevance at least 67% and importance at least 3.0) or Supplemental, and the task ratings file gives importance and frequency per task with error bars.⁴ When you push for an environment for, say, digital copywriting, you should know what a copywriter does in a day, how much of the economy that represents, and which of those tasks your environment targets. That framing separates "we made some tasks" from "we mapped the highest-value copywriting workflows and built evaluation surfaces for them." One of those is valuable. The other gets thrown away.

This helps you prioritize

Not all tasks within a domain are equally valuable to train on. A model that can reliably do the judgment part of a marketing lead's job is a lot more interesting than one that does the reformatting part. Ground the task selection in what moves the needle economically.

The cleaner version of that split is not the wage, it is the kind of knowledge. Stanford's payroll-data study of AI's employment effects finds that declines concentrate where AI substitutes for codified knowledge and stay flat or positive where the work is tacit; workers aged 22 to 25 in exposed roles sit about 19% below their expected employment as of mid-2026.⁸ Codified work is where the labor market is already moving. Tacit work is where a post-trained model earns its keep.

Picking the wrong task does not just fail to help. In the BCG field experiment, consultants using AI on tasks inside the model's frontier finished 12.2% more tasks 25.1% faster at higher quality; on the one task outside the frontier, AI users were 19% less likely to be correct than the control group.⁹ If your task set contains that outside-frontier task and your product ships it, you made your users worse.

The counterargument, and where it wins

The strongest objection is Jason Wei's verifier's law: how well AI learns a task tracks how verifiable it is, and a task is easy to verify when it has objective truth, is fast and scalable to check, is low noise, and gives a continuous reward.¹⁰ On that view, economic value is a distraction early on; verifiability is the whole game. HUD's verifier guidance says the same thing from the builder's side: write the end-state assertion before any reward formula, and if you cannot write it, the task is not ready.¹¹

Volume cuts the same way. GDPval selected occupations by total wages, which is employment times wage, not wage alone.¹ Run the BLS numbers and 2.67 million customer service reps at $21.53 is a bigger annual wage pool than 864 thousand lawyers at $76.76.⁶ And Tulu 3's verifiable-rewards recipe on math and instruction following improved benchmarks it never trained on.¹² Boring, cheap, verifiable tasks are not laziness. They are gradient.

Here is where the objection loses. Danish payroll data across 25,000 workers and 11 occupations found generative AI saved 2.8% of work hours on average and produced precisely estimated zero effects on earnings; only 3 to 7% of reported productivity gains passed through to pay.¹³ Automating the verifiable slice does not by itself create value anyone pays for. So the synthesis I actually use: economic value picks the workflow, verifiability picks the first task inside it. Start where the assertion is easy to write, but only inside a workflow whose annual pool you have already sized.

Startups: check your own product first

If you're a startup trying to set up your own product as an RL environment, to automate workflows with custom post-trained sub-agents, ask yourself before you begin: is my product stable enough and doing well enough to justify post-training on it at all? If you don't have stable product metrics or any indicator of product-market fit (unless building a post-trained model is itself your differentiation), be really, really careful before you start post-training custom models without evidence that the process you're automating is viable.

Make that concrete. Rahul Vohra's version of the Sean Ellis test: survey people who used the product at least twice in the last two weeks, ask how they'd feel if they could no longer use it, and look for 40% or more saying "very disappointed." Superhuman went from 22% to 58% by naming its high-expectation customer and building for that cohort.¹⁴ If you cannot name your high-expectation customer, you cannot write a rubric for what "good" means to them. Hamel Husain puts A/B testing with real users at level three of his eval hierarchy, appropriate only for mature products, which is another way of saying the outer loop that makes a rubric trustworthy does not exist yet at an unstable company.¹⁵ Michael Hammer called the general failure "paving the cow paths" in 1990: using technology to speed up a process that was broken to begin with.¹⁶

It's also going to be really hard to build a rubric if your product metrics and user experience are not stable. My favorite tool in this space is still hud.ai: an SDK for defining environments, tasksets and graders that runs any agent against them, plus trace QA that audits rollouts for false positives, false negatives, reward hacking and prompt-grader misalignment.¹⁷ The trace QA is the part that matters for this post. It is where you find out that your economically valuable task has a broken grader.

Interactive guide: the task-value worksheet

Checklist: is this task worth an environment?

References