Somewhere in your stack, an AI drafts something and a person approves it. Everyone agrees this is the responsible setup. Nobody has priced it.
Human-in-the-loop (HITL) gets treated as a free safety feature, like a seatbelt you bolt on at the end. It isn’t free. It’s an operating model with labor, queues, and rework attached, and if you don’t measure what it costs, you can’t tell whether the AI saved you anything. This post gives you one metric, cost per accepted outcome, and a way to calculate it this week.
Why “we have a human review step” isn’t a cost answer
Here’s the usual pitch for an AI workflow: the model produces a draft in ninety seconds instead of ninety minutes. Time saved, volume up, done. Then a person reviews it, because they should.
That review is where the savings quietly go to be audited. Someone reads the draft, catches a wrong claim, rewrites two paragraphs, sends it back, waits, and approves a version that looks suspiciously like what they would have written from scratch. The dashboard shows “assets produced.” It doesn’t show the forty minutes of review and repair behind each one.
This is the same trap I’ve written about before: “we did more” is the most dangerous sentence in a QBR. Output is a proxy metric. Volume of drafts tells you the model is fast. It tells you nothing about whether the system is cheaper.
The metric: cost per accepted outcome
Pick the unit that matters: a published article, an outbound sequence that passed QA, a campaign brief signed off by sales. Then count only the ones that were accepted, meaning they shipped without being thrown out or redone later.
The formula is plain:
Cost per accepted outcome = (tool cost + generation cost + review time + exception handling + rework) ÷ accepted outcomes
The numerator is everything. The denominator is only what survived. A workflow that produces 100 drafts and ships 40 has a very different cost than one that produces 50 and ships 45, even though the first one looks more productive on a volume chart.
Compare this number against your pre-AI baseline for the same outcome. If you don’t have a baseline, build one from last quarter’s time sheets and agency invoices, imperfect as they are. Decision-grade beats theoretically perfect.
The four cost lines people forget
Tool spend is the easy line. The costs that sink the math are the ones nobody logs.
Review. Minutes per item times the loaded hourly cost of whoever does it. If your most senior person is the reviewer because nobody else is trusted with it, you’re spending your most expensive hours on the cheapest part of the work. Price it that way.
Exceptions. Some items fall outside what the workflow handles well: a regulated claim, a sensitive account, a product the model keeps getting wrong. Track what share of items get escalated and what each escalation costs. A 10% exception rate that consumes your best person’s Thursday is a real line item.
Rework. Count the items that came back after approval: the email a rep refused to send, the page legal pulled, the blog post that needed a full rewrite because the point of view was generic. Rework is the cost of a review that didn’t catch the problem, so it belongs in the same ledger.
Waiting. Queue time isn’t labor cost, but it’s real. A campaign that sits three days in an approval inbox has a cost in pipeline timing, even if you can only estimate it.
What the number tells you to do
Once you have cost per accepted outcome, the decisions get less philosophical. Three patterns show up.
If review time is the biggest line, your reviewers are doing editing the system should have prevented. Fix upstream: tighter inputs, better source material, a clearer brief. Automating a bad decision just gets you the wrong answer faster, and putting a human behind it just gets you the wrong answer slower and more expensively.
If exceptions dominate, split the workflow. Route the routine items through light review and the risky ones through heavy review, instead of applying the same scrutiny to everything. Review effort should follow risk, not habit.
If rework dominates, your review step is checking the wrong things. Look at what came back and why, then change the checklist. A reviewer scanning for typos while the argument is generic is doing a lot of work for no protection.
Sometimes the number says the workflow isn’t worth it for that outcome. That’s a useful answer. Pull it back to a human, or to a smaller AI-assisted role, and spend the effort where the economics hold. [Source needed: no benchmark is provided here for typical review or rework rates, so measure your own.]
Run it on one workflow this week
Don’t instrument everything. Pick one AI-assisted workflow with a clear output and run a small, controlled test.
- Choose the outcome (say, a sales-ready outbound sequence) and define “accepted” in one sentence.
- Log the next 20 items: generation time, review minutes, whether it was escalated, whether it needed edits, whether it was used or discarded.
- Add tool cost and loaded labor rates.
- Divide by accepted items only.
- Compare to your pre-AI baseline for the same outcome.
Twenty items is enough to see which line dominates. You’ll probably find the answer isn’t what you assumed, and that’s the point of measuring.
Keep the human in the loop. Just stop pretending the loop is free. Review, exceptions, and rework are the price of trust, and a number you can put in front of your CEO or CFO is worth more than a reassurance.
Next step: Pick one workflow, log 20 items, and calculate your cost per accepted outcome by Friday. Then reply and tell me which of the four cost lines surprised you.