Why AI in production may need humans
As an illustrative example, even a model that scored 95% in an evaluation would have to handle the remaining 5% without sending a wrong email or writing bad data into the ERP. Actual rates must be measured for each use case; human review covers risk outside the accepted threshold.
HITL isn’t blind distrust of the model. It’s the assumption that generative AI makes confident mistakes, that the cost of some of those mistakes is high or irreversible. The cost of reviewing a draft should be compared with the impact of a wrong reply, not assumed to be lower. For sensitive processes, customer communication, writes to the CRM, decisions with regulatory impact, the human inside the loop is what makes the agent deployable.
Three common HITL patterns
The loop isn’t always the same shape. Three patterns to evaluate against the process risk are:
- Review-before-send. The agent drafts, a person approves with a click. Default pattern for customer emails, review responses, sales proposals and anything going out under the brand. The agent provides the speed; the human absorbs the risk.
- Confidence-gated. The model scores its own confidence. Above a threshold, it acts on its own. Below, it escalates to a person. Useful for classification, ticket triage or data enrichment when the threshold and sampling demonstrate which cases can be automated and which need human eyes.
- Exception escalation. The agent acts on its own in the normal flow but detects exception signals, complaint, urgency, ambiguous data, sensitive policy, and hands the case to a person with full context. Standard pattern for concierge, support and sales agents.
The three combine inside the same process. What matters is that the pattern is chosen on purpose, not by default.
Designing the loop: when, who, with what information
A badly designed loop turns into a bottleneck. A well-designed one feels invisible. Three questions decide which one you get:
- When? Before the output reaches a customer, mutates an external system or writes to a sensitive record. Not after.
- Who? The person with the authority and the context to decide, not whoever happens to be free. Routing reviews to the wrong team turns approval into a rubber stamp nobody really reads.
- With what information? The draft, the sources the agent used, the case context and the specific action being proposed. Without that, the person approves blind, which is worse than having no loop at all.
The loop also needs a clear channel: where the notification lands, what happens if no one picks it up, how long the person has to decide and what fires if the SLA expires. Without those, the queue grows and the project quietly dies.
When you can remove the human (and when you can’t)
HITL isn’t forever. The right question isn’t can we ship it without review? but what would have to be true for review to come off?. Three reasonable conditions:
- The unchanged-approval rate exceeds the threshold defined for that risk over a sufficient period. A sustained 95% would only be an illustrative example, not a universal rule.
- The cost of any individual error is low and reversible, an internal draft, a classification you can fix, a non-critical record.
- There’s an after-the-fact audit mechanism: sampling, alerts, dashboards. The human exits the real-time flow, not the supervision.
For processes with regulatory, financial or safety impact, human oversight may be a necessary control. The EU AI Act recognises it as a mitigation mechanism for high-risk systems. Sector-specific obligations must also be checked for each case.
