AI agents for business pay off on processes that are repetitive, rule-shaped, high-volume, and reversible: lead triage, support reply drafts, and document intake. They’re a poor fit for irreversible, high-stakes, or poorly defined work. For everything in between, the real design decision is where a human approves before the agent acts.

What an AI agent is, and what it is not

A single prompt is one call: text in, text out, and your code decides what happens next. An agent is a model running in a loop. It reads a goal, picks the next step, calls a tool (a CRM lookup, a search, or a draft-email function), reads the result, and decides again until the task is done or a stop condition fires. The pattern goes back to research such as ReAct (Yao et al., 2022), which interleaved reasoning steps with actions against external sources.

Between a prompt and an agent sits the workflow: a fixed code path with model calls at set steps, such as “classify the ticket, fetch the matching policy, and draft a reply.” In a workflow the code picks the path, and in an agent the model does. Many processes pitched as agent projects work better as workflows, because a fixed path is easier to test, cheaper to run, and harder to misuse. We recommend starting with the most constrained design that does the job and giving the model more freedom only where the fixed path breaks.

What makes an agent useful also makes it risky. Tools let it act on real systems. The loop lets it recover from a bad step, or compound one.

Which business processes fit an agent

The processes that work well as agents share most of these traits:

  • Repetitive and high-volume. Dozens or hundreds of instances a week, enough to spread the build cost.
  • Rule-shaped. A new hire could do the work from a written runbook. When the rules live only in one person’s head, writing them down comes first.
  • Reversible actions. A wrong tag, a draft, or an internal note takes seconds to fix. A sent email, a payment, or a deleted record doesn’t.
  • Checkable output. There’s a way to tell whether a result was right, ideally against past cases with known outcomes.
  • Reachable systems. The data and actions sit behind APIs the agent calls with scoped credentials.

Three AI agent use cases match this profile well. In lead triage, the agent enriches an inbound lead, compares it with the ideal customer profile, routes it to the right owner, and drafts a first reply. In support, it reads the ticket and its history, retrieves relevant help articles, and drafts an answer for a person to send. In document intake, it extracts fields from invoices, applications, or contracts, validates them against business rules, passes clean records on, and queues the exceptions.

Poor fits share the opposite traits: irreversible financial actions, decisions about people, and goals with no definition of done. Some decisions about people are also regulated. The EU AI Act lists systems that filter job applications or evaluate candidates, and systems that score creditworthiness, among the high-risk uses in Annex III, which carry human oversight and other obligations.

Decision matrix: process traits, agent fit, and where the human sits

The traits decide the fit, and the fit decides where the human stands.

Process traits Example Agent fit Where the human sits
High volume, rule-shaped, internal, and reversible actions Tagging, enriching, and routing inbound leads Strong Sampled review after the fact, plus escalation on failed checks
High volume, rule-shaped, and customer-facing output Support reply drafts Strong as a drafter Approval gate before send, relaxed per category once edits stay rare
Structured extraction with clear validation rules Invoice or application intake Strong Exception queue for records that fail validation
Judgment-heavy, low-volume, and reversible Account research before a sales call Medium The human is the consumer and reviews before use
Irreversible or financial actions Refunds above a limit, payouts, and record deletion Weak for execution Agent prepares, human executes, and code enforces the limits
Decisions about people Candidate screening, credit decisions Poor, often regulated Human decides. The agent at most summarizes, after legal review
Goal with no definition of done “Improve our marketing” Poor Process definition comes first

Human in the loop: four patterns and how to choose

Human in the loop AI is not one switch. We recommend combining four patterns, chosen per action rather than per agent.

Approval gates before external actions. Anything that leaves the company’s systems or changes a record pauses until a person approves it: a customer email, a refund, or an update to a contract field. OWASP’s guidance on excessive agency recommends requiring a human to approve high-impact actions before they are taken. We add one detail: the reviewer sees the exact action that will execute, not the agent’s summary of it.

Confidence thresholds. Low-confidence cases go to a person, and the rest proceed. The catch is where the confidence comes from. A study of confidence elicitation in LLMs found that models asked to state their confidence tend to be overconfident, so a model that calls itself “very sure” gives a weak signal. Checks computed in code are stronger: a validation rule failed, a required field is missing, no policy matched the request, or a classifier score fell below a cutoff set on labeled data.

Sampled review. Actions run, and a reviewer checks a random slice afterward plus every case in a new category. This fits reversible, high-volume work, where reviewing everything would erase the savings.

Escalation. The agent needs a legitimate exit: “I can’t finish this, here is what I found.” Paired with hard caps on steps, spend, and repeated tool errors, it turns a stuck loop into a handoff instead of a bill.

Where the human sits comes down to two questions per action: what does a wrong action cost, and is it reversible? Cheap and reversible actions get sampled review. Costly but reversible ones get thresholds plus escalation. Irreversible or external actions get an approval gate.

We also recommend a second test: an agent that at once reads untrusted input, touches sensitive data, and can change state or communicate externally should get per-action human approval. The policy works best written down as config, reviewed like code and enforced outside the prompt. A sketch for a support-reply agent, with placeholder limits:

# Oversight policy, enforced by the tool layer, not by the prompt
actions:
  search_help_center:   { mode: auto }
  set_ticket_tags:      { mode: auto, sampled_review: true }
  draft_reply:          { mode: auto }
  send_reply:           { mode: approve }   # a person clicks send
  issue_store_credit:   { mode: approve, max_amount: 50 }
  delete_customer_data: { mode: not_exposed }
limits:
  max_steps_per_task: 15
  max_cost_per_task_usd: 0.40
escalate_when: [validation_failed, no_policy_match, customer_asks_for_human]

Approval gates have their own failure mode: people approve everything. The EU AI Act names this automation bias in its human oversight article, and the OWASP Top 10 for Agentic Applications lists Human-Agent Trust Exploitation (ASI09) as a risk of its own. The usual countermeasures are short review queues, evidence shown next to every proposed action, and an eye on reviewers whose time per approval drops to a few seconds.

Risks specific to agents

A chatbot that hallucinates gives a wrong answer. An agent that hallucinates, or gets manipulated, takes a wrong action. We focus on three risks, and all three appear in OWASP’s Top 10 lists.

Prompt injection (OWASP LLM01). Instructions hide in content the agent reads: an inbound email, a web page, or a file attached to a ticket. Greshake et al. showed that LLM-integrated applications “blur the line between data and instructions,” so text planted in retrieved data can steer the model. For a lead-triage agent, every web form and email is untrusted input. The UK National Cyber Security Centre warns that prompt injection may never be fully mitigated, so the design goal is to limit the damage a successful injection does.

Excessive agency (LLM06 in OWASP’s 2025 list). The OWASP entry traces damaging actions to excessive functionality, permissions, or autonomy. Its first example is a tool that only needs to read documents but can also modify and delete them.

Tool misuse (ASI02 in the agentic list). OWASP describes agents bending legitimate tools into destructive outputs. An agent can do this without exceeding its permissions: deleting data it was allowed to touch, over-calling a paid API, or sending information where it shouldn’t go.

The controls are mostly ordinary engineering:

  • Only the tools the process needs, with read tools split from write tools.
  • Scoped, per-agent credentials, so actions taken on behalf of a user carry that user’s permissions.
  • Validation of every tool argument in code: recipient allowlists, amount caps, and field formats.
  • Tool outputs and retrieved documents treated as data, never as instructions.
  • A log of every tool call with inputs and outputs, plus alerts on unusual volume.
  • A kill switch that stops the agent without a deploy.

How to measure an agent

We recommend tracking four numbers from day one, against today’s baseline.

  1. Task success rate. The share of tasks completed correctly, on a fixed set of past cases with known outcomes and on a sample of live tasks. Our guide to LLM evaluation sets covers how to build that case set.
  2. Human edit rate. The share of agent outputs a reviewer changed before approving, and by how much. A falling edit rate in a category is the evidence for relaxing its approval gate.
  3. Escalation rate. The share of tasks handed to a person. Near zero on a messy process usually means the agent is overconfident. Very high means there are no savings yet.
  4. Cost per task. Model tokens, API calls, and reviewer minutes. The fair comparison is the full cost of the manual process, not zero.

Each test case is worth several runs. The tau-bench paper introduced the pass^k metric for exactly this reason. In its 2024 results, the best function-calling agents it tested succeeded on fewer than 50% of tasks.

Repeated runs showed a wider gap: pass^8 in the retail domain was under 25%. One successful run says little about the next one.

Why start with one process end to end

The model call is the easy part. The work is in integrations, permissions, the oversight policy, logging, and the runbook for when the agent is wrong. Doing that once, from trigger to final outcome, teaches more than a handful of half-built agents and produces real edit, escalation, and cost numbers.

One process also bounds the risk: the team knows which tools exist, which data the agent reaches, and who reviews what. The second process then reuses the plumbing.

Where to start

We recommend this sequence:

  1. One process with the traits from the matrix, plus a record of today’s volume, handling time, and error rate.
  2. A written runbook a new hire could follow: inputs, decisions, actions, and exceptions.
  3. Past cases with known outcomes, including known edge cases.
  4. A list of every tool and action, each marked as reversible or not and internal or external, with an oversight mode assigned.
  5. Shadow mode first, with step caps, spend caps, and a kill switch already in place. The agent proposes, people work as usual, and the team compares the two.
  6. Gates relaxed per action only when edit and error rates meet thresholds agreed in advance.

For teams that want help taking one process through these steps, our AI agent service automates a single workflow end to end in 3 to 6 weeks, with human approval on the steps you choose.