Where Agents Shine — and Where They Fail
The honest map. Agents are spectacular at some tasks and quietly dangerous at others. Knowing the difference is the single most valuable skill in this whole course.
🎯 By the end you'll understand
The kinds of tasks agents are genuinely great at, the kinds where they're risky or useless, a five-question fit test you can run on any task in ten seconds, and why "keep a human in the loop on the high-stakes steps" is the rule that keeps you safe as agents get more capable.
KEY INSIGHTThe one idea to take away
An agent is only as good as your ability to check its work. Where verification is cheap, agents shine. Where it's impossible — or a mistake can't be undone — they're a liability.
You've spent this stage learning what agents are. This lesson is about the harder, more useful question: when should you actually use one? The exciting demos all share a hidden ingredient — a clear, checkable goal. The horror stories all share the opposite. Learn to spot the difference and you'll get value out of agents while everyone else is either over-trusting them or scared of them.
Agents don't fail at "hard" tasks and succeed at "easy" ones. They succeed at verifiable, reversible, well-tooled tasks and fail at fuzzy, irreversible, high-stakes ones — regardless of difficulty. A coding agent can fix a gnarly bug (verifiable: tests pass) but shouldn't wire a payment (irreversible: money moves).
CORE CONCEPTWhere agents shine
The tasks agents are great at share a family resemblance. Look for these five traits — the more of them a task has, the brighter an agent will shine on it:
A clear, checkable goal
"All tests pass," "the report covers these five questions," "the data is deduplicated." The agent — and you — can tell when it's done.
Decomposable into steps
The work breaks into a sequence the agent can grind through: search, then read, then draft, then revise. The loop has somewhere to go.
Good tools exist
There's a real way to act: a test runner, a search API, a database, a file system. No tools, no traction (you saw why in Lesson 1.1).
Output is verifiable
You can confirm the result is right without redoing the whole task yourself — run the tests, eyeball the table, click the link.
Mistakes are reversible
If it gets something wrong, you can undo it cheaply. A bad draft is deleted; a bad bank transfer is not.
Notice that "repetitive but variable" work hits the sweet spot. Tasks that are too repetitive belong to a plain script (you'll see why in Lesson 1.5). But work that repeats with twists each time — every bug is different, every dataset is messy in its own way — is exactly where an agent's flexible reasoning earns its keep.
Coding with tests — the test suite is a built-in verifier; the agent writes, runs, sees red, fixes, sees green. Research & synthesis — search many sources, pull out the relevant bits, draft a summary you can fact-check. Data wrangling — clean, reshape, and join messy files where you can spot-check the output. Triage & routing — read a queue of tickets or emails and tag, sort, or escalate them. First drafts — code, docs, emails, outlines: anything where a human edits the result before it counts.
The common thread: in every one of these, checking the answer is far cheaper than producing it. That asymmetry is the agent's home turf. It does the slow part; you do the fast part — confirming it's right.
WEAKNESSWhere they fail — or are quietly dangerous
Flip every trait above and you get the danger zone. These are the tasks where agents disappoint, mislead, or do real damage — and the dangerous ones don't announce themselves. The agent will sound just as confident when it's wrong.
- Fuzzy or unstated goals. "Make this better," "do something about the website." With no clear target, the agent picks a direction — often the wrong one — and pursues it with total confidence.
- No way to verify the result. If you can't tell whether the output is right without becoming the expert yourself, the agent has handed you confidence, not correctness. That's worse than no answer.
- High-stakes, irreversible actions. Moving money, signing contracts, medical or legal decisions, deleting data, sending the email to the whole company. If "undo" doesn't exist, an agent shouldn't have the final click.
- Real-world judgment, taste, or accountability. Firing someone, pricing a deal, choosing a brand voice, deciding what's tasteful. Someone has to own these — and an agent can't be held accountable.
- Where being confidently wrong is catastrophic. An agent's failure mode isn't "I don't know." It's a fluent, plausible, wrong answer delivered with full conviction. In some contexts that costs a typo. In others it costs a lawsuit.
- Novel problems with no tool support. If there's no way to act and no way to check, the loop has nothing to push against. The agent just talks — and now you're back to a chatbot wearing an agent's costume.
The most dangerous tasks aren't the ones agents refuse. They're the ones agents do eagerly and badly — fuzzy goals with confident output and no verifier. The agent looks like it succeeded. You won't find out otherwise until the bill arrives.
COMPARISONShines vs fails, side by side
The same dimension flips an agent from asset to liability. Read each row across — the difference is rarely the difficulty of the task, it's the shape of it:
| Dimension | Shines ✅ | Fails / risky ⚠️ |
|---|---|---|
| Goal | clear and checkable | fuzzy, unstated, "make it nice" |
| Verification | cheap — run tests, eyeball it | impossible without being the expert |
| Reversibility | undo is easy and free | permanent — money, deletion, legal |
| Tools | real tools to act & check | none — nothing to push against |
| Stakes | low — a bad draft is deleted | high — a confident error is costly |
| Example | fix a bug until tests pass | approve a $40k wire transfer |
If you remember nothing else, remember the two example rows. "Fix the bug until tests pass" and "approve the wire transfer" are the same agent, same intelligence — but one is a perfect fit and the other is malpractice. The task's shape decides, not the agent's smarts.
FRAMEWORKThe five-question fit test
You don't need a spreadsheet to decide whether to hand a task to an agent. You need five yes/no questions you can run in your head in under ten seconds:
Clear goal?
Can you state exactly what "done" looks like — in one sentence the agent couldn't misread?
Verifiable result?
Can you confirm it's right without redoing the whole thing yourself?
Reversible if wrong?
If it botches it, can you undo the damage cheaply?
Good tools exist?
Is there a real way for it to act and check — a test runner, an API, a file system?
Bounded scope?
Is the task fenced — a known set of steps — rather than "go fix the whole business"?
More "yes" = better agent fit. Five yeses and you should reach for an agent without hesitation. Zero or one and you should do it yourself — or rethink the task until it gets a better score. The middle is where judgment lives: use an agent, but supervise the risky step.
Crucially, the test is a design tool, not just a gate. If a task scores low, you can often reshape it into a high score: add a verifier (write a test first), shrink the scope (one file, not the repo), or insert a human approval before the irreversible step. Turning a "no" into a "yes" is most of the skill in Stage 3.
SEE ITAgent Fit Checker
Run the fit test on a real task. Toggle Yes / No for each question and watch the verdict update. Try it on something you actually want to delegate — then try it on "approve a wire transfer" and watch it collapse.
CONTEXTThe honest meta-point
Here's the thing this course will keep being straight with you about: agents are probabilistic, not deterministic. The same request can produce a slightly different path each time, and "usually right" is not "always right." That's not a bug you'll patch in a later lesson — it's the nature of the LLM brain inside (Lesson 1.3).
This is a "zero to capable" course, not "zero to reckless." Capable means you know where the edges are. The reason the fit test matters is that it tells you exactly where to keep a human in the loop: at the high-stakes, irreversible, hard-to-verify steps. Let the agent do ninety steps of grinding work — then put a human between it and the one click that can't be taken back, approving it before it happens.
Human-in-the-loop means a person must approve each consequential or irreversible action before it happens — an approval gate on the high-stakes steps. Human-on-the-loop means the agent runs largely autonomously while a person monitors and can intervene or stop it, but doesn't approve each action. The gating pattern that keeps high-stakes work safe is human-in-the-loop; on-the-loop is the lighter-touch, monitoring version. Both are practical middle grounds between "fully manual" and "fully autonomous," and you'll go deep on them in Lesson 3.4.
None of this is a reason to avoid agents. It's the opposite — it's permission to use them boldly where they shine, precisely because you know where they don't. Confidence and caution aren't opposites here. They're the same skill, applied with judgment.
WEAKNESSCommon mistakes to avoid
Demos are chosen because they score high on the fit test — clear goal, easy to verify. Your real task may be fuzzier and higher-stakes. Run the five questions on your task, not the demo's.
"Done" from an agent means "I stopped," not "it's correct." If you skipped the verification question, you've outsourced the work and the checking to a thing that can be confidently wrong. Always own the verify step.
The goal is the right autonomy for the task. On a low-score task, dialing up autonomy just lets the agent make a bigger, faster mistake. Sometimes the smartest move is to keep the human firmly on the loop — or not use an agent at all (Lesson 3.11).
KEY INSIGHTWhere this connects in your course
- From chatbot to agent (1.1) — agents shine because they can act and check; no tools, no traction.
- Agent vs automation vs assistant (1.5) — "too repetitive" belongs to a script; "repetitive but variable" is the agent's sweet spot.
- The trust gradient (3.4) — how to keep a human in the loop, approving exactly the high-stakes steps (and when monitoring on-the-loop is enough).
- When NOT to use an agent (3.11) — the full version of this lesson's danger zone, with real decision rules.
- Failure modes (2.9) — why agents fail the way they do, once you've seen the loop from the inside.
This lesson closes Stage 1. You now know what an agent is, what it isn't, and — most importantly — when to reach for one. Next you'll open the hood. Stage 2 shows you how an agent actually works under the surface, starting with the heartbeat of the whole thing: the loop.
GLOSSARYKey terms introduced
- Verifiability
- How cheaply you can confirm a result is correct without redoing the task. High verifiability is an agent's best friend.
- Reversibility
- How easily a mistake can be undone. Reversible tasks are safe for agents; irreversible ones need a human gate.
- Stakes
- The cost of being wrong. Low-stakes work tolerates agent errors; high-stakes work demands human accountability.
- Decomposable
- Breakable into a sequence of steps the agent can work through one at a time. Gives the loop somewhere to go.
- Fit (agent fit)
- How well a task suits an agent, scored by the five-question test. More "yes" answers = better fit.
- Human-in-the-loop
- A person must approve each consequential or irreversible action before it happens — an approval gate on the high-stakes steps.
- Human-on-the-loop
- The agent runs largely autonomously while a person monitors and can intervene or stop it, but doesn't approve each action.
- Probabilistic
- Producing outputs with inherent variability — "usually right," not "always right." The nature of the LLM brain inside.
ACTIVE RECALLCheck your understanding
Answer in your own words first, then reveal.
1What's the single trait that best predicts whether an agent will shine on a task?›
Cheap verification. If you can confirm the result is right far more easily than producing it (run tests, eyeball a table), the agent does the slow part and you do the fast part. Where checking is impossible, an agent gives you confidence, not correctness.
2Name the five questions in the fit test.›
Clear goal? · Verifiable result? · Reversible if wrong? · Good tools exist? · Bounded scope? More "yes" answers means better agent fit — and a low score is an invitation to reshape the task, not just abandon it.
3Why are fuzzy, high-stakes tasks the most dangerous — more than tasks the agent refuses?›
Because the agent does them eagerly and badly. It produces a fluent, confident, wrong result with no verifier to catch it — so it looks like success. You don't discover the error until the irreversible damage is done.
4What does "keep a human in the loop on the high-stakes steps" mean in practice?›
Let the agent run the many low-stakes, reversible steps on its own — but put a person between it and the consequential, irreversible action (the wire transfer, the deletion, the send-all), approving it before it happens. That's human-in-the-loop. (Its lighter cousin, human-on-the-loop, just monitors and can step in.) It's the middle ground between fully manual and fully autonomous, and it's how agents are used responsibly given they're probabilistic.