What Is an Agent?
What separates an agent from a chatbot and a workflow, the four parts every agent has, and the honest guide to when you should not build one
What Is an Agent?
TL;DR
An agent is a language model that decides its own next step, in a loop, using tools, until it reaches a goal. The word that matters is decides. A fixed sequence of model calls is a workflow, however clever it is. An agent chooses its own path while running — and that single property is the source of all its power and all its problems.
| Property | Value |
|---|---|
| Level | Beginner — start here |
| Reading time | ~20 minutes |
| Prerequisites | None. If you have used a chatbot, you can read this. |
| You will understand | What an agent is, what it is made of, and when not to build one |
Start From What a Model Cannot Do
A plain LLM call is one shot: text in, text out. That model:
| Cannot | Why |
|---|---|
| Look anything up | It only knows what was in its training data |
| Take an action | It produces text. It cannot send an email or query a database. |
| Know today's date reliably | Its knowledge stopped on some date |
| Check its own work | One pass, no revision |
| Handle a task with unknown steps | It answers once; it cannot go and find out more |
Ask "what was our Q3 revenue in Europe?" and a plain model has three options: refuse, guess, or hallucinate. It has no path to the answer.
An agent gives it a path.
The Four Parts
Every agent, no matter how simple or complex, has these four:
What an agent is made of
The model
Tools
The loop
Stopping rule
Remove any one and you no longer have an agent:
| Remove | What you get |
|---|---|
| The tools | A chatbot that talks to itself |
| The loop | A single model call with function calling |
| The stopping rule | Something that runs until your bill stops it |
| The decision-making | A workflow |
Chatbot vs Workflow vs Agent
This is the distinction people get wrong most often, and it has real money attached.
Three different things
Chatbot — one call, one reply
Text in, text out. Cheap, fast, predictable, easy to test. Use it when the answer is already in the model — writing, summarising, explaining, rewriting.
Workflow — fixed steps, always the same order
RecommendedYou wrote the steps. The path never varies. Classify, then look up, then answer. Still predictable, still testable, still debuggable — and it can use tools. Use it whenever you know the steps in advance.
Agent — the model picks the path
The model decides what to do next based on what it just learned. Powerful, and not predictable. Different path on every run, 3–20× the cost, and genuinely hard to debug. Use it only when you cannot know the steps in advance.
Side by side
| Chatbot | Workflow | Agent | |
|---|---|---|---|
| Who decides the steps | Nobody — one step | You, in advance | The model, at run time |
| Same input, same path? | Yes | Yes | No |
| Uses tools | No | At fixed points | Chooses when and which |
| Typical cost | 1× | 2–4× | 3–20× |
| Typical latency | 1× | 2–3× | 5–20× |
| Can you unit test it | Easily | Easily | Only statistically |
| Failure is | Obvious | Obvious | Often silent |
Most systems marketed as agents are workflows, and that is a compliment. If you can draw the flowchart in advance, build the flowchart. You get something cheaper, faster, and testable, and you keep the ability to find out why it did what it did. Choosing a workflow is not settling for less — it is picking the right tool.
When You Genuinely Need an Agent
The honest test is one question: can you write down the steps before you start?
| Task | Steps knowable in advance? | Build |
|---|---|---|
| "Summarise this document" | Yes — one step | Single call |
| "Classify this ticket, then route it" | Yes — two fixed steps | Workflow |
| "Answer from our help articles" | Yes — search, then answer | Workflow (this is RAG) |
| "Compare our Q3 numbers to last year" | Mostly — but the second lookup depends on the first | Simple agent |
| "Research this company and flag risks" | No — what you look up next depends on what you find | Agent |
| "Debug why this test fails" | No — every finding changes the next move | Agent |
Notice the pattern. Agents earn their cost when the next step depends on the result of the previous one in a way you cannot predict. If step 2 is always the same regardless of what step 1 returned, you have a workflow, and building it as an agent just adds cost and unpredictability.
The Trade You Are Making
Choosing an agent buys you flexibility and costs you four things. Know them going in.
| You gain | You lose |
|---|---|
| Handles tasks you could not script | Predictability — different path every run |
| Recovers from surprises | Testability — you can only measure it statistically |
| Works on open-ended goals | Cost control — a hard task can cost 20× an easy one |
| Uses many tools flexibly | Debuggability — you must reconstruct what it did from a trace |
The loss of predictability is the one that surprises teams in production. A bug you cannot reproduce is a bug you cannot fix easily. When the same input can take a different path each run, "it worked when I tried it" stops being evidence of anything.
A Very Small Example
The whole idea in one trace:
Task: "What was our Q3 revenue in Europe versus last year?"
Thought: I need this year's figure first.
Action: get_revenue(region="Europe", quarter="Q3", year=2025)
Observe: 4,820,000 EUR
Thought: Now I need last year's, same quarter.
Action: get_revenue(region="Europe", quarter="Q3", year=2024)
Observe: 4,110,000 EUR
Thought: I have both. I can answer now.
Answer: €4.82M, up 17.3% from €4.11M.The second action could not have been written in advance — well, in this simple case it could. But change the task to "tell me about any region that declined" and it genuinely could not, because which regions to look up depends on what the first query returned. That is the line between workflow and agent.
See it running: the Agent Loop page has an interactive version of this trace.
Common Misconceptions
Things people get wrong
'The agent runs the tools'
It does not. The model outputs a request — a piece of structured text naming a tool and arguments. Your code decides whether to run it. This is the most important thing to understand about agents, because it is where every safety control lives.
'More agents means better results'
Usually the opposite. Each agent multiplies cost and adds a new place for things to go wrong — including agents disagreeing with each other. One good agent beats five badly-coordinated ones.
'Agents learn from experience'
Not on their own. The model's weights do not change. Anything remembered between runs is a database you built and chose to load into the prompt.
'A smarter model fixes agent problems'
Sometimes, but rarely. Most agent failures are mechanical — a tool that never reached the model, a vague description, an error string read as data. Check the boring causes before paying for a bigger model.
'Agents are for replacing people'
The systems that work in production almost always keep a human at the point where actions become irreversible. Designing the handoff is part of the job, not an admission of failure.
Concept Checks
Check yourself
What is the single test for whether you need an agent rather than a workflow?
Whether you can write down the steps before running anything. If step 2 is the same regardless of what step 1 returned, the path is fixed and a workflow will do it cheaper, faster, and testably. You need an agent only when the next action genuinely depends on a result you cannot predict — like deciding which regions to investigate based on which ones showed a decline.
Why does it matter that the model does not run tools itself?
Because it means there is always a checkpoint between intention and action, and that checkpoint is code you control. The model emits a structured request; your code decides whether the tool is allowed, whether the arguments are sane, and whether a human should approve first. Every guardrail in agent safety lives at that boundary. If you wire model output directly into an action, you have removed the only place you could have stopped it.
Your agent works perfectly in testing. Why is that weaker evidence than it looks?
Because agents are not deterministic — the same input can take a different path on the next run. A passing test shows that one path worked once, not that the system works. Real confidence comes from running each case many times and looking at the spread of outcomes, costs, and step counts. A single green run mostly tells you the happy path exists.
Someone proposes five specialist agents for a task one agent handles adequately. What is the argument against?
Every added agent multiplies cost and latency, and introduces failure modes that did not exist before: agents contradicting each other, context lost in handoffs, and a coordinator that becomes a single point of failure. Debugging also gets much harder, because a wrong result now has five possible origins plus the joins between them. Extra agents should follow a measured failure of the simpler design, not precede it.
Key Concepts Recap
| Concept | One-line summary |
|---|---|
| Agent | A model that decides its own next step, in a loop, with tools |
| The deciding word | Decides — a fixed path is a workflow, not an agent |
| Four parts | Model, tools, loop, stopping rule |
| The model requests | It never runs anything; your code does |
| Workflow first | If you can write the steps down, write them down |
| The test | Can you know the steps in advance? |
| What you trade | Predictability, testability, cost control, debuggability |
| Not learning | Weights do not change; memory is a database you built |
| Boring causes first | Most failures are tools and descriptions, not intelligence |
Next
Now the mechanism that makes any of it possible: Tools & Function Calling.
AI Agents Crash Course
All of AI agents on one page — what an agent is, tools, the loop, planning, memory, multi-agent systems, evaluation, safety, and how to debug them
Tools & Function Calling
How a model asks your code to run a function — schemas, the call cycle, writing descriptions that work, error handling, parallel calls, and MCP