Every software vendor now seems to sell “agents”. Your CRM has one, your helpdesk has one, and someone on your team has seen a demo where an agent books meetings, answers customers and updates the books on its own. It’s hard to tell what is genuinely new, what is a chatbot with a new label, and whether any of it belongs in your company.
This guide explains what an AI agent actually is, what it’s made of and how it handles a real task, step by step. It also covers where agents earn their keep and where a simpler setup does the job better.
The short definition
An AI agent is a language model that works towards a goal by choosing and using tools in a loop. It looks at the situation and decides on the next step. Then it uses a tool (searches the order system, reads a document, drafts an email), checks what came back and repeats until the job is done or it needs a person.
Three things make something an agent:
- A goal, not a script. You tell it what “done” looks like (“resolve this delivery complaint”), not every step to get there.
- Tools. It can look things up and take actions in your systems through connections you give it. Without tools, a model can only write text.
- A loop. It decides the next step based on what it just found. Two similar requests can take different paths.
The agent loop
- Observe Read the request and what it knows so far
- Plan Pick the next step
- Act Use a tool: search, read, draft
- Check Did it work? Is the goal met?
The loop is the important part. A chatbot answers one message at a time and waits. An agent keeps going on its own: look, plan, act, check, and round again.
Agent, chatbot or AI workflow?
The word “agent” now gets attached to almost anything with AI in it. Gartner even has a name for vendors relabelling old chatbots and automation tools as agents: “agent washing”. Three kinds of setup get mixed up most often:
- A chatbot (or assistant) answers questions in a conversation. It may search a knowledge base, but it doesn’t go off and do things in your systems.
- An AI workflow is a fixed sequence of steps you designed, with AI doing one or two of them: reading an invoice, sorting an email, writing a summary. The path is the same every time.
- An AI agent chooses its own steps towards a goal, using the tools you allow.
Chatbot, AI workflow or agent?
| Chatbot | AI workflow | AI agent | |
|---|---|---|---|
| What it does | Answers questions | Runs your fixed steps; AI does one or two | Works towards a goal |
| Who decides the steps | The user, message by message | You, in advance | The model, as it goes |
| Acts in your systems | No | Yes | Yes |
| Copes with cases nobody planned for | Partly | No | Yes |
| Predictable cost and time | Yes | Yes | Partly |
| Easy to audit | Partly | Yes | Partly |
| Typical example | Website FAQ assistant | Invoice fields read from PDFs | Resolving a delivery complaint |
None of these is better in general. A fixed workflow is cheaper, faster and easier to check, which is why most sensible automation is still workflows. We cover the three levels in depth, with plenty of examples, in Workflows, AI workflows and AI agents.
What an agent is made of
Under the hood, every agent has the same parts. When a vendor shows you an agent, ask about each one.
What an agent is made of
- Guardrails Permissions, approvals, step and cost limits, logs
- Instructions Role, goal, rules, tone
- Model Reads everything and picks the next step
- Memory and context The request, earlier steps, documents it looked up
- Tools Read and write actions it may ask for
- Your systems Orders, documents, email, database
The model is the reasoning engine: one of the current models from OpenAI, Anthropic or Google, or an open-weight model such as Llama, Mistral or Qwen that you can run on your own servers. It reads everything in front of it and decides what to do next.
Instructions set its role, the goal, the rules and the tone. For example: “You handle delivery complaints for a building-materials distributor. You may prepare replacement orders, but never approve them.” Good instructions are specific and short. Vague instructions produce a vague agent.
Tools are the actions it can take: functions your developers expose, such as find_order or draft_credit_note. Each has a name, a description and the inputs it expects. The model asks for a tool; your code checks the request and runs it. We explain the mechanics in How AI agents use tools.
Memory and context is everything the model can see while it works: the request, the conversation so far, tool results and relevant documents. Models don’t remember anything between runs on their own. If you want the agent to know what happened with this customer last month, you have to build that in, usually by giving it a tool to look it up.
Guardrails are the limits around it: which tools it may use, which actions need a person’s approval and how many steps it may take. They also cap what one task may cost and keep a log of every step. This is the part demos skip and real systems depend on.
A worked example, step by step
Here is a realistic task for a building-materials distributor with a small customer service team. Complaints about deliveries arrive by email every day. Each one takes a person 15–20 minutes: find the order, check the delivery note, check stock, prepare a replacement, ask accounting for a credit note, reply to the customer.
The agent receives this email: “Order 4471 arrived yesterday with two boxes of tiles missing. Please send them and fix the invoice.”
- You: Order 4471 arrived yesterday with two boxes of tiles missing. Please send them and fix the invoice.
- Tool call: orders.find(id: 4471)
- Result: 12 boxes of tiles · delivered yesterday
- Tool call: delivery_notes.get(order: 4471)
- Result: driver’s note signed for 10 boxes
- Tool call: stock.check(item: tiles, qty: 2)
- Result: 2 boxes available · main warehouse
- AI: Drafted: a replacement order for 2 boxes, a credit note request and a reply to the customer. Waiting for approval.
- Petra: Checked and approved. The reply has gone to the customer.
What happened:
- It read the goal: resolve a complaint about missing goods on order 4471.
- It looked up the order with a read-only tool: 12 boxes of tiles, delivered yesterday.
- It checked the delivery note. The driver’s signed note shows 10 boxes, so the customer is right.
- It checked stock for the same tiles. Two boxes are available in the main warehouse.
- It prepared a replacement order and a credit note request, but only as drafts. Its instructions say a person approves anything that ships goods or changes an invoice.
- It drafted a reply and put everything in front of Petra in customer service. She checked it and approved it with one click.
This is what makes it an agent rather than a workflow: the path depended on what it found. If the delivery note had shown all 12 boxes signed for, the agent would have taken another route, asking the customer for photos and flagging the case for the warehouse. To get the same behaviour from a workflow, someone would have to draw every branch in advance.
Where agents are useful
Agents earn their cost when a task looks like this:
- The steps change from case to case, so a flowchart would sprawl.
- Several systems are involved: an order system, a document store, email, a database.
- Each case takes a person 10 minutes or more of looking things up before they can decide.
- The result can be checked before it does any harm, by a person or by rules.
Some examples from ordinary companies:
- An accounting firm at month end. The agent goes through a client’s bank transactions that have no matching document, searches the document inbox, and drafts one list of questions for the client instead of ten separate emails.
- A recruitment agency. The agent prepares a short brief on each candidate from the CV, interview notes and past placements, so the recruiter starts the call informed. The recruiter makes every decision.
- A team that answers the same data questions every week. An assistant decides which database queries to run and explains the answer. Shovels’ Charlie works like this: it answers questions about construction permits by querying live database records.
- An IT or operations helpdesk. The agent gathers logs, recent changes and similar past tickets, and hands a technician a summary with a suggested next step.
Where agents are overkill
Most tasks in most companies don’t need an agent:
- The steps are the same every time, such as routing invoices for approval by amount. A plain workflow does it for a fraction of the cost.
- There’s one smart step in an otherwise fixed process, such as sorting incoming emails. That’s an AI workflow.
- People just need answers from your documents, such as an internal handbook. That’s an assistant that searches your documents, not an agent.
- The action can’t be undone and nobody checks it: payments, deleting data, sending contracts.
- The volume is tiny. Five cases a month rarely justify building and maintaining anything.
For a fuller decision guide, with questions about the cost of mistakes, volume and audit, read When to use AI agents.
Four common misunderstandings
“The agent learns by itself.” It doesn’t. The model doesn’t change while you use it. An agent gets better because people review its mistakes and improve its instructions, tools and test cases.
“An agent replaces an employee.” In practice it takes over one task, usually the tedious middle part: gathering information and preparing a draft. The person still decides and handles the cases the agent can’t.
“More autonomy is better.” Autonomy is a setting you choose for each action. Start with the agent preparing drafts that a person approves. Once you trust it, let it act alone on low-risk actions, as we describe in our post on keeping people in the loop.
“It’s just a smarter chatbot.” The difference is that an agent acts. Once it can change things in your systems, permissions, approvals and logs matter as much as the model.
What to do next
If you have a task in mind, work through this list before you talk to any vendor:
Before you build an agent
- Write the goal in one sentence And describe what “done” looks like
- List the systems it needs What it must read, and what it must change
- Mark the risky actions Anything that sends, pays, ships or deletes needs a person’s approval
- Collect 20–30 real past cases Including the awkward ones, to test against
- Measure today’s effort Minutes per case and cases per month
- Name an owner Someone who reviews early runs and decides on changes
If you’d like a second opinion on whether your process needs an agent, an AI workflow or just a few good rules, book a 30-minute call. You’ll leave with a suggested first step and a rough price.
