What makes a customer-service system an agent?
An agent keeps context, picks from actions it is allowed to take, uses tools in other systems and follows the result to the end. Vendors use the word loosely, so check all four.
Details
Vendors stretch the word agent across products that have little in common. Intercom calls Fin an AI customer-service agent with help-desk integrations5. Tidio sells its Lyro AI agent on its own or with Tidio6. Zendesk places AI agents inside its service platform7. Same noun, three purchases. Four behaviors make something an agent in practice. It keeps context across the conversation. It chooses among actions it is allowed to take. It uses a tool in another system, like a calendar or an order record. It follows the result to a finished state or to a named person. Census survey data from early 2024 already listed virtual agents among common business uses of AI14. Ask about the four behaviors anyway.
Context
Decision
Tool use
State and follow-through
Five levels of service autonomy
Set authority per task, from suggesting a reply to running a multi-step fix. Each level needs its own controls, and many businesses get most of the value from the lower ones.
Details
Grant authority one task at a time, and treat each level as a risk choice. Suggest: the system proposes an answer to your staff, and nothing reaches the customer. Draft: it writes a reply a person sends. Act with approval: it prepares a change, like an auto shop refund, and waits for a yes. Act within policy: it books or reschedules alone, inside limits you wrote. Coordinate: it runs a multi-step fix across systems. The lower rungs have the best evidence. At one firm, an AI assistant raised the issues support agents resolved per hour by 15%, with people still answering10. Expect help, not instant replacement: a Danish working paper found no measurable effect on earnings or hours two years after ChatGPT arrived12.
Suggest
Draft
Act with approval
Act within policy
Coordinate multi-step resolution
Open Five-level autonomy model
Five-level autonomy model
| Question | How to check |
|---|---|
| Suggest | Check that proposed answers reach only your staff, and that nothing goes to a customer unless a person chooses it. |
| Draft | Check that every drafted reply waits for a person to send it, and that staff can edit it first. |
| Act with approval | Ask for a change such as a refund and confirm it waits for a named person's yes before anything moves. |
| Act within policy | Try a booking just inside your written limits and one just outside, and check that only the first goes through. |
| Coordinate multi-step resolution | Ask the vendor to show a fix across two systems, and what happens when the second step fails halfway. |
Good enough when, the same for every row above: Record the authoritative result or preserve context for the named human handoff. Do not count generated text as resolution.
Show autonomy-level details
Each row compares the same thing. A blank means the vendor's own page does not say.
- Suggest: Check that proposed answers reach only your staff, and that nothing goes to a customer unless a person chooses it.
- Draft: Check that every drafted reply waits for a person to send it, and that staff can edit it first.
- Act with approval: Ask for a change such as a refund and confirm it waits for a named person's yes before anything moves.
- Act within policy: Try a booking just inside your written limits and one just outside, and check that only the first goes through.
Verify every claimed action
Make the vendor show the result in the other system. A message saying the booking is done proves nothing until the calendar, ticket or record shows it.
Details
A message saying an appointment is booked needs a matching calendar record. Ask the vendor to show the starting state, make the permitted change and inspect the result in the other system. Then test a job that requires the customer to act too. Can the assistant explain one step, check what happened and adjust? The 2025 tau2-Bench research separates those skills: its simulated telecom tasks require both the agent and the user to change the environment, and success is checked against the final state3. This is a research benchmark, not evidence about a product you can buy. Use its distinction in your demo: check both the system change and the customer's part in completing it.
Read
Write
Confirm
Recover
Audit
Open Action-verification worksheet
Action-verification worksheet
- 1. ReadCan it see the current state, such as today's open slots? Compare what it reports with the calendar yourself.
- 2. WriteCan it make a change you can reverse? Make a test booking, then cancel it, and check both steps in the calendar.
- 3. ConfirmDoes the calendar, ticket or record show the change, beyond a chat message saying it happened?
- 4. RecoverWhat happens after a failed write? The visitor should hear that nothing is held and what happens next.
- 5. AuditWhat log survives the chat? Find the record of a test action a day later, with the time and what changed.
The table above is the accessible HTML version. Downloads contain the same blank prompts and no hidden scoring.
Design the permission boundary
Write down what the assistant may read, what it may change, the spending limits, the identity checks and who takes over. Enforce those rules outside the AI model itself.
Details
Permissions are the part you fully control, so write them down before launch. Data scope: a contact record is not a payment history, so allow only what the task needs. Action scope: list what it may finish and what it may only prepare. Value limits: set a ceiling on refunds or discounts above which a person signs off. Identity checks: decide what proof comes before any account detail. Rate limits: cap how often one action can fire. Escalation: name the person by role. Enforce these rules in your own systems, outside the AI model, because a model can be talked into things. Your business owns the result: a Canadian small-claims tribunal held Air Canada liable for its chatbot's wrong refund policy13.
Data scope
Action scope
Value limits
Identity checks
Rate limits
Escalation
Open Least-privilege permission matrix
Least-privilege permission matrix
| Question | How to check |
|---|---|
| Data scope | List the fields the bot can read and remove any the task does not need, such as payment history for a booking bot. |
| Action scope | Write which actions it may finish and which it may only prepare for a person, then test one of each. |
| Value limits | Set a ceiling on refunds or discounts, then ask for one above it and check that a person must sign off. |
| Identity checks | Decide what proof comes before any account detail, then ask about an account without it and confirm a refusal. |
| Rate limits | Cap how often one action can fire, then repeat the request quickly and check that the cap holds. |
| Escalation | Name the person by role, such as office manager, and check that escalations reach that role. |
Good enough when, the same for every row above: Record the authoritative result or preserve context for the named human handoff. Do not count generated text as resolution.
Failure modes and recovery
Test timeouts, half-finished actions, conflicting records, the wrong customer, policy exceptions and hidden instructions. The customer should get an honest status and a named person.
Details
Plan for the tool that does not answer, because that case will come. A timeout should end with a plain status message and a retry that cannot book twice. Partial completion is the dangerous one: the job written into the schedule, the confirmation text never sent. Conflicting records need a rule about which system wins. The wrong customer is worse than no action, so check identity before any change. Policy exceptions, like a disputed late-cancel fee at a salon, go to a person by design. Prompt injection is the newest risk. A message or web page can carry hidden instructions, so treat every input as data, never as orders. No product blocks every trick. Review a sample of failed chats each week.
Tool timeout
Partial completion
Conflicting records
Wrong customer
Policy exception
Prompt injection
Open Failure and recovery cards
Failure and recovery cards
- Tool timeout
- The calendar does not answer in time. The chat gives a plain status message and retries in a way that cannot book twice.
- Partial completion
- The job is in the schedule but the confirmation text never went out. Check each step, then finish or report the missing one.
- Conflicting records
- The CRM and the calendar disagree about an appointment. A written rule says which system wins.
- Wrong customer
- Acting on the wrong account is worse than doing nothing, so the bot checks identity before any change.
- Policy exception
- A salon client disputes a late-cancel fee. The bot does not decide; the case goes to a person by design.
- Prompt injection
- A message or web page carries hidden instructions. Treat every input as data, never as orders, and review failed chats weekly.
Agent versus chatbot versus workflow
Run one customer request through an agent, a chatbot and a fixed workflow. See what each can decide, carry out, check and leave to a person. Many good setups combine them.
Details
Fixed steps belong in fixed automation, and only the variable part needs AI. A scripted workflow can be tested branch by branch and never improvises, which is both its value and its limit. An AI agent interprets the request, so it handles a question nobody wrote down and can also be wrong in a new way. The chat window is the conversation surface, and it can sit in front of either. Many products blend the two; ChatBot.com describes itself as an AI chatbot and workflow platform8. Picture a plumber's website chat: the agent works out whether water is leaking right now, then a fixed workflow alerts the on-call tech. Buy interpretation for the variable part only.
Conversation surface
Probabilistic interpretation
Deterministic automation
How to evaluate products
Send every vendor the same cases and pass rules. Check the deal-breakers first, then supported actions, records, the staff workspace, reporting, cost unit and security.
Details
Disqualifiers come first, then the weighted score. Rule out any product that cannot show a completed action in your real system, has no path to a person, or will not say where your data is stored. Then score what is left. Supported actions, named one by one. The system that holds the record afterward. The staff workspace, and who covers it on a Friday evening. Logs, transcripts and samples you can review. The cost unit, which varies a lot: Intercom bills Fin per outcome plus seat charges4, Tidio counts AI conversations6, and Zendesk pairs seats with automated-resolution packaging7. Last, security and data retention. A demo tests the vendor's rehearsal. Your own cases test the product.
Supported actions
System of record
Human workspace
Observability
Cost unit
Security and compliance
Open Vendor scorecard
Vendor scorecard
- 1. Supported actionsGet the actions named one by one, then watch each finish in your real system during a trial, not in a recorded demo.
- 2. System of recordName the tool that holds the booking, lead or note afterward, and confirm the record actually appears there after a test chat.
- 3. Human workspaceSee where staff pick up a handed-off chat and who covers it on a Friday evening. A product with no path to a person is ruled out.
- 4. ObservabilityAsk for logs, transcripts and sample conversations you can review yourself. If you cannot read what the bot said, you cannot fix it.
- 5. Cost unitWrite down the exact event that counts toward the bill: a Fin outcome at Intercom, an AI conversation at Tidio, seats plus automated resolutions at Zendesk.
- 6. Security and complianceAsk where your data is stored and how long it is kept. A vendor that will not say is out, whatever it scored elsewhere.
The table above is the accessible HTML version. Downloads contain the same blank prompts and no hidden scoring.
Where First AI Employee fits
First AI Employee is a managed website assistant that answers, qualifies and books. Do not read help-desk work, account changes or ticket updates into it.
Details
Website actions it publicly supports
First AI Employee answers visitors' questions about the business, qualifies inquiries and collects contact details1. It checks a connected calendar and confirms a booking only when the booking succeeds; if it cannot finish, it collects the details instead. On the autonomy scale above, booking is the one action that changes another system, and it comes with that confirmation check. That fits a salon or an auto shop taking appointments through the website.
Do not infer unsupported autonomy
Do not read more autonomy into it than the official pages state. It is a managed website assistant for questions, lead qualification and booking, not a help desk or ticketing suite1. Google review replies are not available yet. The official pages do not describe refunds, account changes or ticket updates. The rule on this page applies here too: the label does not settle the actions, so ask to watch a live booking.
Frequently asked questions
Answers to common questions about agents, autonomy and control. For a specific action or integration, check the vendor's current page and ask for a live test.
Details
Frequently asked questions
What is an AI customer-service agent?
It is software that works out what a customer wants, uses the information you approved, and may pick and run tools to move the request toward a finished result. The useful definition is the exact action, permission, confirmation and handoff, not the word agent.
Can an AI agent issue refunds or change accounts?
Some systems can, once they are connected to those tools. Actions with real consequences need a check on who the customer is, clear limits, confirmation from the system that holds the record, a log and a way to undo mistakes. A vendor saying it has an integration does not prove every action works.
What is an autonomous customer-service agent?
The term usually means the system can do more than draft an answer, but nobody agrees on a standard for how much it may do alone. Ask for three lists: actions that need your approval, actions it may take within your rules, and actions it may never take.
How do I test an AI agent?
Test normal work, vague requests, the limits of its permissions, tool errors, the same request sent twice, records that disagree, messages that try to trick it, and handoff to a person. Then check the system that holds the record, instead of taking the agent's word that the action worked.
Is First AI Employee an AI customer-service agent?
First AI Employee offers a website assistant that answers visitors' questions about the business. It qualifies inquiries and collects visitors' contact details. It checks a connected calendar and confirms a booking only when the booking succeeds; if it cannot finish, it collects the details instead. Other channels and account changes are not on its official pages.