The system at a glance

Seven parts carry every inquiry: the chat window, a controller that decides the next step, your approved knowledge, the language model, connected tools, a place where staff pick up handoffs, and reports. The model is only one of them.

Details

Only one of the seven parts is the AI model that buyers hear about. The channel is the chat window on your site. The orchestrator is the controller that decides what happens next. Knowledge is the set of pages and documents you approved. The model writes the words. Tools reach into other systems, such as your calendar. The human workspace is where unfinished work lands for your staff. Analytics tell you later whether any of it worked. Two products can run the same model and behave nothing alike. The gap usually sits in the knowledge, the tool permissions and whether anyone watches the workspace. Ask vendors about those first.

Channel

Orchestrator

Knowledge

Model

Tools

Human workspace

Analytics

Open System architecture diagram · Numbered request sequence

System architecture diagram

System architecture diagramFollow each step in order. Every arrow is a point where a call or chat can go wrong.
System architecture diagramFollow each step in order. Every arrow is a point where a call or chat can go wrong. The complete sequence is repeated as text after the diagram.1Inquiryverify before next2Ground answerverify before next3Attempt actionverify before next4Verify resultverify before next5Handoffrecord outcome
  1. Inquiry: Capture channel, intent and the minimum identity context.
  2. Ground answer: Use an approved source and resolve conflicts before responding.
  3. Attempt action: Apply the permitted workflow with least privilege.
  4. Verify result: Check the downstream state instead of trusting fluent text.
  5. Handoff: Send context, attempted actions and the unresolved need to a person.
Channel
The chat window on your site, where the visitor types and reads replies.
Orchestrator
The controller that decides what happens next: look something up, ask a question, use a tool or hand off.
Knowledge
The pages and documents you approved. Wrong or old material here becomes a wrong answer later.
Model
The AI model writes the words. Two products can run the same model and still behave nothing alike.
Tools
Connections into other systems, such as your calendar. Their permissions decide what the assistant can change.
Human workspace
Where unfinished work lands for your staff. It only helps if someone watches it.
Analytics
Reports that tell you later whether the chat answered, booked or handed off, and where it failed.
Show diagram details

Follow each step in order. Every arrow is a point where a call or chat can go wrong.

  • Channel: The chat window on your site, where the visitor types and reads replies.
  • Orchestrator: The controller that decides what happens next: look something up, ask a question, use a tool or hand off.
  • Knowledge: The pages and documents you approved. Wrong or old material here becomes a wrong answer later.
  • Model: The AI model writes the words. Two products can run the same model and still behave nothing alike.

Numbered request sequence

Numbered request sequenceFollow each step in order. Every arrow is a point where a call or chat can go wrong.
Numbered request sequenceFollow each step in order. Every arrow is a point where a call or chat can go wrong. The complete sequence is repeated as text after the diagram.1Inquiryverify before next2Ground answerverify before next3Attempt actionverify before next4Verify resultverify before next5Handoffrecord outcome
  1. Inquiry: Capture channel, intent and the minimum identity context.
  2. Ground answer: Use an approved source and resolve conflicts before responding.
  3. Attempt action: Apply the permitted workflow with least privilege.
  4. Verify result: Check the downstream state instead of trusting fluent text.
  5. Handoff: Send context, attempted actions and the unresolved need to a person.
Channel
A visitor types a question into the chat window, such as whether a dental office has a Saturday opening.
Orchestrator
The controller reads the request and picks the next step: look up an answer, use a tool or pass it to a person.
Knowledge
The system searches your approved pages and documents for the parts that match the question.
Model
The model writes a reply from those matches. It should not add facts that are not in them.
Tools
If the request needs a change, such as a booking, a tool writes it to the calendar and checks the result.
Human workspace
If any step fails or the visitor asks for a person, the conversation lands in your staff's workspace.
Analytics
The exchange is logged with its outcome, so you can review it later and fix what went wrong.
Show lifecycle-step details

Follow each step in order. Every arrow is a point where a call or chat can go wrong.

  • Channel: A visitor types a question into the chat window, such as whether a dental office has a Saturday opening.
  • Orchestrator: The controller reads the request and picks the next step: look up an answer, use a tool or pass it to a person.
  • Knowledge: The system searches your approved pages and documents for the parts that match the question.
  • Model: The model writes a reply from those matches. It should not add facts that are not in them.

Step 1: receive and normalize the inquiry

The system records where the message came from, what language it is in and what was attached. Anything dropped here, like a photo or the page the visitor was on, is gone for good. A typed claim of identity proves nothing on its own.

Details

Picture a message at 9:40 on a Tuesday night: "still waiting on my quote," plus a photo of a leaking water heater. The system first records the channel details, such as the page the visitor was on and whether they are signed in. It detects the language. It either reads the photo or quietly drops it, and the photo may be the whole message. If your chat takes voice notes, speech-to-text runs first and can add words nobody said. In one study of OpenAI's Whisper in 2023, partly with speakers who have aphasia, roughly 1% of transcriptions held whole invented phrases12. Identity is the trap. Typing "I'm the account holder" proves nothing.

Channel metadata

Language and attachments

Session identity

Step 2: interpret intent and risk

The system guesses what the person wants and how sure it is. A good guess is not permission. Fixed rules you wrote should decide what it may handle, and anything risky or unclear should get a question or a person.

Details

Understanding a request and being allowed to handle it are two separate checks. The model labels the intent, such as "reschedule" or "price question," and pulls out details like a date, an address or an invoice number. That label is a probability, so it comes with a confidence level. Eligibility should be a fixed rule you wrote: is this kind of request on the approved list at all? Risk flags sit on top. A billing dispute at a law firm, a tenant reporting a gas smell to a property manager, or a pet owner describing a seizure to a vet clinic should never be automated. When the system is unsure, it should ask or hand off. Never guess.

Intent

Entities

Eligibility

Risk flags

Step 3: retrieve approved context

The assistant pulls passages from the pages and documents you approved. That keeps answers closer to your facts but does not make them true: old, cut-off or conflicting sources still produce wrong answers13.

Details

Retrieval is where most wrong answers start. The system searches the pages and documents you approved and passes the best matches to the model. Answering from your own material helps, but it is no guarantee. In a 2025 study, legal research tools built to answer from their own document libraries still made things up 17% to 33% of the time13. Your FAQ is far simpler than case law, so that rate will not carry over. The lesson does. Watch for a passage cut off from its exceptions, an old price page that still gets found, and two approved documents that disagree. Give each topic one owner who updates it when policy changes.

Source selection

Chunk retrieval

Freshness

Conflicts

Open Retrieval conflict flow

Retrieval conflict flow

Retrieval conflict flowFollow each step in order. Every arrow is a point where a call or chat can go wrong.
Retrieval conflict flowFollow each step in order. Every arrow is a point where a call or chat can go wrong. The complete sequence is repeated as text after the diagram.1Inquiryverify before next2Ground answerverify before next3Attempt actionverify before next4Verify resultverify before next5Handoffrecord outcome
  1. Inquiry: Capture channel, intent and the minimum identity context.
  2. Ground answer: Use an approved source and resolve conflicts before responding.
  3. Attempt action: Apply the permitted workflow with least privilege.
  4. Verify result: Check the downstream state instead of trusting fluent text.
  5. Handoff: Send context, attempted actions and the unresolved need to a person.
Source selection
The system searches only the pages and documents you approved. Give each topic one owner who updates it when policy changes.
Chunk retrieval
It passes the best-matching passages to the model. A passage cut off from its exceptions can produce a wrong answer.
Freshness
An old price page can still be found and quoted. Remove or update retired pages so they stop matching.
Conflicts
When two approved documents disagree, the assistant should say it cannot confirm and pass the question to a person.

Step 4: compose the response

A language model writes the reply from your instructions, the retrieved text and the conversation so far. It predicts likely wording. It does not understand your business, so test what it says when the source runs out.

Details

The language model writes the reply from your instructions, the retrieved passages and the conversation so far. It predicts likely wording. It does not know your business, and it cannot tell whether a passage is current. Polished text is persuasive. In one study, licensed clinicians preferred chatbot answers to doctors' public forum replies in 78.6% of 585 ratings, but those were written forum answers, and the chatbot's ran about four times longer14. Length and polish read as quality. So test the hard case: ask about something your sources do not cover. A good system says it does not have that answer. Also ask whether each reply can be traced to its source.

Instructions

Context window

Uncertainty

Citation or source trace

Step 5: take an approved action

Take booking as the example. The assistant checks the calendar, validates the details, writes the appointment, and says it is booked only after the calendar confirms. Anything less is a promise with nothing behind it.

Details

Booking is where a chat assistant stops talking and changes something real. Take a salon client who wants a color appointment on Friday. The assistant first checks open slots in the connected calendar. Then it validates what the booking needs: name, phone number, service and time. Permission is checked at the moment of writing, against the limits you set. If the request arrives twice after a slow connection, the system must still create one appointment. Only when the calendar returns a confirmed record should the visitor read "you're booked." If the write fails, the assistant should say the time is not held and take details for a callback. A confirmation with no calendar record is worse than none.

Tool call

Validation

Permission

Idempotency

Authoritative result

Open Booking action sequence diagram

Booking action sequence diagram

Booking action sequence diagramFollow each step in order. Every arrow is a point where a call or chat can go wrong.
Booking action sequence diagramFollow each step in order. Every arrow is a point where a call or chat can go wrong. The complete sequence is repeated as text after the diagram.1Authorizeverify before next2Attemptverify before next3Read resultverify before next4Confirmverify before next5Recoverrecord outcome
  1. Authorize: Check identity, permission and action scope.
  2. Attempt: Send the least-privileged request to the destination.
  3. Read result: Interpret success, conflict, validation and availability responses.
  4. Confirm: Return the verified result and durable reference.
  5. Recover: Preserve context for a person when confirmation fails.
Tool call
The assistant checks open slots in the connected calendar, such as Friday times for a salon color appointment.
Validation
It confirms the booking has what it needs: name, phone number, service and time.
Permission
At the moment of writing, the system checks the booking against the limits you set.
Idempotency
If the request arrives twice after a slow connection, the system still creates only one appointment.
Authoritative result
The visitor reads "you're booked" only after the calendar returns a confirmed record. If the write fails, the time is not held.

Step 6: escalate or recover

When the assistant cannot finish, it should keep the whole conversation, tell the customer what happens next, and pass the request to a person who is there to take it.

Details

Handoff is the step your customers judge you on. Five things should trigger it: the assistant does not know, a tool failed, the customer asked for a person, the situation is risky or heated, or nobody is free to take the request. A good handoff passes along the whole conversation, the details collected and the reason, so nobody makes the customer repeat it. That last trigger is the hard one. A transfer into an inbox nobody reads at six on a Friday is a dead end, so the assistant should say when someone will reply. Intercom describes handing Fin conversations to human agents in a help desk3. Zendesk describes handoff inside its own workflows5.

Unknown

Tool failure

Customer request

Risk or emotion

Unavailable staff

Open Handoff packet example

Handoff packet example

Step 7: log and evaluate

Keep a record of each question, answer, action and final result. Read a sample by hand every week and sort each failure by cause: missing knowledge, bad retrieval, a broken tool, a policy gap or the model itself.

Details

Logs help only when they record outcomes, not just activity. Label every inquiry as answered, action taken, handed off or abandoned. Each week, read a sample of real conversations by hand, because averages hide the specific way a system fails. Then sort each failure by cause. Missing knowledge needs a content edit. Bad retrieval needs a cleaner source. A broken tool needs whoever owns the connection. A policy gap needs a written rule. Whatever is left is a model problem. Keep personal details to what you need and delete them on a schedule. People are wary of AI in general: 51% of US adults told Pew they are more concerned than excited about it in daily life11.

Outcome labels

Audit sample

Error taxonomy

Privacy

Improvement loop

Open Error taxonomy table

Error taxonomy table

Error taxonomy table
QuestionHow to check
Outcome labelsCheck that every inquiry carries one label: answered, action taken, handed off or abandoned.
Audit sampleRead a sample of real conversations by hand each week, because averages hide the specific way a system fails.
Error taxonomySort each failure by cause: missing knowledge, bad retrieval, broken tool, policy gap or model problem.
PrivacyCheck that logs keep only the personal details you need and that they are deleted on a schedule.
Improvement loopSend each failure to its owner, such as a content edit for missing knowledge, and recheck it the next week.

Good enough when, the same for every row above: Record the authoritative result or preserve context for the named human handoff. Do not count generated text as resolution.

Two worked architectures

A small-business website assistant and a help-desk AI agent run the same basic steps at different scales. Compare what each reads, what it may change and where unfinished work goes. These are design sketches, not vendor test results.

Details

Send "can you come out Thursday?" into two builds. The website assistant reads your site and a short policy document. It answers, qualifies the inquiry, and books on a connected calendar only when the booking succeeds; otherwise it collects the details1. First AI Employee sells this kind of managed assistant. One owner can look after it. The help-desk AI agent lives in a ticketing platform, can do account work after the customer signs in, and escalates to a staffed queue with the history. Intercom prices Fin per outcome, and seat costs may also apply2. It needs agents behind it. AI has its best evidence there: at one firm, an AI assistant helped human agents resolve 15% more issues per hour on average10.

Small-business website assistant

Help-desk-native AI agent

Frequently asked questions

Short answers on where the assistant gets its facts, what it may change and when it hands off. The exact limits depend on the product and how your account is set up, so ask the vendor.

Details

Frequently asked questions

Does AI customer service search the internet for every answer?

Not usually, and often it should not. Business systems normally answer from your approved website, policies, help articles or account records. If it can reach the open internet at all, that should be a setting you chose and can control.

What is retrieval-augmented generation?

It is a setup where the system first finds the relevant passages in your material and hands them to the AI model before it writes the answer. That keeps answers closer to your facts. Old, incomplete or conflicting sources still need to be caught, with a fallback when they are.

How can an AI agent take actions?

A controller sends the request to a tool you approved, with checked details and only the permissions that task needs. The tool reports back whether the action worked. The system should never announce success just because it tried.

How does human handoff work?

A good handoff gives the person a short summary, the parts of the chat or the details that matter, the reason for the handoff and where things stand. It also tells the customer what happens next, and it never pretends someone is free when nobody is.

How is the system improved?

Read a sample of conversations and failures. For each failure, decide whether the cause was the content, the search, the instructions, the model, a tool, a policy or the handoff, and fix that part. Then rerun the same set of test questions before the change goes live.