AI Phone Agents: What They Can Actually Handle, What a Minute Costs, and the 200ms Problem

The phone in a dental practice rings most at 12:40, when the receptionist is checking someone out and the hygienist is on the other line. The call goes to voicemail. The 42% of people who leave a message get a callback at three; the rest ring the next practice on the map. Everyone in the building knows this happens and nobody can fix it, because the fix is a second receptionist for forty minutes a day.

That forty minutes is what AI phone agents are actually for. Not "replacing the front desk" — the demos that claim that are selling the wrong thing — but answering the calls that currently go nowhere, doing the four boring things well, and handing the rest to a person with the context attached.

We've now built enough of these to have opinions, and the opinions are mostly about three numbers: 200 milliseconds, ten cents, and 91 percent. Here's what each one means for whether you should do this, and how.

First, the missed-call statistic everyone quotes

You've seen "62% of calls to small businesses go unanswered". It's on every vendor's homepage. It comes from a 2016 mystery-shopper exercise across 85 businesses, and I'd not build a business case on it. What I trust more is your own phone system's report, which every VoIP provider produces and almost nobody reads: calls received, calls answered, calls abandoned, average time to answer, by hour of day. Pull last month's. If abandoned plus voicemail is above 20% in any two-hour window, you have the problem this article is about. If it isn't, save your money and read something else.

The 200 millisecond problem

Human conversation runs on a clock most people never notice until a machine breaks it. In 2009, Stivers and colleagues published in PNAS a study of turn-taking across ten languages on five continents. The gap between one person finishing and the next starting had a mode of 0 to 200 milliseconds in every language measured; the cross-linguistic median was +100 ms. As the authors point out, that's faster than people can name an object out loud. We start planning our reply before the other person finishes, and any system that waits for silence, thinks, and then speaks is already late.

This is the design constraint that decides whether an AI agent feels like a competent receptionist or a phone tree with better vocabulary. A cascaded agent — the standard architecture — has to do five things in sequence before you hear anything:

The sum is somewhere between 0.9 and 2 seconds on a good day, and it's the first item — endpointing, the wait to be sure the caller has actually stopped — that people underestimate. Set it short and the agent talks over "and also—". Set it long and every reply starts with a beat of dead air. Speech-to-speech models like OpenAI's gpt-realtime family collapse the STT and TTS hops and get closer to 500–800 ms, at the cost of giving up the text in between, which is where you'd normally validate what the model is about to say.

You will not reach 200 ms. Nobody does. What we've found works is not chasing the number but designing around it: a short acknowledgement ("let me check that") that streams while the calendar lookup runs; endpointing tuned by call type, because someone reading out a policy number pauses differently from someone asking a question; and never, ever a synthesised "um" to fill the gap, which callers clock instantly and hate.

What a minute actually costs

Vendors quote per-minute prices, but the underlying models bill per token, and the conversion is public. On OpenAI's current pricing, audio is metered at one token per 100 ms of caller speech and one per 50 ms of agent speech — 600 and 1,200 tokens per minute respectively. That gives you the arithmetic:

So the raw cost of a well-built inbound agent is somewhere between three and seven cents a minute, before anyone's margin. Practitioners who've measured real sessions report higher figures — roughly $0.06–0.11 a minute on the flagship model once prompt caching is working, and two to four times that when it isn't, because every turn re-sends the conversation so far. Bundled platforms that include telephony land at $0.10–0.18 a minute in the vendor roundups we've checked, and the "bring your own keys" platforms end up at $0.13–0.31 once you add the model, voice, transcription and carrier they don't include. The headline "$0.05 a minute" on a pricing page is almost never the number on the invoice.

Two warnings about that chart. First, the token price is the part that's been falling; it's the carrier, the platform fee and the engineering that don't. Second, Gartner's read of the direction is the opposite of the vendors'. In January 2026 they predicted the cost per resolution for generative AI in customer service will exceed $3 by 2030 — higher than many offshore human agents — as vendors move from subsidised growth to profitability and use cases get more complex and token-hungry. Their conclusion: "full automation will be prohibitively expensive for most organizations". I don't fully agree with the forecast, but I agree with the implication. Build the agent to do a narrow set of things at a low cost per call, not to be a general-purpose employee at any cost.

The 91 percent, and why it's a warning

Gartner surveyed 321 customer service leaders in October 2025 and found 91% were under pressure from executives to implement AI this year. By August, AI spending in those functions was up 38% against total budgets up 2%. That is a lot of money moving on the basis of pressure rather than a measured problem, and it is exactly how you end up with an agent that answers every call, resolves none, and is quietly switched off in month four.

The projects that have worked for us started from the phone system report, not from the board. They picked the narrow band of calls that are (a) high-volume, (b) fully answerable from systems the agent can read, and (c) not the moment the relationship is decided.

If you want the wider adoption picture, GoodFirms maintains a roundup of AI adoption statistics compiled from published surveys. It is useful background, and it is also the trap this section is about: industry-wide adoption rates tell you what everyone else is doing, not whether your own phone is ringing at a volume that justifies any of it.

What to let it handle, and what never to

We use the same five-stage framework for phone that we use for every other intake channel — Capture, Understand, Act, Escalate, Report — and the phone version looks like this.

Let it handle

• Booking, rescheduling and cancelling against the live calendar, with the agent reading real availability rather than guessing. This alone is most of the 12:40 problem for clinics.
• "Where's my order / where's my driver / what's my claim status" — a lookup with a number the caller already has.
• Hours, location, parking, insurance accepted, what to bring — the questions that make up a third of a front desk's day and none of its value.
• After-hours and overflow intake: take the name, number, reason for calling and urgency, create the record, and tell the caller exactly when they'll hear back. This is the highest-return use, because it converts calls that currently vanish.
• Outbound reminders with one-word replies — confirm, reschedule, cancel — where the caller has consented to be contacted.

Never let it handle

• Anything that sounds like a symptom, a chest pain, a medication question. Route to a human on the first sentence, not after a triage attempt.
• A complaint. The AI-handled complaint is the one that ends up on a review site.
• Pricing negotiation, discounts, refunds beyond a fixed policy.
• A caller who has asked for a person. Once. Not after "I understand you'd like to speak to someone, but first…".
• Anything with legal consequence in the insurance world — coverage confirmations, claim decisions, a statement about what a policy does or doesn't cover.

Escalate means a warm transfer with the transcript, the caller's record and a one-line summary already on the human's screen. If the person picking up has to say "sorry, can you start again?", you've built a slower voicemail.

Compliance, in the order it will bite you

Tell people it's an AI. The EU AI Act's transparency duties under Article 50 apply from August 2026: anyone interacting with an AI system must be told, unless it's obvious. Several US states have bot-disclosure laws of their own. Practically, the agent's first sentence says what it is. Callers don't mind; they mind being fooled.

Outbound is regulated; inbound mostly isn't. In February 2024 the FCC ruled that AI-generated voices count as "artificial" under the TCPA, which means an outbound call using one needs the caller's prior express consent, and the penalties per call are real. A caller phoning you is not a robocall. A reminder you place to someone who ticked a box is fine. A cold call from a synthetic voice is a lawsuit. We cover the current state of TCPA on our insurance intake page, because that's where it matters most.

Recording consent. Around a dozen US states, and most of Europe, require all parties to consent to a recording. The agent announces it; the transcript still exists, so treat it as a recording either way.

Healthcare. If a caller can say anything about their health — and they will — every vendor in the chain that touches the audio or the transcript needs a signed Business Associate Agreement: carrier, transcription, model provider, storage. Not all of them offer one. Check before you build, not after. Our healthcare page goes through the BAA question in detail.

How we build one

For readers who want the architecture rather than the pitch, this is the shape that has held up.

1. Telephony via a programmable carrier (Twilio or a SIP trunk into a media server), streaming audio both ways over a WebSocket. Keep your existing number; forward on no-answer after two rings, or overflow, or out of hours.
2. Understanding — either a streaming STT feeding a text model, or a speech-to-speech model. We default to cascaded for anything that touches money or health, because the text in the middle is where validation lives.
3. Tools, not prompts, for facts. The agent never states an appointment time, an order status or a price it didn't just get from a function call against the system of record. The model chooses which tool; the tool returns the truth.
4. Confirmation before commitment. Read back the slot, the name, the number. Then write. This adds two seconds and removes the worst class of failure.
5. Escalation with context: warm transfer where staff are available, structured callback record where they aren't, with urgency scored by rules rather than vibes.
6. Reporting that you'll actually read: calls handled, calls escalated, why, bookings made, callers who asked for a human, and — the one everyone forgets — calls where the agent said something that later turned out to be wrong. Sample transcripts weekly. Sample the escalations, not the successes.

Buy the platform or build it?

Buy if your calls are generic, volume is under a few hundred a month, and you have no systems to integrate — a bundled platform at ten to eighteen cents a minute is the cheapest way to find out whether callers accept it. Build (or have built) when the calls are worth money, the agent has to read and write your calendar, CRM or claims system, compliance requires you to know exactly where the audio goes, or the per-minute margin at your volume costs more than the engineering did. The crossover in our experience sits around two to three thousand call minutes a month, and it moves down every time a token price does.

If you want to see what the numbers look like for your own volume, the patient no-show calculator and the dispatcher time calculator both run on the model above; put your call counts in and it'll show you the breakeven rather than a promise.

Answer the calls that go nowhere. Do four things well. Hand over early, with the context attached. Everything beyond that is the part Gartner's warning about.