Voice

What is a voice AI agent, and how does it connect to the phone network?

A voice artificial intelligence (AI) agent listens and speaks through software, then connects to the phone network through a Session Initiation Protocol (SIP) endpoint.

The phone network has no special category for an AI agent.

Those are two independent problems, and only the conversation introduces a new mechanism. The conversation is a model problem. The phone call is a 2002 protocol, and it does not care what is on your end of it.

What is the agent, and what is the connection?

The agent is your program. The connection is a SIP endpoint, and it is indistinguishable from a softphone.

The loop inside the agent is roughly: audio in, transcription, a model deciding on a reply, synthesis, audio out, repeat until somebody hangs up. Every part of that is yours to build or to buy, and none of it is telephony.

The connection is telephony and nothing else. Your agent, or the platform hosting it, sends the SIP INVITE method that requests a call to a SIP trunk. The trunk authenticates that invitation rather than a registration, exactly as it would for a desk phone connected the same way. The network on the other side has no representation for "this is a machine". What is SIP covers the protocol; the important consequence here is the one the specification states about itself, which is that SIP arranges a session and then gets out of the way of the media.

A SIP call agrees where the audio will flow and then leaves it to a separate media path. An agent that has implemented signalling but has nowhere to terminate the audio can arrange the call but cannot hear it. Whatever you use has to hold both halves, and the media half is where the interruption handling, the silence detection and the turn taking actually live.

How does an inbound call reach an agent?

Through a number whose answer is a trunk your agent is behind.

Three things stand between a caller dialling your number and your agent hearing them, and each is set in a different place:

  1. The number's answer. A number your workspace holds carries one answer at a time, set on the number itself: deliver to a SIP trunk, forward to a verified number, or reject. Every number starts on reject, so a number nobody has configured turns callers away rather than reaching anything.
  2. Inbound enabled on the trunk. Each direction is configured separately, and enabling inbound is not sufficient on its own.
  3. A delivery gateway on that trunk. A trunk with inbound enabled and no gateway refuses every incoming call to its numbers. Inbound setup is incomplete without the gateway.

Receiving calls covers pointing a number at an answer, and SIP trunks covers the gateway. Turning inbound calling off puts every number pointing at that trunk back on reject, and turning it back on does not restore them.

How does the agent place a call?

As a SIP client, from a number the workspace has verified, to a country the workspace has enabled.

Outbound is the same path a private branch exchange (PBX) or a softphone takes, and the same three checks apply before a carrier is involved. The From header has to carry a verified caller ID in the E.164 international number format, or the call is refused as caller_id_not_verified. The destination country has to be switched on. The account's ceilings on concurrent calls, calls per second and daily spend each refuse a call on their own.

An agent fleet is the shape of traffic those ceilings were built to bound. A dialer that starts a hundred conversations a second meets the per-second ceiling well before it meets the concurrent one, and the two report differently and are raised differently. What is toll fraud covers each ceiling and the rejection reason it produces. The same limits apply to capacity planning even when fraud is not your concern.

Authentication has one wrinkle specific to agents that run in a browser or use a browser's media stack. A short-lived session credential is minted over HTTP and spent over SIP. Its password is returned once. A browser client answers a digest challenge with MD5 and nothing else. A trunk that offers only SHA-256 therefore refuses it. WebRTC vs SIP covers why a browser path looks different at the signalling layer while being the same call underneath.

What does the phone network make me deal with?

Everything it makes anyone deal with, and one thing it does not do at all.

An agent is not a category the network recognises, so nothing about it is exempt. The number you present is subject to caller ID authentication and to the analytics that decide what a handset displays, which means an agent campaign can be labelled exactly as any other outbound programme can. Secure Telephone Identity Revisited and Signature-based Handling of Asserted information using toKENs (STIR/SHAKEN) covers what your provider can vouch for, and why is my call labelled spam likely covers what happens when the far end decides against you.

Design around two network constraints:

  • Emergency calls are not carried. A call to an emergency number is refused, on every trunk, in every country, and that does not depend on your plan, balance or destinations. An agent that could plausibly be asked to place one needs another route, and placing calls covers what to configure instead.
  • Setup time is not yours to optimise. The gap between dialling and the far end ringing is a network property, and it sits in front of anything your model does. Budget your first-token latency against a call that has already spent time in the network rather than against a stopwatch that starts when your code runs.

Which half am I actually buying?

The two halves have different properties, and only the telephony half is standardised.

The telephony half is interchangeable. Numbers, trunks, verified caller IDs, enabled destinations, the call record and the events that tell your systems a call was answered or ended have the same shape wherever you get them, because they are described by specifications anyone can read. Moving between providers is a configuration exercise rather than a rewrite. Voice documents that half here.

The model half is not. Transcription accuracy on a narrowband call, how quickly the agent yields when someone interrupts, and what it does when a caller says something nobody anticipated determine how the agent performs, and none of them appears in a protocol or a schema. Price the model and telephony halves separately because the connection does not constrain these properties.

When a product offers to put your agent on the phone, identify which half it sells and which half you still supply. What is a voice API covers how the configuring, signalling and reading halves fit together, which is the shape of the integration whichever agent you put behind it.

In short

  1. Two separate things get sold as one.

    The agent is speech recognition, a model and speech synthesis in a loop. The connection is a SIP endpoint. Only the second is telephony, and it is the ordinary kind.

  2. The network cannot tell an agent from a person.

    Caller ID verification, enabled destinations, concurrency ceilings, what the provider can vouch for and spam labelling all apply exactly as they do to any other call.

  3. Inbound needs a number, an answer and a delivery gateway.

    A number starts on reject. Pointing it at a trunk is set on the number, and a trunk with inbound enabled and no gateway refuses every call to it.

  4. The agent has to terminate the audio, not just the signalling.

    SIP sets the session up and steps aside; the media runs on its own path. An agent that only speaks SIP has arranged a call it cannot hear.

Build on the same network.

A test API key is yours immediately. Production unlocks when you add a payment method and verify a sender.

Start with one channel.
Add the others when you're ready.

A test API key is yours immediately. Production unlocks when you add a payment method and verify a sender.

Using Claude Code, Cursor, or Codex? Copy a setup prompt and your agent installs the Bird CLI and skills for you. Pick yours:

Cursor