Voice AI for Automotive Customer Operations

Customer operations · One of India's largest used-car sellers · Self-hosted speech recognition and Amazon Bedrock

This engagement combined a self-hosted, fine-tuned Whisper model with Amazon Bedrock for a voice AI application in a large Indian automotive customer operation.

The operating context was substantial: approximately 22,000 connected calls per day, with planning for around 90,000 inbound and outbound connected calls per day as utilization increased. The traffic brief also identified 450 average concurrent calls, 1,500 peak concurrent calls and a 2.5-minute average call duration.

These are the customer's supplied traffic figures and planning assumptions. They establish the scale the architecture had to address; they do not establish that every call was AI-handled or that the AI platform had already passed a 1,500-call load test.

The problem: a voice interaction has a live resource footprint

Daily volume is only one dimension of a calling operation. Every active conversation occupies a telephony session, carries audio, maintains conversation state and generates work for speech recognition, reasoning and speech output.

Campaigns concentrate that demand. The brief identified peak windows around 10 AM, 1 PM and 5 PM, alongside month-end promotions and festival campaigns around Holi, Navratra, Diwali, Christmas and New Year. A service sized only for an all-day average could be underprovisioned precisely when the commercial team needed it most.

The engineering challenge was to connect domain-adapted recognition to a responsive conversation, then reason about its capacity at each stage. More connected calls do not translate into the same multiplier for GPU work, model requests and CPU sessions.

The reported stack and its boundaries

The speech-recognition component was a fine-tuned Whisper model served on infrastructure controlled by the application team. Amazon Bedrock supplied the managed foundation-model layer. This separates ownership of the speech checkpoint and serving behavior from access to the conversational model.

The logical conversation path is:

The diagram explains the component responsibilities. It does not identify an unprovided telephony vendor, speech-synthesis provider, Bedrock model ID or orchestration framework.

Bedrock's ConverseStream is one interface available for streaming responses from supported models. It is an implementation option for the reasoning stage, rather than a claim about which API this engagement used. Amazon Bedrock API reference

Why fine-tune and self-host the speech layer?

Recognition errors can change the meaning of a customer interaction before the language model sees it. Vehicle names, variants, locations, prices and spoken numbers are useful evaluation categories for automotive conversations. Indian call audio can also contain code-switching, accents, background noise and telephony compression.

Fine-tuning provides a way to adapt recognition to the target speech distribution. Self-hosting provides control over checkpoint rollout, inference configuration and scaling. It also makes the team responsible for GPU utilization, availability, version rollback and model-serving behavior.

Whisper's underlying research addresses multilingual speech recognition and transfer across tasks and datasets. That establishes the model family; the value of a particular fine-tune still has to be measured on held-out customer-relevant audio. Whisper research paper

The project information supplied here does not specify the Whisper size, training hours, language mix or word-error-rate improvement. Those details are therefore not borrowed from another experiment. In particular, the site's separate Hinglish ASR experiment is an independent research result, not this client's benchmark.

The traffic brief

Connected calls are distinct from dial attempts. Connection rates, ringing time and abandoned attempts can create additional carrier and application load that this table does not quantify.

Translate call counts into capacity

At the stated duration, the current 22,000 calls represent roughly 55,000 connected call-minutes per day. The 90,000-call scenario represents 225,000 call-minutes per day.

Spread uniformly across 24 hours, those totals imply approximately 38 and 156 concurrent calls respectively. The brief's average of 450 therefore cannot also be interpreted as a uniform 24-hour average for the same 90,000-call population at a 2.5-minute mean duration.

There is a plausible operating-window interpretation: 90,000 calls at 2.5 minutes occupy 225,000 call-minutes; dividing by 450 concurrent calls gives 500 minutes, or about 8 hours 20 minutes, of equivalent active time. This is a reconciliation of the inputs, not a claim that the client used exactly that calling window.

For a steady period with a 150-second mean duration, the relationship between connected-call starts and concurrency is:

concurrent calls ≈ connected-call starts per second × mean call duration

450 / 150 seconds  = 3 starts/second  = 10,800 starts/hour
1,500 / 150 seconds = 10 starts/second = 36,000 starts/hour

The peak calculation is a local planning envelope if the peak lasts long enough for steady-state reasoning. It is not a daily forecast or a measured arrival rate. Real bursts, call-duration distributions and scheduled outbound dialing require a time-based demand model.

Separate session capacity from inference capacity

A concurrent telephone session is not continuously consuming the same amount of speech-recognition compute. The customer may be listening, speaking, pausing or waiting for an external action. A speech server's workload depends on how audio is segmented, whether silence is processed, and how overlapping recognition windows are used.

One useful first-order sizing relationship is:

required speech throughput
  ≈ concurrent sessions × fraction of call time submitted to ASR

GPU workers
  ≈ required speech throughput / measured throughput per worker

The denominator must come from a benchmark of the selected checkpoint, precision and streaming adapter at an acceptable tail latency. An offline batch transcription result does not establish how many live calls a GPU can support. Audio overlap and re-decoding can also increase work beyond the underlying speech duration.

The same separation applies to the rest of the platform. CPU workers maintain connections and state. Bedrock demand depends on turn frequency, context length and output length. Speech-output demand depends on assistant speaking time. A GPU count inferred only from “1,500 concurrent calls” would hide these different constraints.

Conversation quality and the operating controls

The following are the engineering acceptance criteria for this arrangement, rather than additional measured results from the traffic slide:

• Recognition quality: evaluate real telephony conditions and critical entities such as names, locations and numbers, alongside aggregate transcription error.
• Turn latency: measure from the customer's speech ending to the first useful audible response, including recognition, reasoning and synthesis delays.
• Interruption: stop obsolete playback and invalidate late responses when the customer starts speaking again.
• Capacity and overload: admit work within measured speech, reasoning and session limits, with a deliberate overflow or human-handoff policy.
• Campaign readiness: warm required capacity before known peaks and test realistic mixtures of inbound and outbound traffic.
• Safe rollout: retain versioned model and runtime configurations, compare against a baseline and preserve a rollback path.

These controls connect a speech model to a usable customer operation. A lower transcription error is valuable when it improves the completed interaction without making the caller wait too long or leaving the system fragile during a campaign.

Scope of work and the delivery lesson

The confirmed technical scope is the combination of self-hosted fine-tuned Whisper and Amazon Bedrock for automotive voice AI. The supplied operating brief provides the current connected-call volume, growth scenario, concurrency assumptions, call duration and campaign pattern.

The capacity brief is a starting point for engineering acceptance. Production throughput, customer outcomes and operating cost belong in the next layer of evidence, measured against the same traffic definitions and model configuration.

The architectural lesson is concrete: adapt recognition to the customer domain, preserve a clear boundary between speech and reasoning, and size the service against a reconciled traffic model. Daily call volume establishes the business scale; measured per-stage capacity establishes what the system can reliably carry.

Related engineering: cloud deployment and GPU serving, turn-taking and latency and voice-agent evaluation.