Conversational AI: From Bolna to Pipecat

The calling platform was built to take over the initial conversation needed to understand a customer’s request. Insurance calls captured coverage requirements and arranged callbacks; healthcare calls gathered appointment requests and routing details. The useful result was information the next person could act on, collected through a conversation rather than a staff member working through every call.

The initial client platform used a customised Bolna foundation. Its delivery informed Zoice, the configurable Pipecat product subsequently developed at Kuzushi Labs. Across both phases, the call had to end with useful customer information or a clear next action, while handling interruptions and preserving the conversation’s context.

The seed round belongs to the client’s company journey. Zoice was our subsequent product, carrying forward the engineering lessons into a more configurable platform.

The business problem: a conversation has to lead somewhere

The client served three sectors, each with a different definition of a successful call. An insurance enquiry needed qualification and an actionable follow-up. A healthcare enquiry needed appointment context and the right handoff. An education call needed a usable record of interest and a next step.

These requirements shaped the product before we selected individual models. A natural voice was useful only if the agent collected the required information, handled uncertainty and handed over a coherent result. A recording alone left the customer with the work of listening to and interpreting every conversation.

Indian telephone conversations added practical constraints: variable audio quality, Hindi-English switching, names and numbers, caller interruptions, silence and external service delays. We had to connect those conditions to the customer’s workflow while keeping the call responsive.

Our process: discover, deliver, then productize

1. Turn customer conversations into a delivery scope

I worked on sales discovery and requirements definition, translating the customer’s workflows into agent instructions, call context and integration contracts. We needed to establish what the agent should ask, what information had to survive the conversation and when the next action belonged to a person.

That made the first delivery a complete calling journey: initiate or receive the call, run the conversation, handle lifecycle events and retain useful interaction records. It also gave us a way to assess the platform beyond whether a demonstration sounded convincing.

2. Build the first end-to-end version on Bolna

We forked and modified Bolna instead of writing every orchestration component from scratch. Plivo provided the telephony boundary, while adapters connected speech recognition, a language model and speech synthesis. Python asynchronous tasks and queues coordinated incremental processing during each call.

Each active call needed its own identity, conversation context and execution state. Telephony callbacks, media transport and model output had to refer to the same session. Production delivery used containers and Kubernetes; Redis supported context around the calling workflow.

The useful separation was between the carrier, the conversation runtime and the model services. It let us customize the call experience and select providers without making the telephony integration depend on one fixed model combination.

3. Work through the real-time failure modes

Once a call connected, the difficult work moved to timing and cancellation. A caller could start speaking while the model was still generating, while TTS was synthesizing or while the carrier was playing buffered audio. A disconnected call could leave tasks and provider connections alive if the lifecycle was incomplete.

We worked through those seams in the Bolna implementation: sequence state, asynchronous cancellation, output queues and Plivo playback control. Recognition timing, first model output and first synthesis output provided stage-level evidence for investigating delay.

4. Carry the lessons into Zoice

The next phase broadened the problem. We wanted agent definitions, providers, voices and pipeline choices to become reusable product configuration. Zoice combined a Pipecat runtime with a web portal, shared data models and a post-call processing path.

The shift to Pipecat gave us a composable pipeline for transport, recognition, turn handling, context, generation and output. The product also supported a native-audio branch. This was a further product build at Kuzushi Labs, rather than simply renaming the initial client implementation.

The first platform: owning the call lifecycle

The Bolna path separated telephone media from the three main model stages:

Provider adapters included Deepgram and Whisper for recognition, OpenAI and LiteLLM-backed routes for reasoning, and configurable synthesis services. Provider choice happened through configuration. That flexibility let us change the components used for a call; transferring a live conversation between providers would require additional state-transfer and recovery logic.

The architecture had to preserve a simple relationship: the response being generated, synthesized and played must belong to the current conversational turn.

Interruption handling crossed every layer

When a caller interrupted, cancelling the LLM request alone was insufficient. Old text could already be waiting for synthesis, generated audio could sit in a local queue, and the carrier could still be playing it.

Our implementation coordinated a Plivo clearAudio event with cancellation of generation, synthesis and output tasks. Buffered output was cleared, and request and sequence state distinguished abandoned work from the new turn. Output processing then resumed for the caller’s current request.

This was one of the clearest examples of application engineering determining conversation quality. The model might understand an interruption perfectly while the caller continued hearing an obsolete answer because another layer had retained its audio.

Latency and concurrency: what we delivered

By early 2025, the client implementation achieved approximately 1.1-second response latency, delivered 40 concurrent calls, and reached a maximum of 100 concurrent calls in testing. Sub-second response, with an 800 ms preferred budget, remained an optimization target.

These figures describe the initial delivery period. The 100-call test is distinct from the 40-call delivered workload, and the latency figure is not presented as a percentile or a later Pipecat benchmark.

The response path crossed endpointing, recognition, language-model generation, synthesis and transport. Reducing a single model’s processing time could leave the overall experience unchanged if another stage was waiting or buffering. A server-ready audio chunk and the first word heard on the telephone were also different measurement points.

The same reasoning applied to concurrency. Each live call consumed open connections, per-session state and capacity from external services. Kubernetes supplied the deployment environment; reliable calling still depended on the application owning session cleanup and the capacity of its dependencies.

Zoice: two speech architectures behind one product

In July–September 2025, we developed Zoice around FastAPI, Pipecat, Plivo WebSockets, PostgreSQL and Redis, with a separate web application. The inspected implementation used Pipecat 0.0.91.

Two conversation paths were implemented:

The platform selected the pipeline from saved configuration for each call. Language quality, interruption behavior, cost and the degree of conversation control all influenced that choice.

In the cascade, transport input flowed through STT muting, recognition, idle handling, user-context aggregation, the LLM, TTS, transport output and assistant-context aggregation. The opening greeting went directly to TTS and was added to assistant history, avoiding an extra LLM request for that first greeting.

On disconnect, the implementation submitted conversation messages to Redis, ended the pipeline task and closed the per-call HTTP session. Process-level signals remained with Uvicorn. These details mattered because a finished call should also release the resources that kept it running.

Provider interoperability required real adapter work

Zoice exposed Google, Deepgram, Gladia and Groq recognition adapters; OpenAI and Groq language-model adapters; ElevenLabs, Google and a patched Sarvam synthesis service; and Gemini native audio. Silero VAD provided configurable voice-activity detection.

A saved provider name was only the beginning. The service factory translated top-level settings, nested parameters, voice IDs and language representations into the arguments expected by each provider. The Sarvam adapter explicitly mapped regional language enums, including Hindi-India and Telugu-India, into provider language codes.

The practical lesson was that “supports multiple providers” needed a working configuration path all the way from the operator’s form to a live service instance. Each adapter also had to fit the conversation’s transport and lifecycle.

The portal and the work after the call

The Next.js portal made agent and pipeline configuration part of the product. It included provider forms, pipeline creation, a pipeline visualizer and realtime-agent controls. React Query managed server data and Zustand managed client state. PostgreSQL stored pipeline definitions, while Redis supplied call-associated configuration and context.

Post-call processing connected recordings to a text representation, workflow-specific extraction questions, persisted results and downstream delivery. That gave the customer an actionable record alongside the conversation.

For the ClearTax pilot, the workflow involved classifying the customer’s profile, gathering relevant income context and arranging an appropriate expert handoff. The value depended on the information available to the next person in the process.

Review also required distinguishing evidence sources. The recording-derived Whisper translation path produced English text for extraction. That could support downstream processing, but it could not serve as a source-language Hindi or Hinglish recognition reference. Live model history, a recording-derived transcript and the backend’s action result answered different evaluation questions.

My contribution

My role was sales, senior architecture and hands-on development, working within a three-person team in each phase.

• Customer discovery and scoping: connected sector-specific calling needs to agent behavior, required context, outputs and follow-up workflows.
• Initial architecture and implementation: helped build the modified Bolna platform, Plivo integration and asynchronous streaming path through production delivery.
• Conversation engineering: worked across interruption handling, sequence state, model integration and latency-sensitive call behavior.
• Product architecture: carried the earlier experience into Zoice’s Pipecat backend, configurable provider integration and operator-portal integration.
• Delivery across boundaries: connected the live voice runtime to deployment, call records and the customer’s operational workflow.

The delivery was a team effort. My contribution was the continuity across the commercial brief, architecture and code, so that product decisions remained connected to what we could reliably ship.

Outcomes and the next layer of work

The initial engagement produced a working concurrent calling platform, and the client subsequently raised seed funding. That is a meaningful company milestone following the early product delivery; no funding amount or sole-cause attribution is claimed here.

Zoice extended the work into our own configurable voice product, with cascaded and native-audio paths, multiple providers, operator controls and the ClearTax pilot. The broader achievement was a repeatable path from a customer workflow to a configured agent, a live telephone conversation and a usable business result.

Later speech research is documented separately. The public Hinglish ASR reproduction and the 2026 local TTS experiments provide additional technical depth without being folded into the historical client’s performance figures.

Explore the engineering in depth

• The Bolna implementation and call lifecycle
• The Pipecat platform and staged pipelines
• Latency, turn-taking and interruptions
• Evaluation and post-call evidence
• Business impact and cost
• Zoice’s product overview and the complete twelve-article voice series