Pipecat Labs releases tiny CPU classifiers for tone and intent in voice agents

Pipecat Labs released Pipecat Tone, two small text classifiers that read caller tone and tag agent replies for TTS in voice pipelines.

Pipecat Labs has released Pipecat Tone, a pair of small text classifiers that sit on either side of the LLM in a voice pipeline, per its Oct 28, 2026 post. If you build text-cascade voice agents, this targets a known gap: an exasperated "oh, great" reaches the LLM looking like a delighted one, and replies reach the TTS as flat words.

The tone reader classifies each caller turn for emotion, sarcasm and intent, using the agent's previous line as context. It is a 48.2 MB ONNX file that runs in about 5 ms a turn on CPU. The tone tagger labels each sentence of the LLM's reply with delivery, sounds such as laughs and breaths, slowed spans and stressed words, at 63.4 MB and about 14 ms a sentence. Pipecat says both work with any LLM, drop into a Pipecat 1.15.0 pipeline, use the Real-time Context Standard, and are BSD-2-Clause along with their data and training code.

Pipecat frames them as a stop-gap for LLMs that can't write these tags themselves. On its own tone-bench, phonellm-mini plus the tagger scored 90.9 against 90.2 for gpt-5.6-luna, which writes its own tags. Pipecat lists the limits itself: the hand-written rules alone score 90.0, delivery macro-F1 is 0.64, scores come from LLM labels rather than human listeners, coverage is English only, and a flat transcript reads as neutral.

The benchmark is the vendor's own, and we found no independent testing. The sources don't compare alternatives such as speech-to-speech models or audio emotion classifiers, apart from Pipecat's point that prompting an LLM to tag is weaker. Audio models look like the next step. @kwindla says a Pipecat Voice Reader, built on a fine-tuned Whisper encoder, now listens for tone of voice, energy and sarcasm.

Sources