Dashboard

llama.cpp Realtime Audio Arrives in v0.8.0

llama.cpp v0.8.0 ships a realtime audio endpoint, a text-to-speech endpoint and the LFM3 audio models. Two of the three are safe to build on today.

Cecilia Iona
Cecilia Iona
Senior Editor, AI & Product
30 October 20261 min read

llama.cpp released v0.8.0 on 28 October 2026, and the headline is audio. The llama.cpp realtime audio work lands as two new server endpoints and one experimental C API: a /v1/realtime endpoint implementing the OpenAI realtime protocol for conversation and transcription, a POST /tts endpoint for text-to-speech models, and mtmd_audio_stream, which turns PCM audio fed in as it arrives into embedding rows the decoder can consume.

Those three are at different levels of maturity, which the summaries going round mostly flatten into "adds audio". The difference matters if you are deciding what to build on this week.

What shipped in the llama.cpp realtime audio release

The /v1/realtime endpoint

This is the one most people will use. It implements the OpenAI realtime protocol, covering both conversation and transcription, and it supports tool calls. Because it speaks an existing protocol, client libraries written against that protocol can point at a local server with a URL change rather than a rewrite. The release notes specify that it sends response.done once the committed audio has drained, which is the kind of detail that decides whether your client hangs or closes cleanly.

The /tts endpoint

A plain POST /tts for serving text-to-speech models, alongside support for Soprano TTS. Simpler surface, fewer moving parts, and the easier of the two to wire into something that already exists.

mtmd_audio_stream

The C API underneath, added to support voxtral-realtime. The release calls it experimental, and that label is doing real work: it is the layer most likely to change shape in the next few versions. Fine to experiment against, risky to build a product on without pinning a version.

The LFM3 models that come with it

The release adds Liquid AI's LFM3 family. LFM3-1.7B for text and LFM3-VL-2B for vision were verified against the reference Hugging Face implementation. The pair that matters for audio work is LFM3-Audio-2B and LFM3-Omni-2B, which handle speech recognition, text-to-speech and voice chat from a single model with streaming audio in both directions.

A 2-billion-parameter model doing all three jobs is the interesting part. The usual local voice stack is three separate models chained together, each adding latency at the handoff. Collapsing that into one model is how you get a local voice loop that does not feel like a walkie-talkie, which is the same problem our explainer on speech-to-speech models describes from the architecture side. The release also adds MiniCPM-OCR 1.0 and, per the changelog, video input for the LFM3 family.

Why a local realtime endpoint is worth attention

Audio is the workload where sending data to a third party is hardest to justify and hardest to avoid. Call recordings, voice notes and meeting audio are exactly the material that makes a privacy review slow, and until recently the realtime options were hosted ones. A local server speaking the same protocol changes that calculation for anyone whose blocker was data handling rather than cost.

It also changes the latency story, because the round trip to a hosted endpoint is a floor you cannot optimise below. Interruption handling, the thing our piece on barge-in in voice AI is about, gets easier when the model is on the same machine as the microphone.

How to try it without wasting an afternoon

  1. Build or pull v0.8.0 specifically. The endpoints do not exist in earlier versions and the error you get is a confusing 404 rather than a clear message.

  2. Start with /tts. It is the smallest surface and it tells you quickly whether your build has working audio support.

  3. Then point an existing realtime client at /v1/realtime rather than writing a new one. The protocol compatibility is the whole point of the endpoint.

  4. Pin the version if you touch mtmd_audio_stream. Experimental means the signature can move.

If you have not run a model locally before, the Ollama walkthrough in how to run an AI model locally is a gentler starting point, though it will not give you these endpoints. For the broader question of which local transcription quality is good enough for your use, the testing method in what speech recognition is and how to test it is the one to run on your own audio before committing.

How did this land?

About the author

Cecilia Iona
Cecilia Iona

Senior Editor, AI & Product

Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.

Share

Get the next post in your inbox

One email a month. Product updates, engineering posts, and the best of Built with Swarmz.

I agree to receive emails about AI building tips and Swarmz product news. Unsubscribe any time.