Beyond Whisper: shipping NVIDIA Parakeet in a local-first meeting scribe

Greffier is a desktop app that records your meetings and transcribes them on your own machine. Your audio stays on your computer unless you turn on cloud transcription, so the speech-to-text engine that runs there is the heart of the product. For local transcription, the default answer has been the same for years: wrap whisper.cpp, pick a model size, ship.

We shipped that answer too, and it works. But when we benchmarked our lineup, the numbers were worse than we wanted. Whisper small, our multilingual standard model, measured 7.65% word error rate on our English fixture set. base.en, the default most users actually got, measured 9.04%. Other on-device models had moved well past those numbers, and “local means a worse transcript” is the trade-off we built Greffier to refuse.

So we went looking for what else could run on a laptop CPU. Everything below was measured on our own hardware (an Apple M4 Pro, 24 GB) against our own fixture sets, and we get into their limits near the end.

The evaluation: Parakeet on sherpa-onnx

The candidate that survived was NVIDIA’s Parakeet family: transducer ASR models released under CC BY 4.0, running on sherpa-onnx, the Apache-2.0 inference runtime from the Next-gen Kaldi project, which ships ready-made int8 ONNX builds of these models.

On our English fixture set (a LibriSpeech clean-validation subset: 73 utterances, 481 seconds, scored with jiwer under standard normalization), CPU-only:

*base.en scored on Metal (its usual mode on our hardware); its CPU-only run measured 9.13%. The 110M real-time factor is derived from its measured 0.063 s latency per 8-second chunk.

Three things jumped out.

First, the smallest Parakeet more than halves the error rate of our old default. TDT 110M is smaller on disk than base.en (130 vs 142 MiB) at less than half the error rate: 4.43% against 9.04%. On this fixture set it even edges out large-v3-turbo-q5_0, our biggest Whisper model, at a quarter of the size.

Second, our high-quality Whisper tier can’t keep up without a GPU. large-v3-turbo-q5_0 runs at 1.24× real time on CPU, and that’s on an M4 Pro, which is faster than most Windows laptops. A user without a GPU who picks high quality gets a transcriber that falls further behind as the meeting goes on. Parakeet 0.6B v2 is more accurate (2.96% vs 4.78%) at RTF 0.022, about 45× faster than real time, on CPU alone.

Third, the multilingual story holds up. Parakeet 0.6B v3 covers 25 European languages. On a small German test set (Multilingual LibriSpeech, 8 samples), v3 measured 13.72% WER on CPU. Whisper small, our multilingual default, measured 18.14%, and large-v3-turbo-q5_0 11.95%, both with Metal acceleration. So v3 beats the model most of our multilingual users were getting, and nearly matches our biggest Whisper at a fraction of the compute.

Public benchmarks point the same way: the Open ASR Leaderboard (March 2026 update) puts Parakeet TDT 0.6B v2 at 6.05% average English WER against 7.83% for whisper large-v3-turbo, on a harder and more varied test suite than ours.

One smaller call: the 110M model comes in CTC and TDT decoder variants at about the same speed. We measured TDT at 4.43% against CTC’s 5.13%, and CTC produced the kind of garbling that makes you stop trusting a transcript (it once rendered “manner” as “manatter”). We ship TDT.

The engineering: a native ONNX runtime inside Electron

Benchmarking a model is a weekend. Shipping a native inference runtime inside an Electron app that must never lose a meeting is the actual work.

Our whisper.cpp pipeline transcribes in 8-second micro-batches, spawning a fresh CLI process per batch, which means re-paying model load every eight seconds. Sherpa-onnx is a Node native addon, so we can finally keep a resident recognizer alive: it lives in an Electron utilityProcess worker, loads the model once (~485 ms measured), then decodes each 8-second batch in about 120–130 ms end-to-end on our Apple Silicon test machine. (That’s the full in-app path, worker and pipeline included; the bare bench-harness decode of the same 8-second chunk is the 0.063 s behind the RTF table above. Either way, a fraction of the audio it processes.) The accumulate-to-8-seconds loop from the whisper worker ported over almost verbatim; only the “transcribe one batch” primitive changed.

The war story: “External buffers are not allowed”

Electron’s V8 configuration rejects externally-backed buffers, which rules out some of the most natural parts of the sherpa-onnx API.

Any sherpa call that returns a typed array backed by native memory throws inside Electron. Concretely: readWave(), the obvious way to feed audio in, is unusable because its sample buffer is external. So are the typed-array fields on every recognition result: timestamps, tokens, ys_log_probs. Touch any of them and the worker dies.

The safe subset turns out to be exactly what a transcription pipeline needs: allocate your own plain Float32Array (internal memory, because you created it), hand it to acceptWaveform(), and read only .text, a string, off the result. Our audio path was already producing PCM16 from the capture bus, so converting to our own Float32Array came free. But the constraint is a footgun with a long handle. It’s invisible in plain Node, where all of those APIs work fine, and one innocent-looking result.timestamps in a future refactor would take down the worker at runtime. So we wrote a static guard test that fails the build if the worker source ever references the forbidden APIs, and we keep the matching rule next to it: never load sherpa in the main process, which has the same constraint and none of the isolation.

Two smaller findings from the same spike. Sherpa-onnx 1.13.4 exposes no free() for streams or recognizers; they’re GC-managed, and a 300-batch soak (about 40 minutes of audio) showed no leak, with worker RSS peaking around 607 MB and settling back to ~278 MB. On packaging, the addon resolves its shared libraries relative to its own directory on all three platforms (@loader_path on macOS, $ORIGIN on Linux, same-directory DLL search on Windows), so no environment-variable surgery was needed.

Shipping the runtime and the models

The native payload is about 32 MB per platform: a 610 KB addon, a 4 MB C-API library, and the 27 MB ONNX Runtime that dominates it. The npm package ships that 27 MB library twice: a byte-identical, unversioned duplicate that nothing links against (we checked with otool -L). Our packaging script deletes the spare. We tried the polite fix first, a symlink, and learned that the packager rewrites relative symlinks into absolute paths that dangle on user machines. Deletion it is.

Model delivery had its own trap. Upstream distributes models as .tar.bz2 archives on a rolling GitHub release, where assets can be re-uploaded under the same name, and Node has no bzip2 support in the standard library, so extracting archives at runtime would have meant a new native dependency. We sidestepped both problems at once: a release-time script downloads the pinned upstream tarballs, verifies their SHA-256 hashes, and republishes the individual model files to our own mirror. The app then downloads plain files, each verified against its own pinned SHA-256, along the same download-verify-rename path our whisper models already used, with zero extraction code. If upstream ever silently swaps an asset, the pipeline fails a checksum instead of shipping mystery weights.

The routing: Parakeet where it wins, Whisper everywhere else

The product decision compresses into one pure function over (quality, language):

Whisper isn’t legacy here; it’s load-bearing. It covers the long tail of languages Parakeet doesn’t (CJK, Arabic, Hindi, and the rest of Whisper’s ~99), it handles auto-detection, and it’s the fallback: if the native addon fails to initialize on some unusual machine, the meeting still transcribes on Whisper. That fallback is loud, never silent. It’s recorded in the meeting’s provenance with a warning. Every meeting records which engine and model transcribed it, because a silent downgrade from 4.4% to 9% WER is something you want to diagnose, not discover.

What Whisper still does better

This section is the reason to trust the previous ones.

Language auto-detection. Parakeet models transcribe fixed language sets; they can’t tell you what language is being spoken. If you leave Greffier’s meeting language on auto-detect, you get Whisper, full stop.

The long tail. Parakeet v3’s 25 languages are European. Whisper covers ~99. For Japanese, Korean, Mandarin, Arabic, Hindi and dozens more, Whisper remains the only serious local option, and it’s what we run.

Punctuation on messy speech. On a real captured meeting (spontaneous speech, filler words, far-field mic), the 110M model punctuates and capitalizes more sparsely than base.en, producing longer lowercase runs. Content accuracy is equivalent: proper nouns, numbers and disfluencies all preserved. And the problem doesn’t show up on clean read speech. But it’s a real readability regression on the default tier, and we’re evaluating a punctuation-restoration pass.

Memory. The 0.6B Parakeet models peak around 1.4–1.8 GB of RAM while transcribing, versus ~814 MB for Whisper small. The 110M default sits in Whisper territory (~586 MB), but on an 8 GB machine the high-quality tier is a real cost.

A word about our numbers. They come from our fixture sets: read-speech corpora (LibriSpeech and Multilingual LibriSpeech), not meetings, with a German sample of just 8 files. We’ve committed a WER regression harness with hard thresholds so a future model or runtime swap can’t silently regress accuracy. Its 110M row runs end-to-end against a mirror-downloaded model, the same files users download, and measures 2.04% on the harness fixture, both full-file and in context-free 8-second chunks. What we have not measured yet: the 0.6B v2/v3 harness rows on a clean machine, real-meeting WER against ground truth, and low-end Windows hardware. We’ve also seen Parakeet v3 truncate a word once at a hard 8-second chunk boundary; rare, but on the list.

The benchmarks page has the full methodology: fixtures, scoring, hardware, the regression thresholds, and the list of what we haven’t measured.

Where this leaves Greffier

If you pick English or a European language, you now get a transcript that’s more accurate than our old Whisper lineup, from a model smaller than the one it replaced, faster than real time on a CPU, with your audio never leaving the machine. Pick anything else and you get Whisper, same as before, and the app tells you which engine ran.

Greffier is a local-first meeting scribe for macOS (Windows soon, Linux later). It records, transcribes, and summarizes, all on your machine. It costs $5, once: no subscription, no account. Download it here.

Attribution: the Parakeet models are © NVIDIA Corporation, licensed under CC BY 4.0; we ship int8 ONNX conversions produced by the sherpa-onnx / Next-gen Kaldi project (Apache 2.0). Multilingual and fallback transcription uses whisper.cpp and OpenAI Whisper models (MIT). Full notices ship with the app.