Beyond Whisper: shipping NVIDIA Parakeet in a local-first meeting scribe

Greffier is a desktop app that records your meetings and transcribes them on your own machine. No audio leaves the device, so the local speech-to-text engine isn’t a feature of the product — it more or less is the product. And local transcription today is dominated by one answer: wrap whisper.cpp, pick a model size, ship.

We shipped that answer too. It works, and whisper.cpp is a remarkable piece of engineering. But when we benchmarked our own lineup honestly, the numbers said we were leaving a lot of accuracy on the table: our multilingual standard model, Whisper small, measured 7.65% word error rate on our English fixture set, and base.en — the default we actually installed for most users — measured 9.04%. That lineup was no longer the accuracy/speed frontier for on-device transcription, and “local means a worse transcript” is exactly the trade-off we built Greffier to refuse.

So we went looking for what else could run on a laptop CPU. This is the story of what we found, what it took to ship it inside an Electron app, and what we still haven’t measured. All numbers below were measured on our own hardware (an Apple M4 Pro, 24 GB) against our own fixture sets; we describe those sets and their limits honestly further down.

The evaluation: Parakeet on sherpa-onnx

The candidate that survived was NVIDIA’s Parakeet family — transducer-based ASR models released under CC BY 4.0 — running on sherpa-onnx, the Apache-2.0 inference runtime from the Next-gen Kaldi project, which ships ready int8 ONNX builds of these models.

On our English fixture set (a LibriSpeech clean-validation subset: 73 utterances, 481 seconds, scored with jiwer under standard normalization), CPU-only:

*base.en scored on Metal (its usual mode on our hardware); its CPU-only run measured 9.13%. The 110M real-time factor is derived from its measured 0.063 s latency per 8-second chunk.

Three things jumped out.

First, the little model beats our old default by more than double. Parakeet TDT 110M is smaller on disk than base.en (134 vs 142 MB) with less than half its error rate — 4.43% vs 9.04%. On this fixture set it even edged out large-v3-turbo-q5_0, our largest Whisper model, at a quarter of the size.

Second, our “high quality” Whisper tier couldn’t actually keep up without a GPU. large-v3-turbo-q5_0 runs at a real-time factor of 1.24 on CPU — on an M4 Pro’s CPU, which is faster than most Windows laptops. A GPU-less user selecting high quality got a pipeline that falls progressively behind the meeting. Parakeet 0.6B v2 delivers better accuracy (2.96% vs 4.78%) at RTF 0.022 — roughly 45× faster than real time, on CPU alone.

Third, the multilingual story holds up. Parakeet 0.6B v3 covers 25 European languages. On a German test set (Multilingual LibriSpeech, 8 samples — small, and we say so below), v3 measured 13.72% WER on CPU, against 18.14% for Whisper small — our multilingual standard default — and 11.95% for large-v3-turbo-q5_0, both of those running with Metal acceleration. So v3 clearly beats the model most of our multilingual users were actually getting, and comes close to our biggest model at a fraction of the compute.

Our measurements also agree directionally with the public record: the Open ASR Leaderboard (as of its March 2026 update) reports Parakeet TDT 0.6B v2 at 6.05% average English WER versus 7.83% for whisper large-v3-turbo across its harder, more varied test suite.

One sub-decision worth recording: Parakeet’s 110M model comes in CTC and TDT decoder variants at essentially the same speed. We measured TDT at 4.43% vs CTC’s 5.13%, and CTC produced the kind of garbling that erodes trust in a transcript (it once rendered “manner” as “manatter”). We ship TDT.

The engineering: a native ONNX runtime inside Electron

Benchmarking a model is a weekend. Shipping a native inference runtime inside an Electron app that must never lose a meeting is the actual work.

Our whisper.cpp pipeline transcribes in 8-second micro-batches, spawning a fresh CLI process per batch — which means re-paying model load every eight seconds. Sherpa-onnx is a Node native addon, so we could finally hold a resident recognizer: it lives in an Electron utilityProcess worker, loads the model once (~485 ms measured), and then decodes each 8-second batch in roughly 120–130 ms end-to-end on our Apple Silicon test machine. (That’s the full in-app path — worker, pipeline and all; the bare bench-harness decode of the same 8-second chunk is the 0.063 s behind the RTF table above. Either way, a fraction of the audio it processes.) The accumulate-to-8-seconds loop from the whisper worker ported over almost verbatim; only the “transcribe one batch” primitive changed.

The war story: “External buffers are not allowed”

Electron’s V8 configuration rejects externally-backed buffers — and that constraint quietly forbids some of the most natural parts of the sherpa-onnx API.

Any sherpa call that returns a typed array backed by native memory throws inside Electron. Concretely: readWave(), the obvious way to feed audio in, is unusable — its sample buffer is external. So are the typed-array fields on every recognition result: timestamps, tokens, ys_log_probs. Touch any of them and the worker dies.

The safe subset turns out to be exactly what a transcription pipeline needs and nothing more: allocate your own plain Float32Array (internal memory, because you created it), hand it to acceptWaveform(), and read only .text — a string — off the result. Our audio path was already producing PCM16 from the capture bus, so converting to our own Float32Array was natural. But the constraint is a footgun with a long handle: it’s invisible in plain Node (where all of those APIs work fine), and one innocent-looking result.timestamps in a future refactor would take down the worker at runtime. So we wrote a static guard test that runs in every test pass and fails the build if the worker source ever references the forbidden APIs — and the rule “never load sherpa in the main process” is documented in the same breath, because the main process has the same constraint and none of the isolation.

Two adjacent findings from the same spike: sherpa-onnx 1.13.4 exposes no free() for streams or recognizers — they’re GC-managed — and a 300-batch soak (about 40 minutes of audio) showed no leak, with worker RSS peaking around 607 MB and settling back to ~278 MB. And on the packaging side, the addon resolves its shared libraries relative to its own directory on all three platforms (@loader_path on macOS, $ORIGIN on Linux, same-directory DLL search on Windows), so no environment-variable surgery was needed.

Shipping the runtime and the models

The native payload is about 32 MB per platform: a 610 KB addon, a 4 MB C-API library, and the 27 MB ONNX Runtime that dominates it. The npm package actually ships that 27 MB library twice — a byte-identical, unversioned duplicate that nothing links against (we checked with otool -L) — so our packaging script deletes the spare. We tried the polite fix first, a symlink, and learned that the packager rewrites relative symlinks into absolute paths that dangle on user machines. Deletion it is.

Model delivery had its own trap. Upstream distributes models as .tar.bz2 archives on a rolling GitHub release — assets can be re-uploaded under the same name — and Node has no bzip2 support in the standard library, so extracting archives at runtime would have meant a new native dependency. We sidestepped both problems at once: a release-time script downloads the pinned upstream tarballs, verifies their SHA-256 hashes, and republishes the individual model files to our own mirror. The app then downloads plain files, each verified against its own pinned SHA-256 — the exact download-verify-rename path our whisper models already used, with zero extraction code. If upstream ever silently swaps an asset, the pipeline fails a checksum instead of shipping mystery weights.

The routing: Parakeet where it wins, Whisper everywhere else

The product decision compresses into one pure function over (quality, language):

Whisper isn’t legacy here — it’s load-bearing. It covers the long tail of languages Parakeet doesn’t (CJK, Arabic, Hindi, and the rest of Whisper’s ~99), it handles auto-detection, and it is the fallback: if the native addon fails to initialize on some exotic machine, the meeting still transcribes on Whisper. Crucially, that fallback is loud — it’s recorded in the meeting’s provenance along with a warning, never silently. Every meeting records which engine and model actually transcribed it, because a silent downgrade from 4.4% to 9% WER is the kind of thing you want to be able to diagnose, not discover.

What Whisper still does better

This section is the reason to trust the previous ones.

Language auto-detection. Parakeet models transcribe fixed language sets; they can’t tell you what language is being spoken. If you leave Greffier’s meeting language on auto-detect, you get Whisper, full stop.

The long tail. Parakeet v3’s 25 languages are European. Whisper covers ~99. For Japanese, Korean, Mandarin, Arabic, Hindi and dozens more, Whisper remains the only serious local option, and it’s what we run.

Punctuation on messy speech. On a real captured meeting — spontaneous speech, filler words, far-field mic — the 110M model punctuated and capitalized more sparsely than base.en, producing longer lowercase runs. Content accuracy was equivalent (proper nouns, numbers and disfluencies all preserved), and the behavior doesn’t appear on clean read speech, but it’s a real readability regression on the default tier and we’re evaluating a punctuation-restoration pass.

Memory. The 0.6B Parakeet models peak around 1.4–1.8 GB of RAM while transcribing, versus ~814 MB for Whisper small. The 110M default is comparable to Whisper territory (~586 MB), but on an 8 GB machine the high-quality tier is a real cost, and we say so rather than hide it.

And about our numbers themselves: they come from our fixture sets — read-speech corpora (LibriSpeech and Multilingual LibriSpeech), not meetings, with a German sample of just 8 files. We’ve committed a WER regression harness with hard thresholds so a future model or runtime swap can’t silently regress accuracy; its 110M row has been executed end-to-end against a mirror-downloaded model — the same files users download — measuring 2.04% on the harness fixture, both full-file and in context-free 8-second chunks. Things we have not yet measured: the 0.6B v2/v3 harness rows on a clean machine, real-meeting WER against ground truth, and low-end Windows hardware. We’ve also observed Parakeet v3 once truncate a word at a hard 8-second chunk boundary — rare, but on the list.

The benchmarks page has the full methodology — fixtures, scoring, hardware, the committed regression thresholds, and an explicit list of what we haven’t measured.

Where this leaves Greffier

If you pick English or a European language, you now get a transcript that’s measurably more accurate than our old Whisper lineup — from a model that’s smaller than the one it replaced, faster than real time on a CPU, with your audio never leaving the machine. If you pick anything else, you get Whisper, same as before, and the app tells you exactly which engine ran.

Greffier is a local-first meeting scribe for Windows, macOS, and Linux: it records, transcribes on-device, and summarizes locally too. It costs $5, once — no subscription, no account, no audio upload. You can download it here.

Attribution: the Parakeet models are © NVIDIA Corporation, licensed under CC BY 4.0; we ship int8 ONNX conversions produced by the sherpa-onnx / Next-gen Kaldi project (Apache 2.0). Multilingual and fallback transcription uses whisper.cpp and OpenAI Whisper models (MIT). Full notices ship with the app.