Measured, not marketed.

English accuracy

Word error rate on the English fixture set described above — lower is better. All rows are CPU-only except where footnoted; Parakeet models are int8 ONNX builds running on sherpa-onnx.

* base.en scored in its usual Metal mode on our hardware; its CPU-only run measured 9.13% WER at RTF 0.169. † The 110M real-time factor is derived from its measured 0.063 s latency per 8-second chunk; the other RTFs are measured directly. Whisper timings include per-invocation model load, because that is how the shipped whisper pipeline actually ran (a fresh process per batch); sherpa-onnx models run as a persistent recognizer, which is how Greffier ships them.

Real-time factor (RTF) is processing time divided by audio duration — an RTF of 0.022 means one hour of audio transcribes in about 80 seconds; anything above 1.0 falls progressively behind a live meeting. Note the structural point in the third row: our old "high quality" Whisper tier could not keep up with real time on CPU — on an M4 Pro's CPU, which is faster than most Windows laptops. Parakeet 0.6B v2 is more accurate than that model and roughly 45× faster than real time, on CPU alone.

European languages (German fixture)

Parakeet 0.6B v3 covers 25 European languages and serves both quality tiers for the 24 non-English ones. Our multilingual check used the Multilingual LibriSpeech German test set — only 8 samples / 123 seconds, so treat these as directional, not precise. They are consistent with published multilingual benchmarks (see the leaderboard note below).

Read the asymmetry honestly: v3 on CPU clearly beats the model most multilingual users were actually getting (Whisper small, here with GPU acceleration), and lands close to our largest Whisper model at a fraction of the compute — but it does not beat large-v3-turbo on this small fixture.

What it costs: memory and disk

Accuracy isn't free. Peak resident memory measured while transcribing 8-second chunks:

The default 110M model costs about 246 MB more RAM than base.en for less than half its error rate, at a comparable download size. The 0.6B models are a real memory cost — fine on 16 GB machines, tight on 8 GB machines running alongside an Electron app — which is exactly why they are the opt-in high-quality tier, not the default.

Methodology

Fixtures

English: a LibriSpeech clean-validation subset — 73 utterances, 481 seconds, with ground-truth reference transcripts. German: the Multilingual LibriSpeech German test set — 8 samples, 123 seconds. Both corpora are public and both are read speech (audiobook recordings), not meetings; what that does and doesn't tell you is covered under "What we haven't measured" below.

Metric

Word error rate computed with jiwer under standard normalization (case and punctuation folded), hypothesis vs. ground-truth reference, aggregated over each fixture set.

Hardware, and why CPU-only

All measurements ran on an Apple M4 Pro with 24 GB of RAM. We report CPU-only numbers as the primary column even though this machine has a GPU, because a large share of real users — most Windows laptops — have no usable GPU acceleration, and a model that only keeps up with a GPU would be misleading to advertise. Where a Whisper row was measured with Metal acceleration instead, it is footnoted as such. One honest caveat cuts the other way: an M4 Pro's CPU is itself faster than most laptop CPUs, so treat absolute speeds as optimistic and the relative ordering as the transferable result.

Runtimes and quantization

Parakeet models are the int8-quantized ONNX conversions published by the sherpa-onnx project — the exact builds Greffier downloads and runs — executed with a persistent recognizer on 4 threads. Whisper models ran through whisper.cpp the way the app shipped them: a fresh process per batch, so those timings include model load on every invocation. That difference is part of the product comparison, not an unfair handicap — it is what each engine's real pipeline did.

The 8-second-chunk case

Greffier transcribes live meetings in 8-second micro-batches, so we also measured that exact shape: a 62.5-second clip split into hard 8-second chunks, each decoded without context. Per-chunk latency on CPU: Parakeet 110M 0.063 s, Parakeet 0.6B v2 0.17 s — and Whisper large-v3-turbo-q5_0 8.22 s per 8-second chunk, which cannot keep up. Hard cuts can split a word at each boundary, so chunked decoding is also asserted in our regression harness with an explicit +4-point WER allowance.

Regression thresholds, committed to the codebase

The numbers above are not a one-off press run. The app's codebase carries a WER regression harness with hard, committed thresholds — if a future model or runtime swap exceeds them, the harness exits non-zero:

Two honest details about that harness. First, its fixture is different from the tables above: it synthesizes clean, single-speaker reference audio per language — a deliberately easy case, which is why its 110M measurement (2.04%) is better than the LibriSpeech figure (4.43%). It exists to catch regressions and gross failures, not to generate marketing numbers. Second, it is opt-in and manual — it is not part of the default test run, and the 0.6B rows have thresholds committed but have not yet been executed on a clean machine. The 110M row has been run end-to-end against the same mirror-downloaded model files users receive.

Reproducibility

The methodology on this page is complete: the corpora are public, the Parakeet builds are the publicly published sherpa-onnx int8 conversions, and the scoring is standard jiwer WER. Anyone with the same public data and models can check our numbers. The harness, fixture pins, and thresholds themselves live in the app's codebase — Greffier is commercial software, so that code isn't public, and we don't claim otherwise; what we commit to is that those thresholds gate what we ship.

What Whisper still does better

This section is the reason to trust the ones above. Greffier ships whisper.cpp alongside Parakeet — not as legacy, but because Whisper wins outright in several places:

• Language auto-detection. Parakeet models transcribe fixed language sets and cannot detect what is being spoken. Leave Greffier's meeting language on auto-detect and you get Whisper, full stop.
• The long tail. Parakeet v3's 25 languages are European. Whisper covers ~99 — for Japanese, Korean, Mandarin, Arabic, Hindi and roughly 70 more, Whisper remains the only serious local option, and it is what Greffier runs for them.
• Punctuation on messy speech. On a real captured meeting (spontaneous speech, far-field mic), the 110M model punctuated and capitalized more sparsely than base.en. Content accuracy was equivalent, and the effect does not appear on clean read speech — but it is a real readability regression on the default tier, and we are evaluating a punctuation-restoration pass.
• Memory at the high tier, per the table above.

Whisper is also the always-available fallback: if the native Parakeet runtime fails to initialize on some machine, the meeting still transcribes on Whisper — and the app records loudly, in the meeting's provenance, which engine and model actually ran.

What we haven't measured

• Real-meeting WER with ground truth. Our fixture sets are read speech. We ran a qualitative A/B on a 90-second slice of a real meeting (content accuracy was equivalent across engines; the punctuation note above came from it), but we have no ground-truth meeting corpus of our own yet. Published meeting-corpus results (e.g. AMI) show the same model ordering, which is reassuring but is not our measurement.
• Low-end Windows hardware. CPU-only mode on an M4 Pro approximates a GPU-less laptop, but an M4 Pro's CPU is faster than most. Absolute latencies on a budget machine will be worse; the orderings should hold.
• The 0.6B v2/v3 regression-harness rows, as stated in the methodology — thresholds committed, not yet executed locally.
• German at scale. 8 samples is a sanity check, not a study. External corroboration: the Open ASR Leaderboard (as of its March 2026 update) reports Parakeet TDT 0.6B v2 at 6.05% average English WER versus 7.83% for whisper large-v3-turbo across its harder, more varied suite — directionally consistent with what we measured on our own fixtures.
• A rare chunk-boundary artifact. We observed Parakeet v3 truncate a word at a hard 8-second chunk boundary once. It is on the list (overlap or VAD-aligned batching are the candidate fixes).

Why not just wrap whisper.cpp?

Most local transcription apps are, at the engine level, the same app: a UI over whisper.cpp with a model-size picker. There is nothing wrong with that — whisper.cpp is excellent, we shipped exactly that for a long time, and it still runs inside Greffier today. But the accuracy/speed frontier for on-device ASR moved past it for English and European languages, and shipping past the wrapper is what the numbers above buy: less than half the error rate of the old default at a comparable download size, and a high-quality tier that beats our biggest Whisper model while staying dozens of times faster than real time on a plain CPU. Getting there took a native ONNX runtime inside Electron, per-language model routing, checksummed model delivery, and a loud fallback story — that engineering write-up is Beyond Whisper.