Benchmarks · Measured November 2026
Published results. Conditions included.
Does Murmur’s cleanup keep what you said, and how long does it take? Each answer says how it was measured.
Cleanup that keeps your meaning
Murmur Cloud and on your Mac
Murmur’s cleanup is built to keep your names, numbers and “not” intact. On real dictation, none were lost.
We ran 60 real dictations from one person through each cleanup style, three times each. Murmur Cloud’s Polish and Professional styles broke a rule in 0 of 359 runs. Natural, which runs on your Mac, broke a rule in 0 of 180.
A run fails if the rewrite answers or obeys what you dictated, changes or drops a name, number, negation or quotation, applies a spoken self-correction more than once, copies text from your screen, or returns anything but plain text.
Natural uses a small model on your Mac. Murmur checks each result, and when the model drops something, it falls back to simpler rule-based cleanup. That happened in 54 of the 180 runs. Rule-based cleanup on its own also had 0 failures on the 60 dictations.
Hand-written trick sentences are harder. Local cleanup failed 9 of 72 (rules alone failed 5 of 24), missing a “not” and two weekday names. Murmur Cloud failed 1 of 420 Professional and 1 of 348 Polish runs on similar sentences. In one run, Apple’s on-device model failed 8 of 35 Professional and 8 of 29 Polish cases. Faithful never calls a model, so it has no failure count.
Speed on a Mac
Apple silicon · median times
With cleanup on your Mac, 5 seconds of speech takes about 0.26 s to become clean text.
Processing time for a 5 s, 15 s and 45 s clip, with the speech model already loaded:
Natural, rule-based: 0.06 s, 0.15 s, 0.38 s
Natural, local model: 0.26 s, 0.72 s, 1.99 s
Murmur Cloud Polish: 1.17 s, 1.35 s, 1.78 s
Murmur Cloud Professional: 1.00 s, 1.35 s, 2.03 s
Local cleanup is fastest for short dictations, and Murmur Cloud catches up around 45 seconds. The clips were synthetic speech, 20 per length, and the times leave out typing the text into your app.
In everyday use, across 70 real dictations with Murmur Cloud cleanup, the median time from letting go of the key to seeing text was about 1.5 s. One in twenty took about 3.2 s or longer.
All of this comes from a single Mac on one network connection. Your Mac, connection and voice will give different numbers. None of these figures compare Murmur with other apps.
Read the method
Open source
Murmur Cloud results use its model (gpt-oss-20b) and prompt. Natural uses the Vedu scribe 0.8b model. All tests ran on Murmur 0.3.3.
Test cases, scoring rules and full tables are in the Murmur repository: the cleanup benchmark and the latency benchmark. The real dictations stay private.
For what Murmur Cloud receives, see privacy. To choose a style, see the dictation guide.