How to Evaluate Speech-to-Text Accuracy With Your Own Recordings

The best speech-to-text accuracy test starts with recordings you actually expect to make. A perfect transcription of a carefully read sentence tells you little about your project names, numbers, hesitations, microphone, or mixed-language updates.

Use this practical method to compare the dictation workflow you need. Build a short sample set around your everyday writing, keep the scoring consistent, and include names, numbers, and specialist terms so the result helps you choose your setup.

Decide what counts as a correct result

Create a reference transcript by listening to the recording and checking every spoken word. Google's accuracy guidance describes this human-provided reference as the basis for comparing machine output. An incorrect reference also produces an incorrect evaluation. Google speech accuracy guidance.

Keep two targets separate. For recognition, evaluate whether the words match the audio. For a writing workflow, evaluate whether the resulting draft preserves the intended meaning while following your formatting instructions. Removing a deliberate repetition may help an email but would fail a reference that requires every spoken word.

Write your comparison rules before seeing the results. Decide how to handle capitalization, punctuation, and spoken numbers. For example, choose whether “twenty four” and “24” should be equivalent in your test. Apply the same rule to every output, and retain the original text so another reviewer can understand your choices.

Build a small test that exposes useful differences

Start with twelve short clips divided across four conditions. Use three ordinary updates, three clips containing names or specialized terms, three with important numbers or negation, and three from your normal working environment. Add mixed-language clips if that is part of your work.

Keep the audio identical when comparing tools that accept the same input. If you must record separately in each app, note that limitation: differences in speaking pace or microphone placement may explain part of the result. Record the date, app version, selected recognizer, language setting, and enabled cleanup options.

Make the test repeatable without making it artificial. Include natural pauses and self-corrections. Do not choose only the clearest takes after hearing the result. A useful evaluation should resemble the recordings you will ask the tool to handle tomorrow.

Calculate word error rate

Word error rate, or WER, counts substitutions, deletions, and insertions relative to the number of words in the reference. NIST's OpenSAT evaluation plan uses this calculation for its speech recognition task. Lower values indicate fewer word errors under the chosen comparison rules. NIST OpenSAT evaluation plan.

WER = (substitutions + deletions + insertions) / reference words

Consider this invented six-word example:

Reference: Please send Mina the revised draft
Output:    Please send Nina the draft now

“Mina” becomes “Nina,” “revised” disappears, and “now” is added. That is one substitution, one deletion, and one insertion: 3 / 6 = 50% WER. This is illustrative arithmetic, not a result from a speech model.

For a collection of clips, total the errors and reference words before dividing. Simply averaging clip percentages gives a short clip the same influence as a long one. Keep the per-clip results too, so a good aggregate cannot conceal a failure on the language or vocabulary you need.

Track consequential errors and correction effort

Add a second review alongside WER. Mark incorrect names, quantities, dates, and changes to uncertainty or negation. “Do not publish yet” and “Publish yet” have radically different implications even though the missing phrase is short.

Use a simple record for each clip:

Avoid turning one personal result into a claim about every accent or environment. Keep unresolved audio ambiguities marked, and have a second listener check examples that affect the decision.

Change one setting and repeat

In Sonicribe, start with Exactly as spoken, which has no AI formatting pass. Check any enabled text transformations when constructing a fidelity test. Then compare a language setting or a small vocabulary list, changing one factor at a time. Hints are best effort and depend on the provider.

Use the model guide and user guide for the current controls. Evaluate formatted drafts separately using the meaning and effort columns. Keep whichever configuration improves your observed work, then retest when the model or workflow changes.

Sources reviewed October 8, 2026.