What is WER?

WER is calculated by comparing the system's output against the reference transcript word by word, counting substitutions, deletions and insertions, then dividing that total by the word count.

One caution when buying: the WER a supplier quotes is usually measured on their own test set. To judge the data, ask under what conditions it was measured.

Where WER is measured

A WER figure has no meaning on its own. It is a property of a system measured on a particular set of audio, and the same system scores differently on clean read speech and on accented conversation in a noisy room. So the first question about any quoted number is what it was measured on, and whether that audio appeared in training.

The second question is the transcript convention. WER counts differences between the system output and a reference text, and the reference text is a human product. If the reference drops filler words and the system writes them, every filler becomes an error that is really a disagreement about conventions. Normalising case, punctuation and number formatting before scoring moves the figure too.

WER also has a family of relatives. Character error rate counts characters instead of words, which suits languages written without spaces and tasks where spelling matters. And WER is not capped at 100 percent, because inserted words are added to the numerator, so a system that produces far more words than were spoken can score above 100. Latency and real-time factor are separate axes.

What to ask before buying

Ask a supplier to name the test set behind their number. If it is a held-out split of the same collection the model trained on, the figure describes performance on familiar conditions and will be optimistic. A number measured on public audio the supplier never trained on is a harder and more useful claim. If they cannot say what the set was, it is a marketing number.

When you buy data rather than a system, WER enters as an acceptance criterion. The contract can require that a delivered transcript stay below a stated error rate on a held-out sample, with the measurement convention written down. Suppliers check their own work the same way: a second transcriber redoes a sample, the two texts are compared, and the disagreements are counted.

Two comparisons worth refusing. Do not compare two suppliers whose numbers were measured on different sets, because the gap may be the audio rather than the work. And do not accept an accuracy target without asking which conditions it applies to. A target set on clean studio audio is met by different data, and a different price, from the same target on noisy recordings.