What is Ground Truth?
Human-verified labels good enough to serve as the training target.
A machine learning model learns by copying the reference labels, so if the reference is wrong the model learns the wrong thing.
That is why QC cannot be skipped on a data project. An unchecked batch of annotation is really just one person's opinion.
How ground truth is produced
Ground truth is a standard, not a file format. Labels qualify as ground truth when they have been checked well enough to be treated as correct, which in practice means a chain: a first pass by one annotator, a second pass by another, and a decision rule for the files where the two disagree. The number of passes is what the buyer is paying for.
It also comes in degrees. A single unchecked batch is one person's opinion. A batch where every file was reviewed by a second person is stronger. A batch where disagreements went to a third person and were settled by a written rule is stronger still, and that is the version a test set should be built from.
There is a cheaper substitute: labels generated by a model and spot-checked by people. These are sometimes called silver labels, and they are useful for pretraining on a large volume. They are not a substitute for human ground truth in an evaluation set, because measuring a model against another model's output measures agreement between the two, not accuracy, and the mistakes they share stay invisible.
Where it shows up in a quote
QC depth is a line item. A supplier can quote single-pass annotation, annotation with review, or annotation with review plus adjudication, and the price steps up with each. The buyer should ask which one the number reflects, and how the QC sample was chosen, because reviewing a hand-picked easy batch proves very little about the rest of the delivery.
Ask how much of the batch is actually checked. Some pipelines review every file; others review a percentage and accept the rest on the assumption that quality is uniform. The second approach is reasonable at scale, but the percentage and the rule for what happens when a reviewed file fails should be stated in the brief, not discovered at delivery.
Ground truth also decides what the data can be used for. Training tolerates some label noise; evaluation does not, because the score is only as trustworthy as the reference. If a set is going to be used to accept or reject a model, it needs the full chain, and it should be held back from training entirely.
Related terms
-
Annotation
Attaching machine-readable labels to raw data, a transcript, an intent, a speaker identity.
-
WER
Word Error Rate, the standard accuracy metric for speech recognition. Lower is better.
Keep reading
-
Full glossary
Every term we explain, in one list.
-
How to buy training data
Where these terms actually show up, and which ones change a quote.
-
Help center
How a project runs, from specification to delivery.
Submit a sourcing request
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.