How to specify a speech data project so it does not get rejected

A rejected dataset is expensive for everyone. The buyer loses weeks, the producer loses the work, and the relationship usually ends. In almost every case we have seen, the root cause was fixed before a single hour was recorded: in the specification.

The problem is that specifications feel obvious to the person writing them. You know what you mean by "natural conversational speech." The producer has a different picture, and that difference does not surface until delivery.

Write down the number that actually constrains the project

Almost every specification leads with total hours, and almost every experienced producer treats that number as the least informative one on the page.

Hours are easy to produce and easy to pad. The number that determines whether a dataset is useful is distinct speaker count, because that is what decides whether a model generalizes beyond the people in the recording.

A dataset of 500 hours from 20 speakers and a dataset of 500 hours from 500 speakers are not variations on the same product. They support different models. Write the speaker count, and write the minimum, not the target.

Define the recording conditions precisely

"Studio quality" and "natural environment" are not specifications. They are categories, and within each one the variation is wide enough to change the result.

• If studio: sample rate, microphone type, and whether all sessions use the same equipment.
• If natural: the specific environments, and a target signal-to-noise ratio rather than the word "noisy."
• If mobile: device types, and whether the recording app is fixed across all speakers.
• In all cases: whether background speech from other people is permitted, and whether it should be transcribed if present.

Specify annotation depth as a separate line item

Transcription is not one thing. It ranges from a rough orthographic pass to a full phoneme-level alignment with prosodic markup, and the cost difference between those ends is large.

The details that most often get left out: how to handle filler words, whether to normalize numbers and dates, how to treat utterances that switch language mid-sentence, and what to do when a speaker is unintelligible.

Every one of those decisions has more than one defensible answer. The specification does not need to pick the best answer. It needs to pick one, so that all annotators pick the same one.

Ask for the guideline before production starts

A producer who has done this before will have a written annotation guideline, and will send it to you on request. Read it before you commit.

If it answers the questions above, you are working with someone who has run a real project. If it does not exist, the annotators are making those decisions individually, and your dataset will contain all of their different answers.

Build in a pilot batch

The single highest-leverage clause in any data contract is a pilot batch that you can reject.

A pilot of a few hours costs little, and it surfaces every misalignment while there is still time to fix it. Specifications that looked complete on paper usually turn out to have two or three gaps, and a pilot finds them at a fraction of the cost of finding them at delivery.

Delivery format is part of the specification

Format is usually the last thing agreed and the first thing that causes a rejected delivery. It is also the cheapest thing to get right, because it costs nothing to decide early and a re-export to decide late.

State the audio container and codec, the sample rate, and the channel layout. Say whether files are one utterance each or one session each, because it changes both the annotation workflow and how you will consume the data. Specify the transcript format and whether it carries timestamps, speaker labels, and confidence values. And ask for a manifest: one row per file with duration, speaker ID, and any per-file attributes the specification requires, such as recording environment or noise condition.

The manifest is the part buyers most often forget to request and most often wish they had. Without it, every later question about the dataset becomes a manual audit of the files themselves.

Write the acceptance criteria before you write the order

Acceptance criteria are what turn a specification into something testable, and they need to be written while the project is still on paper. Two decisions do most of the work: how you will sample, and who decides when a marginal case passes.

On sampling, state the method rather than the size. A random sample of files answers a different question than a sample stratified by speaker, and if the risk you care about is one bad speaker contributing disproportionately, random sampling will miss it. For most projects the right sample is stratified by speaker with a fixed number of files per speaker, plus a random draw for overall quality.

On adjudication, name a single decision-maker on each side and a tie-break rule. "We will review it together" sounds collaborative and turns into a negotiation when the delivery is late and both parties are under pressure. A pre-agreed threshold, applied by a named person, with a defined remedy for a failed batch, is what keeps a marginal delivery from becoming a dispute.