What is Translation Data?
Parallel corpora pairing source and target language, used for machine translation training and evaluation.
Translation data is parallel text pairing a source language with a target language. It is the training material for machine translation, and the property that determines its value is the quality tier of each segment.
The quality tiers are human translation, post-edited machine translation, and raw machine translation, and they are not interchangeable. Mixed together they drag the whole set toward the weakest tier, which is why they have to be delivered separately and labelled per segment.
The uses are translation model training, quality estimation, and automatic post-editing. The third trains on exactly the distinction between raw machine output and its human correction, which requires the tiers to be preserved rather than merged.
What buyers get wrong
Quality tiers within parallel corpora vary enormously: human translation, post-edited MT, and raw MT mixed together drag the whole set down, so they have to be delivered separately.
The specification detail that decides everything
Translation method (human / post-edited / machine) must be annotated per segment, not described as a blanket note.
This is the line item that most often separates a dataset that works from one that gets re-ordered. It belongs in the specification before production starts, not in the delivery review.
How we source it
We do not hold inventory in this category. A requirement comes in, we match it to a producer who can meet the specification, and we agree the quality bar with both sides before recording begins. A pilot batch goes first so misalignments surface early.
Scenarios this covers
Buyers rarely ask for a category by name. They describe a situation, and the situation is what tells us whether this category is the right one. If yours is in the list, name it in the brief and it goes straight into the specification. If it is not, describe it anyway, because it usually means the requirement sits in a neighbouring category.
What good Translation Data looks like
Translation data is judged on per-segment provenance. A blanket note describing the corpus is not sufficient.
-
Translation method labelled per segment
Human, post-edited, or machine. Not as a corpus-level description, because most real corpora contain more than one tier.
-
Direction labelled per segment
Translation quality is not symmetric, and a corpus that does not record direction cannot be split by it.
-
Domain and register recorded
Legal text and product reviews translate differently, and a model trained across both without labels cannot be adapted to either.
-
Alignment verified
Sentence-level alignment checked on a sample with the accuracy reported, because misaligned pairs train noise.
-
Terminology consistency checked where a glossary exists
Whether the translation adhered to a supplied glossary, because that is a separate quality axis from fluency.
Public datasets in this category
These are openly documented resources, not products we sell. Each one covers part of the category, and knowing where each stops is usually what decides whether a project needs a collection of its own.
-
OPUS
A large open collection of parallel corpora gathered from many sources and aligned automatically. Breadth across language pairs is its strength and the alignment quality varies by source.
-
WMT news translation tasks
Yearly news translation test sets and training data across a set of language pairs. They define the standard evaluation and they are news-domain and prepared text.
-
Europarl
Proceedings of the European Parliament in many languages, aligned at the sentence level. Long-standing, formal, and one register throughout.
-
UN Parallel Corpus
United Nations documents across the six official languages. Formal diplomatic register, and the alignment is high quality because the documents are official translations of each other.
Questions to ask a supplier
Questions that establish whether the quality tiers were tracked or averaged.
- Is the translation method labelled per segment, or described for the corpus as a whole?
- What proportion of the corpus is human, post-edited and machine?
- Is the direction labelled per segment?
- How was alignment produced, and what is its measured accuracy?
- Was terminology consistency checked against a glossary?
The vague answer in this category concerns what the pairs actually are. A supplier who says the corpus is parallel without saying whether the source was a translation or a comparable document has described a hope. Comparable documents are not translations of each other, and alignment between them produces pairs that are merely similar.
A verifiable answer names the source of each side, the alignment method, the direction of translation, and the register. Ask for a sampled alignment error rate, because in this category a plausible-looking corpus and a usable one differ by exactly that number.
Technical attributes
The items below are the ones a specification can hold a producer to, and the ones that decide whether a delivery is usable for training rather than only present.
| Translation method | Labelled per segment, not per corpus |
|---|---|
| Direction | Labelled per segment |
| Domain and register | Recorded per segment or per document |
| Alignment | Method stated, accuracy measured on a sample |
| Terminology | Glossary adherence checked where applicable |
| Delivery format | Aligned segment pairs with per-segment provenance metadata |
The alignment method column decides the corpus quality, and it is the one most often described simply as automatic. Automatic alignment on genuinely parallel text is reliable; on comparable text it is not, and the difference is not visible in the files. The method and the sampled error rate are what make the corpus checkable.
The values come from the register the product will meet. Legal, medical and customer support text differ in vocabulary, sentence length and formality, and a model trained on one translates another poorly. The register belongs in the specification next to the language pair.
A mistake here produces a corpus that looks right and trains a model that mistranslates. The errors are not random, they concentrate where the alignment was wrong, and they are invisible in a corpus-level metric. Recovering means re-aligning, which is a full pass, or collecting again.
How Translation Data is collected
The work is in tracking provenance, which has to be captured at the moment each segment is produced.
- Source text selection by domain. The source material is chosen for the domain the model will serve, because translation conventions differ by domain.
- Translation with method recording. Each segment is produced by a known method, and the method is recorded at production rather than reconstructed afterward.
- Post-editing tracked as its own tier. Where machine output is corrected, both the raw output and the correction are retained, because the pair is what trains automatic post-editing.
- Alignment and verification. Segments are aligned by a stated method and the alignment is checked on a sample, with the accuracy reported.
- Quality assessment and delivery. A sample is assessed for adequacy and fluency, and the results are delivered with the per-segment provenance metadata.
After assembly, the pairs are aligned, sampled for alignment quality, filtered for length and language, and packaged with per-pair metadata covering the source, the direction and the register. The sampled error rate is the figure that makes the rest of the manifest interpretable.
The pilot for a translation corpus is one pair aligned and sampled for alignment quality before the full set is assembled, because the property being tested is whether the two sides are genuinely translations of each other. A short run establishes the alignment error rate on a sample, and that number decides whether the pair is usable. Comparable documents align plausibly and produce pairs that are merely similar.
Acceptance is checked on the sampled alignment error rate and on the register labels for each pair, both against figures stated before assembly. The specification should name the register and the direction in advance. Where the pilot shows the rate is too high, the source or the alignment method is changed rather than the sample widened, since a plausible corpus and a usable one differ by exactly that number.
How a delivery goes wrong
The defects below are the ones that recur across this category. Each is decided at collection or annotation time, which is why none of them can be repaired in delivery review.
-
Alignment was automatic and unchecked
Sentence alignment from comparable rather than parallel text produces pairs that are not translations of each other. A sample check with a reported error rate is what separates a usable corpus from a plausible one.
-
Direction was not recorded
A corpus of translated documents may be a translation into the target language or out of it, and the two have different characteristics. Without a direction label the corpus cannot be used for one-directional training.
-
One register presented as general
Parliamentary and legal text is formal and syntactically complex. A model trained on it translates a customer message badly, and register belongs in the specification alongside the language pair.
Why this matters to buyers
Machine translation training depends on knowing what the reference text is. A model trained on a mix of human and raw machine output learns to reproduce the machine output's errors, and the mix is invisible unless it is labelled.
The second driver is automatic post-editing, now a standard production step. It trains on the difference between raw machine output and the human correction of it, and that difference only exists in a corpus where both tiers were kept.
A translation model trained on comparable rather than parallel text produces output that reads plausibly and is wrong in the details. The errors are hard to characterise because they are not systematic in any obvious way, and the team spends a long time on decoding settings before suspecting the corpus.
The second cost is domain. A model trained on parliamentary and legal text translates a customer message badly, and the register mismatch cannot be corrected by fine-tuning on a small in-domain set when the base corpus sits far from the target. The collection has to be redone in the right register.
A translation corpus with a measured alignment error rate gives a model whose mistakes can be anticipated. The team knows the register it was built for and does not deploy it outside that register, and the direction labels mean a one-way gap is visible before launch rather than after.
The second gain is that the corpus can be extended by register. Legal, medical and support text differ in vocabulary and formality, and a delivery that documents the alignment method and the register can take a second batch without the two being mixed. That record is what lets a programme serve more than one domain.
Specification at a glance
| Category | Translation Data |
|---|---|
| Group | Non-speech data |
| Critical spec | Translation method (human / post-edited / machine) must be annotated per segment, not described as a blanket note. |
| Included by default | Pilot batch, metadata schema, source and consent documentation |
| Pricing | Quoted from specification. No published rates |
Related categories
-
LLM Training Data
Text corpora for LLM pretraining and fine-tuning: instruction data, preference data, and multi-turn dialogue data.
-
Computer Vision Data
Annotated image and video data, with annotation types including classification, bounding boxes, segmentation masks, and keypoints.
-
NLP Data
Annotated text for NLP tasks: named entities, dependency syntax, sentiment polarity, and relation extraction.
-
Multilingual Language Data
Parallel or non-parallel text corpora spanning multiple languages, used for multilingual models and low-resource language research.
-
Dictionary Data
Structured dictionary entries with definitions, parts of speech, pronunciation, and example sentences, used for dictionary products and lexical understanding tasks.
-
Text Data
General-purpose text corpora from web pages, forums, reviews and similar sources, used for language model pretraining and text classification.
Questions about Translation Data
Why label the method per segment rather than per corpus?
Because real corpora contain more than one tier. A corpus described as human-translated that contains a machine-translated section trains the model toward that section's errors, and a corpus-level note makes the mix impossible to filter out.
Should raw machine output be included at all?
Yes, as its own labelled tier. It is the input side of the post-editing pair and it is useful for quality estimation. What is not useful is machine output presented as translation.
How is alignment accuracy measured?
By sampling aligned pairs and checking them against a stated criterion, then reporting the proportion that pass. Sentence alignment is not automatic even for clean parallel text.
Does terminology consistency matter separately from fluency?
Yes. A translation can be fluent and use the wrong term for a domain-specific concept, and a model trained on it learns the wrong term. Where a glossary exists, adherence is a separate check.
Do you have Translation Data available now?
No. We do not hold inventory. Everything is produced against a specification. That means the timeline starts when the spec is agreed rather than immediately, and it also means the data matches your requirement instead of approximately matching something already built.
Can this be combined with other categories in one delivery?
Yes, and it is common. A single specification can cover several categories across several languages, with one pilot batch and one quality bar. It usually reduces total cost because contributor recruitment and project setup are shared.
What does the consent documentation cover?
Signed contributor consent stating the intended use, the collection methodology, and the annotation guideline the data was produced under. For projects involving EU data subjects the chain is built to satisfy GDPR, including the transfer mechanism. The terms for moving the data across a border.
Request Translation Data
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.