What is NLP Data?
Annotated text for NLP tasks: named entities, dependency syntax, sentiment polarity, and relation extraction.
NLP data is text annotated for a linguistic or semantic task: named entities, syntactic structure, sentiment, relations, or intent. The annotation scheme is the product, and the text is the medium.
The defining property is that quality lives in the guideline. For speech, quality lives substantially in the recording; for NLP data the text is usually given and the entire value is in how consistently the scheme was applied to it.
The uses are training and evaluating models for the specific task the scheme describes. Because schemes are task-specific, a corpus annotated for one task is rarely reusable for another, which is why the guideline document is a deliverable rather than an internal artifact.
What buyers get wrong
The quality of the annotation guideline decides everything. Write it vaguely and the annotated data is unusable.
The specification detail that decides everything
Whether the annotation guideline document and inter-annotator agreement (IAA) figures are provided.
This is the line item that most often separates a dataset that works from one that gets re-ordered. It belongs in the specification before production starts, not in the delivery review.
How we source it
We do not hold inventory in this category. A requirement comes in, we match it to a producer who can meet the specification, and we agree the quality bar with both sides before recording begins. A pilot batch goes first so misalignments surface early.
Scenarios this covers
Buyers rarely ask for a category by name. They describe a situation, and the situation is what tells us whether this category is the right one. If yours is in the list, name it in the brief and it goes straight into the specification. If it is not, describe it anyway, because it usually means the requirement sits in a neighbouring category.
What good NLP Data looks like
NLP data is judged on the guideline and on the agreement figures that show it was applied.
-
The annotation guideline delivered as a document
Not a summary, the document itself, including the boundary cases and the decisions taken on them.
-
Inter-annotator agreement reported per class
Overall agreement hides the classes where the scheme is weak, and those are the classes a model will fail on.
-
Adjudication described
How disagreements were resolved and by whom, because a corpus where disagreements were averaged is a different artifact from one where they were decided.
-
Negative and hard cases included
Sentences that are genuinely ambiguous are the useful ones, and a corpus of clear examples does not test a model.
-
Scheme version recorded
A scheme revised mid-project produces two corpora in one delivery unless the version is tracked per record.
Public datasets in this category
These are openly documented resources, not products we sell. Each one covers part of the category, and knowing where each stops is usually what decides whether a project needs a collection of its own.
-
Wikipedia dumps
Full periodic exports of every language edition, with structure and edit history. The most widely used open text resource and its editorial standards make it cleaner and narrower than the open web.
-
Common Crawl
The open web at scale, unfiltered. Every web-text dataset is a filtering decision applied to this, and the filtering is where the value and the risk both sit.
-
OSCAR
Web text filtered and classified by language from Common Crawl, covering well over a hundred languages. Convenient, and its per-language quality varies with how much web text that language has.
-
GLUE and SuperGLUE
Benchmark suites for language understanding across several task types. They are evaluation sets and their saturation is why newer benchmarks keep appearing.
Questions to ask a supplier
Questions that establish whether the scheme is documented and whether it survived contact with real text.
- Is the annotation guideline delivered as a document, including boundary cases?
- What is the inter-annotator agreement, reported per class?
- How were disagreements adjudicated, and by whom?
- Are genuinely ambiguous examples included, and how are they marked?
- Is the scheme version recorded per record?
The vague answer in this category concerns the language labels. A supplier who says the corpus is English without saying how the language was determined has described an assumption. Web documents mix languages, and a document-level label puts multilingual documents into a single-language bucket.
A verifiable answer states the language identification method, the measured language distribution, the boilerplate removal method and the benchmark decontamination result. Ask for the distribution as measured rather than as intended, because the two differ in every corpus assembled from the open web.
Technical attributes
The items below are the ones a specification can hold a producer to, and the ones that decide whether a delivery is usable for training rather than only present.
| Annotation scheme | Delivered as a document with boundary cases |
|---|---|
| Agreement | Inter-annotator, reported per class |
| Adjudication | Method and adjudicator described |
| Ambiguous cases | Included and marked |
| Scheme version | Recorded per record |
| Delivery format | Text with annotations, plus the guideline document |
The boilerplate removal column decides the token distribution, and it is the one most often skipped. Navigation, cookie notices and footers repeat across millions of pages, and left in place they dominate the frequent-token distribution. A model trained on them generates them.
The values come from the task and the language. A model for classification needs less volume and more balance, while a model for generation needs volume and tolerates imbalance. The thresholds should be set against the task rather than copied from a published pipeline built for a different purpose.
A mistake here is measured as contamination. Web-sourced corpora frequently contain the evaluation sets, and a model tested on data it was trained on reports scores that do not reflect generalisation. Decontamination has to be done against the specific benchmarks the model will be evaluated on.
How NLP Data is collected
The scheme is written, tested against real text, and revised before the main annotation run.
- Scheme design from the task definition. Labels and their definitions are derived from what the model has to do, not from what is convenient to annotate.
- Pilot annotation on real text. A pilot batch is annotated and the disagreements are used to find where the scheme is under-specified.
- Scheme revision and versioning. The scheme is revised and versioned, because a scheme that changes mid-project produces inconsistent data unless the change is tracked.
- Main annotation with agreement measurement. Annotation runs with agreement measured per class during the run rather than only at the end, so a weak class is caught while it can still be fixed.
- Adjudication and delivery. Disagreements are adjudicated by a named person against the scheme, and the guideline document is delivered with the data.
After assembly, the corpus is language-identified at segment level, boilerplate is removed, documents are deduplicated, and the files are packaged with a manifest stating the measured language and domain distributions. The language measurement is what makes the corpus comparable with others.
The pilot for a text corpus is a sample run through language identification, boilerplate removal and deduplication, with the measured distributions compared against the specification before the full assembly. Its purpose is to establish the language mix and the boilerplate share, because those two decide whether the corpus is what the specification described. Both are measured rather than intended.
Acceptance is checked on the measured language and domain distributions and on the decontamination result against the evaluation sets. The specification should name the benchmarks and the thresholds before assembly starts. Where the pilot shows the distribution is off, the sources are reweighted rather than the manifest rewritten, since a corpus described differently from what it is produces a model that fails where nobody looked.
How a delivery goes wrong
The defects below are the ones that recur across this category. Each is decided at collection or annotation time, which is why none of them can be repaired in delivery review.
-
Language identification was done at document level
Web documents mix languages. A document-level label puts multilingual documents into one language bucket, which then contaminates the training data for that language and is invisible in aggregate statistics.
-
Boilerplate was not removed
Navigation, cookie notices and footers repeat across millions of pages. Left in, they dominate the frequent-token distribution and the model learns to generate them.
-
Benchmark contamination
Web-sourced corpora frequently contain the evaluation sets. Without decontamination against the benchmarks the model will be tested on, the reported scores are not measuring generalisation.
Why this matters to buyers
The guideline is what makes the annotations mean anything. Two corpora with the same labels but different schemes are not comparable, and a model trained on one is not transferable to the other. The document is what makes the corpus usable by someone who was not in the room.
The second driver is that scheme failures are invisible without agreement figures. A corpus can look complete and be internally inconsistent, and the only evidence is the agreement measurement per class.
A model trained on a corpus with boilerplate and contamination reports quality it does not have. The gap appears when it meets real input, and by then the training run and the evaluation cycle have both been paid for. Teams usually respond by training again, which is the most expensive way to fix a data problem.
The second cost is per-language performance. A corpus described as multilingual with document-level labels performs unevenly across languages, and the unevenness is invisible in aggregate. Markets served by the weaker languages report it as a product defect long before anyone looks at the corpus.
A text corpus with measured distributions and a decontamination report gives evaluation results that mean something. The team can compare its model against published work without the comparison being invalidated by leakage, and per-language performance is visible before the markets report it.
The second gain is that the corpus can be rebalanced. Sources and thresholds are documented, so a second batch aimed at an under-represented language or domain can be assembled to complement the first rather than replace it. That record is what makes a text programme cumulative.
Specification at a glance
| Category | NLP Data |
|---|---|
| Group | Non-speech data |
| Critical spec | Whether the annotation guideline document and inter-annotator agreement (IAA) figures are provided. |
| Included by default | Pilot batch, metadata schema, source and consent documentation |
| Pricing | Quoted from specification. No published rates |
Related categories
-
LLM Training Data
Text corpora for LLM pretraining and fine-tuning: instruction data, preference data, and multi-turn dialogue data.
-
Computer Vision Data
Annotated image and video data, with annotation types including classification, bounding boxes, segmentation masks, and keypoints.
-
Multilingual Language Data
Parallel or non-parallel text corpora spanning multiple languages, used for multilingual models and low-resource language research.
-
Translation Data
Parallel corpora pairing source and target language, used for machine translation training and evaluation.
-
Dictionary Data
Structured dictionary entries with definitions, parts of speech, pronunciation, and example sentences, used for dictionary products and lexical understanding tasks.
-
Text Data
General-purpose text corpora from web pages, forums, reviews and similar sources, used for language model pretraining and text classification.
Questions about NLP Data
Why deliver the guideline document rather than a summary?
Because the boundary cases are the substance. A summary describes the labels; the document records the decisions taken where the labels were ambiguous, which is what a user needs to apply the corpus correctly.
What is a reasonable agreement figure?
It depends on the task, and a threshold is less useful than the per-class breakdown. What matters is that agreement was measured, that it is reported per class, and that the low-agreement classes are visible to the buyer.
How are ambiguous examples handled?
They are included and marked rather than removed. Ambiguous sentences are where a model is most likely to fail, and a corpus that has filtered them out produces an evaluation that overstates performance.
Can the corpus be reused for a different task?
Rarely without re-annotation. Annotation schemes are task-specific, and text annotated for one set of labels is not annotated for another. That is why the scheme document is a deliverable: it is what lets a buyer judge whether the corpus fits.
Do you have NLP Data available now?
No. We do not hold inventory. Everything is produced against a specification. That means the timeline starts when the spec is agreed rather than immediately, and it also means the data matches your requirement instead of approximately matching something already built.
Can this be combined with other categories in one delivery?
Yes, and it is common. A single specification can cover several categories across several languages, with one pilot batch and one quality bar. It usually reduces total cost because contributor recruitment and project setup are shared.
What does the consent documentation cover?
Signed contributor consent stating the intended use, the collection methodology, and the annotation guideline the data was produced under. For projects involving EU data subjects the chain is built to satisfy GDPR, including the transfer mechanism. The terms for moving the data across a border.
Request NLP Data
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.