What is Dictionary Data?
Structured dictionary entries with definitions, parts of speech, pronunciation, and example sentences, used for dictionary products and lexical understanding tasks.
Dictionary data is structured lexical entries: headword, part of speech, definition, pronunciation and examples. It is structured rather than free text, and the structure is the product.
The distinguishing problem is rights. Dictionary content is usually owned by a publisher, and a large share of existing dictionaries cannot be licensed for commercial use as they stand. The rights chain has to be established before the content is useful.
The second problem is format. Lexical data has no single standard interchange format in practice, so the delivery has to be shaped to the buyer's system, and the mapping decisions have to be documented because that is where information is lost.
What buyers get wrong
Rights in dictionary data are tangled, and many existing dictionaries cannot be used commercially as they are. The structured format also has to align with the buyer's system.
The specification detail that decides everything
Output format (JSON / XML / CSV) and whether pronunciation audio is included.
This is the line item that most often separates a dataset that works from one that gets re-ordered. It belongs in the specification before production starts, not in the delivery review.
How we source it
We do not hold inventory in this category. A requirement comes in, we match it to a producer who can meet the specification, and we agree the quality bar with both sides before recording begins. A pilot batch goes first so misalignments surface early.
Scenarios this covers
Buyers rarely ask for a category by name. They describe a situation, and the situation is what tells us whether this category is the right one. If yours is in the list, name it in the brief and it goes straight into the specification. If it is not, describe it anyway, because it usually means the requirement sits in a neighbouring category.
What good Dictionary Data looks like
Dictionary data is judged on its rights chain and on the completeness of its structure.
-
Rights established per source
Which dictionary or corpus the entries derive from, and the terms under which they can be used commercially, documented rather than asserted.
-
Entry structure defined and complete
Headword, part of speech, sense division, definition and examples, with the sense granularity stated, because sense splitting is where dictionaries differ most.
-
Pronunciation present or explicitly absent
Phonemic transcription, and audio where the specification needs it, because a lexical dataset without pronunciation serves a different set of uses.
-
Inflection and morphology included where relevant
Base forms alone are insufficient for morphologically rich languages.
-
Output format documented with a mapping
The target schema and the mapping from the source structure to it, because lexical schemas differ and the mapping is where information is lost.
Public datasets in this category
These are openly documented resources, not products we sell. Each one covers part of the category, and knowing where each stops is usually what decides whether a project needs a collection of its own.
-
Wiktionary dumps
Machine-readable dictionary content in well over a hundred languages, including definitions, translations, and inflection tables. The most broadly useful open lexical resource and its structure varies between editions.
-
WordNet
A lexical database of English organising words into synonym sets with relations between them. Precise and structured, and it covers English only, with smaller projects for a few other languages.
-
BabelNet
A multilingual encyclopedic dictionary combining WordNet with Wikipedia and other sources. Broad language coverage and a mix of licensed and open components, so the terms vary by language.
-
PanLex
A large collection of translation pairs across many languages and variants, assembled from existing dictionaries. The emphasis on endangered and minority languages is what makes it distinctive.
Questions to ask a supplier
Questions that establish the rights chain before any technical evaluation.
- Which dictionary or lexical source do the entries derive from, and what terms cover commercial use?
- How are senses divided, and is the granularity documented?
- Is pronunciation included: transcription, audio, or neither?
- Are inflected forms included, or only base forms?
- What is the output schema, and is a mapping document delivered?
The vague answer in this category is about the source dictionary. A supplier who says the corpus contains definitions without naming the dictionary it derives from has described a corpus that cannot be checked. Dictionaries disagree with each other, and without the source there is no tie-break rule for conflicting entries.
A verifiable answer names the source dictionary and its edition, the sense granularity, the treatment of inflected forms, and the coverage against a stated word list. Ask how many entries are headwords rather than inflected forms, because that ratio determines whether the corpus supports morphological work at all.
Technical attributes
The items below are the ones a specification can hold a producer to, and the ones that decide whether a delivery is usable for training rather than only present.
| Rights source | Named per source, with terms documented |
|---|---|
| Entry structure | Defined, with sense granularity stated |
| Pronunciation | Transcription, audio, or none |
| Inflection | Included or not, stated |
| Output schema | Named, with a mapping document |
| Delivery format | Structured records with a schema document and a mapping |
The sense granularity column decides what the corpus can support, and it is the one most often left as a count of definitions. Merged senses cannot support word sense disambiguation, and split senses multiply the entry count without adding information. The granularity has to be stated against a named dictionary.
The values come from the application. Machine translation, search and language learning need different sense granularities and different amounts of morphological detail. The specification should name the application, because the same language pair needs different lexical resources for each.
A mistake here is discovered as an unexplainable model behaviour. A corpus with merged senses trains a model that cannot distinguish meanings, and the failure appears as inconsistent output rather than as missing data. Rebuilding the sense layer means re-deriving it from the source, which may no longer be licensed.
How Dictionary Data is collected
Lexical data work is rights clearance followed by structuring.
- Rights clearance per source. Each source is identified and its terms reviewed before any structuring work begins, because the structuring is wasted if the rights do not permit commercial use.
- Schema design or mapping. Either a target schema is designed for the specification, or an existing schema is mapped to, and the mapping decisions are documented.
- Entry structuring. Headwords, parts of speech, senses and definitions are parsed into the schema, with sense division applied consistently.
- Pronunciation and inflection attachment. Transcription and audio are attached where required, and inflected forms are generated or sourced and linked to their base forms.
- Validation and delivery. The structured output is validated against the schema, and the schema document and mapping are delivered with the data.
After assembly, the entries are normalised, inflected forms are linked to their headwords, senses are numbered against the source, and the files are packaged with the source attribution per entry. The headword link is what makes the corpus usable for anything beyond lookup.
The pilot for a lexical corpus is a slice of the source dictionary normalised and linked, run before the full extraction, because the property being tested is whether the sense granularity of the source matches what the specification asks for. A slice shows the headword-to-inflected-form ratio and the sense count per entry, and those two figures decide whether the corpus supports the work it was bought for.
Acceptance is checked on the sense numbering against the source and on the source attribution per entry, both verified before the batch is accepted. The specification should name the source dictionary and its edition before extraction starts. Where the pilot shows the granularity is wrong, the source is changed rather than the senses merged or split afterwards, since re-deriving them requires the original licence.
How a delivery goes wrong
The defects below are the ones that recur across this category. Each is decided at collection or annotation time, which is why none of them can be repaired in delivery review.
-
Senses were merged
A word with several meanings delivered as one entry cannot support sense-aware tasks. Sense granularity has to be decided against a stated dictionary and applied consistently.
-
Inflection and lemma were not separated
Delivering surface forms without their lemmas breaks morphological analysis and inflates the entry count. The distinction is standard in lexicography and often lost in conversion.
-
Source dictionary was not named
Dictionaries disagree with each other. A corpus that does not say which dictionary it derives from cannot be checked, and conflicting entries have no tie-break rule.
Why this matters to buyers
Lexical data underpins a range of products: dictionaries, language learning, spell and grammar checking, and lexical grounding for language models. Each needs a different subset of the structure, which is why the schema and the completeness of the entries determine what the corpus is worth.
The second driver is that rights are the binding constraint. Much of the best lexical content is publisher-owned and unavailable commercially, which is why a corpus with a documented chain is a different product from one with the same content and no chain.
A lexical resource with merged senses produces a model that confuses meanings in ways that look like reasoning failures. The output is fluent and wrong, which makes it harder to catch than an obvious error, and it erodes trust in the product faster than a lower-quality but more predictable model would.
The second cost is licensing. Dictionaries are licensed works, and a corpus that does not record its source cannot demonstrate that it was used within terms. Buyers in publishing and education audit this before anything else, and an unclear answer removes the corpus from consideration entirely.
A lexical corpus with senses numbered against a named source gives a model that distinguishes meanings consistently, and the headword links mean morphological work is possible without a separate derivation. The team also knows which dictionary it is arguing with when an entry is disputed.
The second gain is the licence record per entry. Dictionaries are licensed works, and a delivery that attributes each entry to its source and edition can be extended with a second dictionary without the two being confused. That record is what buyers in publishing and education audit first.
Specification at a glance
| Category | Dictionary Data |
|---|---|
| Group | Non-speech data |
| Critical spec | Output format (JSON / XML / CSV) and whether pronunciation audio is included. |
| Included by default | Pilot batch, metadata schema, source and consent documentation |
| Pricing | Quoted from specification. No published rates |
Related categories
-
LLM Training Data
Text corpora for LLM pretraining and fine-tuning: instruction data, preference data, and multi-turn dialogue data.
-
Computer Vision Data
Annotated image and video data, with annotation types including classification, bounding boxes, segmentation masks, and keypoints.
-
NLP Data
Annotated text for NLP tasks: named entities, dependency syntax, sentiment polarity, and relation extraction.
-
Multilingual Language Data
Parallel or non-parallel text corpora spanning multiple languages, used for multilingual models and low-resource language research.
-
Translation Data
Parallel corpora pairing source and target language, used for machine translation training and evaluation.
-
Text Data
General-purpose text corpora from web pages, forums, reviews and similar sources, used for language model pretraining and text classification.
Questions about Dictionary Data
Can you use an existing published dictionary?
Only where the terms permit commercial use, and the chain has to be documented. Many published dictionaries cannot be licensed for this purpose as they stand, and that is established before the work begins rather than after.
How fine should sense division be?
Per the specification, and it has to be consistent. Sense granularity is where dictionaries differ most, and a corpus that splits senses at one level in one section and another level elsewhere cannot be used for sense disambiguation.
Is pronunciation audio necessary?
Only if the specification needs it. Phonemic transcription is sufficient for many lexical uses, and audio changes both the cost and the rights picture, since the recordings are a separate work.
What output format do you deliver?
The format the specification names, with a mapping document describing how the source structure was transformed. The mapping is what lets a buyer see where information was dropped.
Do you have Dictionary Data available now?
No. We do not hold inventory. Everything is produced against a specification. That means the timeline starts when the spec is agreed rather than immediately, and it also means the data matches your requirement instead of approximately matching something already built.
Can this be combined with other categories in one delivery?
Yes, and it is common. A single specification can cover several categories across several languages, with one pilot batch and one quality bar. It usually reduces total cost because contributor recruitment and project setup are shared.
What does the consent documentation cover?
Signed contributor consent stating the intended use, the collection methodology, and the annotation guideline the data was produced under. For projects involving EU data subjects the chain is built to satisfy GDPR, including the transfer mechanism. The terms for moving the data across a border.
Request Dictionary Data
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.