What is News Data?
News articles and reportage with date, source, and topic tags, used for event extraction and temporal analysis.
News data is articles with date, source and topic. Its two defining problems are rights and recency, and both are structural rather than technical.
Rights are murky because a large share of news content is aggregated or republished, and the chain from the original publisher to the current holder is often not documented. A corpus assembled from aggregators frequently cannot be licensed for commercial use.
Recency is a value curve. Older stories lose value for a model quickly, because events, actors and language change, and a corpus that spans a decade is not equivalent to one that spans a year.
What buyers get wrong
Copyright and recency are both problems with news data: aggregated and republished content has murky rights, and older stories lose value for a model fast.
The specification detail that decides everything
Time span, number of sources, and whether event annotation is included.
This is the line item that most often separates a dataset that works from one that gets re-ordered. It belongs in the specification before production starts, not in the delivery review.
How we source it
We do not hold inventory in this category. A requirement comes in, we match it to a producer who can meet the specification, and we agree the quality bar with both sides before recording begins. A pilot batch goes first so misalignments surface early.
Scenarios this covers
Buyers rarely ask for a category by name. They describe a situation, and the situation is what tells us whether this category is the right one. If yours is in the list, name it in the brief and it goes straight into the specification. If it is not, describe it anyway, because it usually means the requirement sits in a neighbouring category.
What good News Data looks like
News data is judged on the rights chain and on the time span, both of which are stated rather than implied.
-
Rights documented per source
Which publisher, and under what terms, with aggregators distinguished from originals, because the aggregator is not the rights holder.
-
Time span stated, with the distribution rather than the endpoints
A corpus from 2015 to 2025 with eighty percent in the last year is a different product from one evenly distributed.
-
Source diversity described
How many outlets, and whether they are independent, because a corpus from one wire service is one source repeated.
-
Duplicate and syndicated content identified
The same story appears in many outlets, and a corpus that counts each appearance inflates its own size.
-
Topic and event annotation where required
Which topics, and whether events were annotated, since event extraction needs event labels rather than topic tags.
Public datasets in this category
These are openly documented resources, not products we sell. Each one covers part of the category, and knowing where each stops is usually what decides whether a project needs a collection of its own.
-
GDELT
A continuously updated index of global news events with coded actors and event types, plus article references. The event coding is its contribution and the underlying article text is generally not included.
-
CC-News
A large collection of news articles extracted from Common Crawl, with a documented pipeline. Useful at scale and the extraction quality varies by publisher.
-
CNN/DailyMail
News articles with human-written summaries, widely used for summarisation. One genre and one era, and it has been used long enough that contamination is a real concern.
-
Newsroom
A summarisation corpus built from news articles with multiple reference summaries and editorial metadata. Better annotated than most, and small.
Questions to ask a supplier
Questions that establish the rights chain and whether syndicated copies were counted as distinct articles.
- Which publishers, and are aggregators distinguished from original sources?
- What is the time span, and what is the distribution across it?
- How many distinct outlets, and are they independent?
- How was syndicated or duplicate content identified?
- Is event annotation included, or only topic tags?
The vague answer in this category is about deduplication. A supplier who reports an article count without stating the deduplication method has described a number that may include the same wire story hundreds of times. Syndicated copies inflate the count and put identical content on both sides of a train and evaluation split.
A verifiable answer states the source outlets, the deduplication method and rate, the publication date handling, and the rights position for each outlet. Ask for the rights chain specifically, because a corpus assembled from aggregators has no path to the rights holder, and that is discovered at contract time.
Technical attributes
The items below are the ones a specification can hold a producer to, and the ones that decide whether a delivery is usable for training rather than only present.
| Rights source | Publisher named per source, aggregators distinguished |
|---|---|
| Time span | Stated, with the distribution reported |
| Source diversity | Outlet count, with independence noted |
| Deduplication | Syndicated content identified and reported |
| Annotation | Topic tags, event labels, or neither |
| Delivery format | Articles with per-article source, date and rights metadata |
The publication date column is the axis news analysis runs on, and it is the one most often missing or wrong. Where articles were extracted from crawling rather than from feeds, the date may be the crawl date rather than the publication date. The distinction decides whether temporal analysis is possible at all.
The values come from the analysis the corpus serves. Trend detection needs a long time series with accurate dates, event extraction needs coverage across outlets, and summarisation needs article pairs. The three need different compositions, so the task belongs in the specification.
A mistake here is measured as a duplicated corpus. Without deduplication, evaluation leaks into training and the reported quality is inflated. The effect is large in this category because syndication is normal rather than exceptional, and it stays invisible without a per-outlet breakdown.
How News Data is collected
The rights work comes first, because it determines which sources can be included at all.
- Source selection with rights review. Each publisher is identified and its terms reviewed before inclusion, and aggregators are separated from originals because they do not hold the rights.
- Acquisition within terms. Articles are acquired in the way each source permits, including rate limits and any restriction on redistribution.
- Syndication and duplicate detection. The same story across outlets is identified and either deduplicated or marked, so the corpus size reflects distinct content.
- Time and topic metadata. Date, outlet and topic are recorded per article, and the time distribution is measured rather than assumed.
- Event annotation where required. Events are annotated against a stated scheme where the specification needs them, with agreement measured on the event boundaries.
After acquisition, articles are deduplicated across outlets, dates are resolved to publication rather than crawl time, and the files are packaged with per-article metadata covering the outlet, the date and the rights position. The deduplication rate belongs in the manifest, because it changes what the article count means.
The pilot for news is a slice from the outlets the specification names, deduplicated and dated before the full acquisition. Its purpose is to measure the duplicate rate and the share of articles where the date is a crawl date rather than a publication date, because those two figures decide whether temporal analysis is possible at all. Both are measured on the sample.
Acceptance is checked on the deduplicated article count and on the per-outlet rights record, both against figures stated before acquisition. The specification should name the outlets and the date handling in advance. Where the pilot shows the duplicate rate is high, the source set is changed rather than the count reported as delivered, since syndicated copies put identical content on both sides of a split.
How a delivery goes wrong
The defects below are the ones that recur across this category. Each is decided at collection or annotation time, which is why none of them can be repaired in delivery review.
-
Syndicated copies counted as distinct articles
The same wire story appears across hundreds of outlets. Without deduplication the corpus size is inflated and the same content lands in both training and evaluation splits.
-
Publication date was not preserved
The date is the axis news analysis runs on. Missing or wrong dates make temporal analysis impossible, and the error is common where articles were extracted from crawling rather than from feeds.
-
Rights chain stops at the aggregator
A corpus assembled from aggregators has no path to the rights holder. This is a commercial blocker discovered at contract time, after the collection work is done.
Why this matters to buyers
Event extraction and temporal analysis need dated, sourced articles. The date and the source are the axes along which news is analysed, and a corpus without them cannot support the task regardless of how many articles it contains.
The second driver is that news is the category where rights are most often unclear. Aggregation and republication are standard practice, and a corpus built without tracing the chain is a corpus that cannot be sold commercially.
A news model trained on a corpus with syndicated duplicates reports quality it does not have. The evaluation sets contain material the model has already seen, so the scores are inflated and the model disappoints in production. The corpus cannot be repaired without re-running deduplication, which changes the training set.
The second cost is rights. News content is licensed by its publishers, and a corpus assembled from aggregators has no chain to them. That is a commercial blocker discovered at contract time, after the collection work is done, and the usual outcome is that the corpus gets withdrawn.
A news corpus with measured deduplication and resolved publication dates gives evaluation sets that do not leak into training. The reported quality predicts production performance, and trend analysis runs on a time series that means what it says.
The second gain is the rights chain per outlet. News is licensed by its publishers, and a delivery that records the position per outlet can be extended with new sources without re-clearing the existing ones. That record is what keeps a news programme from being withdrawn at contract time.
Specification at a glance
| Category | News Data |
|---|---|
| Group | Non-speech data |
| Critical spec | Time span, number of sources, and whether event annotation is included. |
| Included by default | Pilot batch, metadata schema, source and consent documentation |
| Pricing | Quoted from specification. No published rates |
Related categories
-
LLM Training Data
Text corpora for LLM pretraining and fine-tuning: instruction data, preference data, and multi-turn dialogue data.
-
Computer Vision Data
Annotated image and video data, with annotation types including classification, bounding boxes, segmentation masks, and keypoints.
-
NLP Data
Annotated text for NLP tasks: named entities, dependency syntax, sentiment polarity, and relation extraction.
-
Multilingual Language Data
Parallel or non-parallel text corpora spanning multiple languages, used for multilingual models and low-resource language research.
-
Translation Data
Parallel corpora pairing source and target language, used for machine translation training and evaluation.
-
Dictionary Data
Structured dictionary entries with definitions, parts of speech, pronunciation, and example sentences, used for dictionary products and lexical understanding tasks.
Questions about News Data
Why distinguish aggregators from original publishers?
Because the aggregator usually does not hold the rights. A corpus assembled from aggregators has a chain that cannot be traced to the rights holder, and that is a commercial blocker regardless of the content quality.
How does recency affect value?
Older content loses value quickly for most models, because events, actors and language change. What matters is the distribution rather than the endpoints, since a corpus spanning a decade may still be concentrated in the last year.
How is syndicated content handled?
It is identified and either deduplicated or marked. The same wire story appears in many outlets, and a corpus that counts each appearance reports a size that does not correspond to distinct content.
Are topic tags enough for event extraction?
No. A topic tag says what an article is about; an event label says what happened, when and to whom. Event extraction needs the second, and it is a separate annotation task.
Do you have News Data available now?
No. We do not hold inventory. Everything is produced against a specification. That means the timeline starts when the spec is agreed rather than immediately, and it also means the data matches your requirement instead of approximately matching something already built.
Can this be combined with other categories in one delivery?
Yes, and it is common. A single specification can cover several categories across several languages, with one pilot batch and one quality bar. It usually reduces total cost because contributor recruitment and project setup are shared.
What does the consent documentation cover?
Signed contributor consent stating the intended use, the collection methodology, and the annotation guideline the data was produced under. For projects involving EU data subjects the chain is built to satisfy GDPR, including the transfer mechanism. The terms for moving the data across a border.
Request News Data
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.