What is Coding Data?
Software engineering records for training code models and coding agents: issue-to-patch pairs, pull requests with review discussion, test suites with pass and fail outcomes, and repository states where the build and tests can be re-run.
Coding data is the record of software work: the state of a repository, the issue or failing test that defines a problem, the patch that resolves it, and the verification that the resolution holds. It trains code models, coding agents, and the review and testing behaviour around them.
The distinguishing property is executability. Code either compiles and passes or it does not, which means this category can carry machine-checkable ground truth more cheaply than almost any other. That is also its trap: the check must run on the extracted task, and patches copied without their repository state often cannot.
The second property is context. A diff without the surrounding repository teaches what a change looks like, not how to find it. The issue text, the failing test, the file layout and the review discussion are all part of the training signal, and collections that strip them produce data that is easy to count and hard to learn from.
What buyers get wrong
The temptation is to collect patches without the surrounding context. A diff alone does not teach an agent to find the bug, and a patch that was never re-tested after extraction may not even apply. The repository state, the failing test, and the verified fix have to travel together.
The specification detail that decides everything
Whether each task carries the repository state before the change, the issue or failing test that defines done, and evidence that the patch passes after extraction. This is what separates executable training data from static text about code.
This is the line item that most often separates a dataset that works from one that gets re-ordered. It belongs in the specification before production starts, not in the delivery review.
How we source it
We do not hold inventory in this category. A requirement comes in, we match it to a producer who can meet the specification, and we agree the quality bar with both sides before recording begins. A pilot batch goes first so misalignments surface early.
Scenarios this covers
Buyers rarely ask for a category by name. They describe a situation, and the situation is what tells us whether this category is the right one. If yours is in the list, name it in the brief and it goes straight into the specification. If it is not, describe it anyway, because it usually means the requirement sits in a neighbouring category.
What good Coding Data looks like
Coding data is judged on whether each task can be re-executed and whether it carries the context that made the change meaningful.
-
Repository state before the change
The snapshot the task starts from, because a patch applied to the wrong base is not a task, and the state is what makes replay possible.
-
A checkable definition of done
A failing test, a reproduction or a specification that a verifier can run, because code ground truth is only ground truth when it executes.
-
Verification after extraction
Evidence that the patch still passes once it has been lifted out of its original repository, since extraction is where most of the silent breakage happens.
-
Review discussion retained
The comments that shaped the change, because review behaviour is half of what coding agents are asked to do and it exists nowhere else in the record.
-
License and provenance per repository
Source repositories carry licences that decide whether the derived data can be used commercially, recorded per repository rather than per collection.
Public datasets in this category
These are openly documented resources, not products we sell. Each one covers part of the category, and knowing where each stops is usually what decides whether a project needs a collection of its own.
-
The Stack
A large open collection of permissively licensed source code across many languages, with licence and provenance metadata per file. It is the base layer most open code corpora build on; what it does not contain is task structure, so it trains language modelling rather than problem solving.
-
SWE-bench
Real GitHub issues paired with the patches that resolved them, evaluated by running the repository tests before and after. It is the reference for executable issue resolution; its boundary is that it is an evaluation set, sized for measurement rather than training volume.
-
HumanEval and MBPP
Small function-level programming problems with test-based evaluation, widely used as the entry benchmark for code generation. They verify correctness mechanically at a small scale, which is exactly where their coverage stops.
-
CodeContests
Competitive programming problems with hidden tests and solutions, filtered for evaluation leakage. It provides hard, automatically verified problems; what it samples is contest style rather than the repository work most coding agents actually do.
Questions to ask a supplier
Questions that decide whether the tasks can be scored, or only read.
- Does every task carry the repository state it starts from?
- How is done defined, and can the verifier be re-run after extraction?
- Was each patch verified to pass after being lifted out of the original repository?
- Is the review discussion included, or only the final diff?
- What licence covers each source repository, recorded how?
The vague answer in this category is a patch collection described as verified. Ask how a task is reconstructed: the base state, the failing test, the re-run after extraction. A supplier who cannot describe the replay has not performed it, and the tasks will not score.
A verifiable answer can take one task and show the whole chain: repository state, failure reproduction, patch, passing run, licence field. One demonstrated chain is worth more than any description of scale, because the chain is the product.
Technical attributes
The items below are the ones a specification can hold a producer to, and the ones that decide whether a delivery is usable for training rather than only present.
| Task base state | Repository snapshot recorded per task |
|---|---|
| Done condition | Failing test, reproduction or specification, runnable |
| Post-extraction verification | Patch re-run after extraction, result recorded |
| Context retained | Issue text and review discussion included |
| Licence record | Per-repository licence and provenance field |
| Delivery format | Structured tasks with base commit, patch and verifier |
The base state column is what makes everything else possible: reproduction, verification and scoring all read from it. Collections that drop it save storage and lose the category, and no later processing can restore a state that was never captured.
The verification column decides what the success labels mean. A task verified in its home repository is a claim; a task verified after extraction is a measurement, and the difference is one re-run that most bulk pipelines skip.
A mistake here is measured as an agent that writes patches that do not pass. The model has learned the statistics of diffs rather than the loop of reproduce, fix and verify, and no prompt engineering restores a loop that is absent from the training data.
How Coding Data is collected
Coding data is assembled from repositories and reviewed changes, and every step is built around keeping the tasks executable.
- Source and licence review. Repositories are selected with their licences checked per source, because a collection is only as usable as its most restricted repository.
- Task extraction. Issues, failing tests and their resolving changes are paired, and the pair is kept only where the problem statement and the resolution refer to the same base state.
- Replay verification. Each task is reconstructed in a clean environment, the failure is reproduced, and the patch is applied and re-tested, because a task that cannot be reproduced cannot be scored.
- Context capture. The issue discussion, review comments and relevant file structure are captured alongside the change, since the change alone loses the reasoning that produced it.
- Labelling and packaging. Tasks are labelled by language, task type and verification status, and packaged with the base commit, the verifier and the licence record.
After assembly, each task is reconstructed from its base commit, the failure is reproduced, the patch is applied and the suite re-run, and the result is recorded per task. The reproduction pass is what converts a collection of changes into a set of solvable tasks.
The pilot for coding data is a small task set reconstructed and verified end to end: base state captured, failure reproduced, patch applied, suite re-run, licence recorded. Its purpose is to measure the reproduction rate and the post-extraction pass rate, because those two numbers decide what the collection is actually worth.
Acceptance is checked by re-running a sample of tasks independently and by inspecting the licence fields against the source repositories. The specification should fix the language and task-type mix before extraction, because those choices determine which of the public benchmarks the set can be compared against.
How a delivery goes wrong
The defects below are the ones that recur across this category. Each is decided at collection or annotation time, which is why none of them can be repaired in delivery review.
-
Patches without their base state
A diff stripped of its repository cannot be replayed, so neither the failure nor the fix can be reproduced. The defect is created at extraction and is the most common reason bulk code data cannot be scored.
-
Verification skipped after extraction
A change that passed in its home repository can fail in isolation. Collections that skip the re-run deliver tasks whose success labels are wrong, and wrong labels teach the model to prefer failing patches.
-
Licence recorded at collection level
A corpus described as open source as a whole hides the repositories whose terms are stricter, and those are the ones that decide commercial usability. The defect surfaces during the buyer's legal review, after the collection is built.
Why this matters to buyers
Coding is the domain where machine-checkable supervision is cheapest, and that advantage only exists if the tasks are kept executable. A collection of diffs trains autocomplete; a collection of reproducible tasks trains an agent that can work, and the difference is created entirely by the collection method.
The second driver is that coding agents are judged by their failures: an agent that produces a plausible patch that does not pass has learned the appearance of the work. Verified tasks are what turn patch generation from generation into problem solving.
The cost of unverified coding data is an agent that fails at the last mile: patches that look right, reviews that miss, and a success rate that is unknowable until deployment. Verified tasks are what make the difference measurable before the agent meets a real repository.
The second cost is context loss. An agent trained on diffs alone cannot find the problem, because finding is a behaviour trained by issue text, failing tests and repository layout. Teams that buy diffs discover this when their agent can complete but not diagnose.
Reproducible tasks with verified outcomes give a coding programme training data whose labels are real and evaluation that matches deployment. The same set serves training and benchmarking, and failures in the model can be traced to task types rather than guessed.
The second gain is diagnosability. With base states, issues and reviews in the record, a model's mistakes can be attributed to finding, fixing or verifying, and the next collection can be aimed at the weak part. That attribution is what turns a data budget into an improvement loop.
Specification at a glance
| Category | Coding Data |
|---|---|
| Group | Non-speech data |
| Critical spec | Whether each task carries the repository state before the change, the issue or failing test that defines done, and evidence that the patch passes after extraction. This is what separates executable training data from static text about code. |
| Included by default | Pilot batch, metadata schema, source and consent documentation |
| Pricing | Quoted from specification. No published rates |
Related categories
-
LLM Training Data
Text corpora for LLM pretraining and fine-tuning: instruction data, preference data, and multi-turn dialogue data.
-
Computer Vision Data
Annotated image and video data, with annotation types including classification, bounding boxes, segmentation masks, and keypoints.
-
NLP Data
Annotated text for NLP tasks: named entities, dependency syntax, sentiment polarity, and relation extraction.
-
Multilingual Language Data
Parallel or non-parallel text corpora spanning multiple languages, used for multilingual models and low-resource language research.
-
Translation Data
Parallel corpora pairing source and target language, used for machine translation training and evaluation.
-
Dictionary Data
Structured dictionary entries with definitions, parts of speech, pronunciation, and example sentences, used for dictionary products and lexical understanding tasks.
Questions about Coding Data
Why does the repository state matter if the patch is included?
Because a patch is only meaningful against the tree it was made from. Without the base state the task cannot be reconstructed, the failure cannot be reproduced, and the patch cannot be verified, which reduces the data to text about code.
What does post-extraction verification catch?
The cases where a change worked in its home repository because of something else that was also present, and fails when lifted out. That category of breakage is invisible in the source repository and common in bulk extraction.
Is review discussion really necessary?
It is if the buyer wants review behaviour from the same collection. The discussion is where reviewers explain what is wrong and why, and it is the only part of the record that shows the judgment around a change rather than the change itself.
How do licences work across many repositories?
Per repository, recorded as a field on every task. Collections described as open source as a whole hide the minority of sources whose terms are stricter, and that minority decides what the collection can be used for.
Do you have Coding Data available now?
No. We do not hold inventory. Everything is produced against a specification. That means the timeline starts when the spec is agreed rather than immediately, and it also means the data matches your requirement instead of approximately matching something already built.
Can this be combined with other categories in one delivery?
Yes, and it is common. A single specification can cover several categories across several languages, with one pilot batch and one quality bar. It usually reduces total cost because contributor recruitment and project setup are shared.
What does the consent documentation cover?
Signed contributor consent stating the intended use, the collection methodology, and the annotation guideline the data was produced under. For projects involving EU data subjects the chain is built to satisfy GDPR, including the transfer mechanism. The terms for moving the data across a border.
Request Coding Data
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.