What is Agentic Trajectories?
Complete operation records of software agents carrying out multi-step tasks: the screen or system state before each action, the tool call made, the result returned, and the final state that shows whether the task completed, used to train and evaluate computer-use and tool-calling models.
Agentic trajectories are the complete records of software agents doing multi-step work: the state of the screen or system before each action, the tool call or click the agent made, what came back, and the final state that shows whether the task was completed. They are the training material behind computer-use agents and tool-calling models, and the evaluation material that shows where such models fail.
The distinguishing property is that a trajectory is an execution record rather than a text. What makes it usable is the step-level chain: observation, action, result, repeated until the task ends. A transcript of the conversation around a task is not a trajectory, and much of what is collected cheaply is conversation rather than execution.
The second property is verifiability. Agent work either reached the goal state or it did not, and that outcome can be checked by re-running the environment rather than by reading the answer. This is what makes agentic data valuable for reinforcement learning, and it is also why sloppy collection cannot be repaired later: a trajectory that lost its intermediate states has lost exactly the part the training consumes.
What buyers get wrong
The hard part is verifiability. A trajectory is only training-grade if every step records what the agent saw, what it called, and what came back, and if the final state can be checked without a human reading the transcript. Trajectories captured without step-level state cannot be repaired afterwards, and most cheap sources of agent logs fail exactly here.
The specification detail that decides everything
Whether each step carries the observation, the action taken, and a machine-checkable completion signal at the end. This decides whether the data supports reinforcement learning and evaluation, or only supervised imitation of the action stream.
This is the line item that most often separates a dataset that works from one that gets re-ordered. It belongs in the specification before production starts, not in the delivery review.
How we source it
We do not hold inventory in this category. A requirement comes in, we match it to a producer who can meet the specification, and we agree the quality bar with both sides before recording begins. A pilot batch goes first so misalignments surface early.
Scenarios this covers
Buyers rarely ask for a category by name. They describe a situation, and the situation is what tells us whether this category is the right one. If yours is in the list, name it in the brief and it goes straight into the specification. If it is not, describe it anyway, because it usually means the requirement sits in a neighbouring category.
What good Agentic Trajectories looks like
An agentic trajectory is judged on whether the execution can be replayed and the outcome checked, not on how fluent the recorded reasoning sounds.
-
Step-level observations
The screen state or tool response before every action, captured in a form that can be replayed. A trajectory that records only the actions is a log, not training data.
-
Machine-checkable outcome
A final state or test that determines success without a human reading the transcript, because outcome verification is what the reward signal is built on.
-
Failed and recovered runs included
Trajectories that hit errors and recovered are the ones that teach recovery behaviour. A set of only clean runs teaches a model to assume clean runs.
-
Environment specification recorded
The application versions, sites or simulated environment the run took place in, because a trajectory without its environment cannot be replayed or validated.
-
Privacy review of captured state
Screen captures and tool responses can contain personal data. Trajectories collected without a scrubbing step cannot be used commercially, and the review has to happen at capture time.
Public datasets in this category
These are openly documented resources, not products we sell. Each one covers part of the category, and knowing where each stops is usually what decides whether a project needs a collection of its own.
-
OSWorld
A benchmark of real computer tasks across operating-system applications, with execution-based evaluation: success is checked by inspecting the resulting system state. It covers general desktop work, and its task set is finite, which is the boundary: it measures agents rather than supplying training volume.
-
WebArena
Self-hosted web environments with realistic sites and scripted tasks, evaluated by functional checks on the resulting page state. It gives reproducible browser tasks; what it does not give is coverage of the long tail of real sites and workflows.
-
Mind2Web
A generalist agent dataset of tasks across many real websites with action sequences annotated by humans. It documents what the action looks like on the page; it stops short of execution feedback, so recovery behaviour is not part of it.
-
GAIA
A benchmark of assistant-style questions that require multi-step tool use, web browsing and reasoning, with verifiable answers. It is designed to measure general assistants rather than to train a specific workflow, and the answer check is the part that transfers to collection design.
Questions to ask a supplier
Questions that decide whether trajectories can support training and evaluation, or only demonstration.
- Does every step carry the observation as well as the action?
- How is task success determined, and can it be checked without reading the transcript?
- Are failed runs and recovery attempts included, or only successful ones?
- What environment did the runs take place in, and is it recorded per trajectory?
- What privacy review is applied to captured screens and tool responses?
The vague answer in this category is a promise of clean runs. A supplier who cannot say how each step was captured, or how success was determined, is describing a demonstration rather than a dataset. Ask for one replayed trajectory and the environment record that came with it.
A verifiable answer names the capture stack, the action schema, the outcome check and the privacy criteria, and can show a failed run alongside a successful one. The failed run is the tell: collections that kept failures can show them, and collections that did not cannot reconstruct them.
Technical attributes
The items below are the ones a specification can hold a producer to, and the ones that decide whether a delivery is usable for training rather than only present.
| Step structure | Observation, action and result recorded per step |
|---|---|
| Outcome signal | Machine-checkable completion state or test result |
| Failure coverage | Failed and recovered runs kept as their own segment |
| Environment record | Application or environment version logged per trajectory |
| Privacy review | Screen and response content scrubbed, criteria recorded |
| Delivery format | Structured step records with action schema and outcome field |
The step structure column decides what the data can train. Actions without observations support behaviour cloning on the action stream alone; observations with outcomes support reinforcement learning and evaluation. The gap between the two is created by the capture design, not by post-processing.
The outcome column is where most projects under-specify. A checkable outcome is a property of the task definition, agreed before recording, and a task written without one cannot be fixed by annotating harder afterwards.
A mistake in this category is measured as an agent that passes review and fails deployment. The review reads transcripts, and transcripts are fluent; the deployment runs environments, and environments are not. Trajectories with replayed outcomes are what close that gap.
How Agentic Trajectories is collected
Agentic data is recorded rather than annotated: the work is building environments where the recording is reliable and the outcome is checkable.
- Environment setup. The task environment is defined and made reproducible, whether that is a simulated application, a sandboxed browser or a controlled desktop image, because an environment that drifts between sessions makes the trajectories incomparable.
- Task definition with checkable outcomes. Each task is written with a completion condition that can be evaluated automatically, which is what turns a recording into a reward signal.
- Execution capture. Human operators or instrumented agents perform the tasks while the recorder captures screen state, actions and results at every step, including the steps that fail.
- Privacy and content review. Captured states are reviewed and scrubbed against stated criteria before storage, because retroactive scrubbing of screen content is not possible once a run is flattened into clips.
- Verification and packaging. Each trajectory is replayed or its outcome re-checked, failed runs are labelled as such rather than discarded, and the set is packaged with the action schema and the environment record.
After capture, each trajectory is replayed against its environment where possible, outcomes are re-evaluated, failed runs are labelled and kept, and the set is packaged with the action schema, the environment record and the privacy review criteria. The replay step is what makes the success labels trustworthy.
The pilot for agentic data is a small task set run end to end: environments defined, outcomes made checkable, trajectories captured with full step state, and a privacy review applied. Its purpose is to test the capture stack and the outcome checks together, because a check that works on five tasks and fails on the fiftieth is a task-definition problem, not a capture problem.
Acceptance is checked by replaying pilot trajectories in their environments and re-running the outcome checks, with the failed runs included in the sample. The specification should fix the action schema and the privacy criteria before recording, because both are expensive to change once a batch exists.
How a delivery goes wrong
The defects below are the ones that recur across this category. Each is decided at collection or annotation time, which is why none of them can be repaired in delivery review.
-
Actions recorded without observations
A log of clicks and calls without the state they were made against cannot be replayed or used for training. The defect is created at capture time and is not repairable in review.
-
Only successful runs kept
Discarding failures removes exactly the data that teaches recovery, and it also removes the base rate: a set of clean runs makes an agent's real success rate unknowable.
-
Outcome left to human judgement
If completion is decided by reading the transcript, the reward signal is opinion. Tasks without a checkable outcome produce data that can be demonstrated but not evaluated.
Why this matters to buyers
Agent capability is measured by what happens over many steps, and that is not visible in any single response. A model that can name the right tool but cannot recover from a failed call fails at the task level, and the training data that teaches recovery is the trajectory, not the instruction pair.
The second driver is evaluation. An agent that scores well on question-answer benchmarks can still be unable to finish a ten-step task, and the only way to know is to run it against checkable tasks. Trajectories with machine-checkable outcomes are the evaluation material as much as the training material.
The cost of bad agentic data is discovered at deployment: an agent that was demonstrated to work stalls on the third step of a real workflow, and the debugging has to happen in production because the training set never contained a failed step. Recovery behaviour cannot be prompted into existence; it has to be in the data.
The second cost is evaluation credibility. An agent team that cannot measure completion rates on checkable tasks is guessing, and the guess is usually optimistic. Machine-checkable outcomes are what turn an internal demo into a number that decisions can rest on.
Trajectories with step-level state and checkable outcomes give an agent team training data that matches deployment and evaluation numbers that mean something. Failed runs become a labelled segment rather than an embarrassment, and the recovery behaviour they teach is what separates a demo from a product.
The second gain is transfer. A capture stack and action schema that are documented can be pointed at the buyer's own environment, and the next batch of trajectories is comparable with the first. Agent programmes that improve over time depend on that comparability.
Specification at a glance
| Category | Agentic Trajectories |
|---|---|
| Group | Non-speech data |
| Critical spec | Whether each step carries the observation, the action taken, and a machine-checkable completion signal at the end. This decides whether the data supports reinforcement learning and evaluation, or only supervised imitation of the action stream. |
| Included by default | Pilot batch, metadata schema, source and consent documentation |
| Pricing | Quoted from specification. No published rates |
Related categories
-
LLM Training Data
Text corpora for LLM pretraining and fine-tuning: instruction data, preference data, and multi-turn dialogue data.
-
Computer Vision Data
Annotated image and video data, with annotation types including classification, bounding boxes, segmentation masks, and keypoints.
-
NLP Data
Annotated text for NLP tasks: named entities, dependency syntax, sentiment polarity, and relation extraction.
-
Multilingual Language Data
Parallel or non-parallel text corpora spanning multiple languages, used for multilingual models and low-resource language research.
-
Translation Data
Parallel corpora pairing source and target language, used for machine translation training and evaluation.
-
Dictionary Data
Structured dictionary entries with definitions, parts of speech, pronunciation, and example sentences, used for dictionary products and lexical understanding tasks.
Questions about Agentic Trajectories
How is this different from tool-use conversation data?
A conversation records what was said; a trajectory records what was executed. The observation at each step, the exact call, and the result are the training signal, and they exist only if the collection captured the environment rather than the chat.
Why do failed trajectories matter?
Because recovery is a behaviour, and it is learned from runs that needed it. A set of clean runs teaches a model that environments cooperate, which is the opposite of what deployment looks like.
What does the privacy review cover?
Screen states and tool responses can contain personal data from accounts, messages or documents that appear during the task. The review scrubs those at capture time against stated criteria, and the criteria are delivered so a buyer can audit them.
Can you collect in our own environment?
Yes, and it is often the point: trajectories from the buyer's own application or workflow are the ones that match deployment. The environment and the outcome check are defined together with the buyer before recording starts.
Do you have Agentic Trajectories available now?
No. We do not hold inventory. Everything is produced against a specification. That means the timeline starts when the spec is agreed rather than immediately, and it also means the data matches your requirement instead of approximately matching something already built.
Can this be combined with other categories in one delivery?
Yes, and it is common. A single specification can cover several categories across several languages, with one pilot batch and one quality bar. It usually reduces total cost because contributor recruitment and project setup are shared.
What does the consent documentation cover?
Signed contributor consent stating the intended use, the collection methodology, and the annotation guideline the data was produced under. For projects involving EU data subjects the chain is built to satisfy GDPR, including the transfer mechanism. The terms for moving the data across a border.
Request Agentic Trajectories
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.