What is Vision-Language-Action Model?

The core idea is that actions become another language. RT-2, built on a 55-billion-parameter vision-language backbone, tokenizes each action as text and trains on robot episodes and web data together, which is where its ability to follow instructions that never appeared in robot data comes from. OpenVLA is the open version: 7 billion parameters, trained on about 970,000 episodes from Open X-Embodiment, fine-tunable with LoRA by updating roughly 1.4% of its parameters.

For data buyers the unit of value changes. A VLA does not need more hours of one task; it needs episodes that pair an instruction with the right behavior across many objects, scenes and robot bodies. The reason Open X-Embodiment pools 22 embodiments rather than scaling one. Failure episodes matter more here than anywhere else, because the model has to learn what an instruction does not mean.

There is also a rate mismatch to plan around: large VLAs run closed-loop control at only a few hertz, so demonstrations recorded at 50 Hz are downsampled or chunked before training, and the chunking convention has to match the deployment. Teams training a VLA usually specify data at the episode level (embodiment mix, instruction style, failure coverage), and collection or conversion is built to that specification rather than sold by the hour.

What a VLA model is built from

Three parts make up the model: perception, language and action. Camera frames enter through a vision encoder, the instruction enters as text, and an action decoder turns the two into motor commands. Because the output is an action, the training data that matters is paired: frames, the instruction and the exact sequence of commands the robot executed, all aligned on one clock.

Training usually mixes two sources. Robot episodes teach the link from observation and instruction to action. Broad image-text data, far larger and far cheaper, teaches the general knowledge that lets the model respond to an instruction it never saw demonstrated on a robot. The ratio between the two is a design choice with a visible consequence: more web data buys breadth, more robot data buys precision on the platform.

Within the robot portion, the axis that matters most is embodiment breadth. A model trained on one arm learns that arm; a model trained across many platforms is the one that can be handed to a new robot with limited fine-tuning. Because camera layouts, action spaces and control rates differ between platforms, converting them into one shared format is part of building the dataset, not an afterthought.

What to specify when buying

Fix the instruction format first. Imperative or descriptive, short or full sentences, one language or several, one phrasing per episode or several independent ones. Instruction following is the whole point of the model, so inconsistency here is a defect in the data rather than a matter of style. A multilingual target needs the language mix stated, not assumed.

Fix the action representation and the rate. Joint positions, end-effector poses and delta actions are not interchangeable, and the recorded representation has to match the robot the policy will run on. Say what rate the collection rig records at and what rate the policy will control at, because if the two differ, the downsampling or chunking convention is a decision for the brief, not a patch applied later.

Fix the breadth and the evaluation split. Name how many embodiments, scenes and task types are in scope, and hold out whole scenes or task types for evaluation; splitting randomly across episodes of the same task measures memorization, not generalization. Whether failure episodes are included should be a written requirement, because what an operator happens to record is not a specification.