What is LeRobot?

LeRobot is an open-source project from Hugging Face that does for robot data roughly what a standard image format did for computer vision: it fixes a layout, so a dataset written by one lab can be read by another lab's training script without a conversion project. Low-dimensional state and action live in Parquet files, camera streams live as MP4, timestamps tie the two together, and datasets and trained checkpoints are hosted next to the code.

The value is in the tooling around the format. Recording, converting from other formats, training and evaluating are all covered, and the library ships reference implementations of several policy architectures that train on a LeRobot dataset without modification. For a team holding data in a homegrown format, conversion is usually the first piece of work, and doing it into a standard format rather than another bespoke one is what makes the data reusable by the next person who touches it.

For a buyer, the format question belongs at specification time, not at delivery. A dataset delivered in a documented, widely supported format is one you can hand to a new engineer without a briefing; a folder of HDF5 files with a README describing half the fields costs a week before anyone trains anything. If the target format is LeRobot, put it in the brief: retrofitting after collection is possible, but it is work that did not have to happen.

The parts of a LeRobot dataset

The format fixes a layout, not a content. What sits inside varies by robot: the number of cameras, the frame rate, the resolution, the dimensions of the state and action vectors, and the name of every field. Those choices are recorded in the dataset's metadata, and the metadata is what makes a dataset readable by an engineer who has never seen the robot that produced it.

The parts are deliberately plain. Low-dimensional signals (joint positions, gripper state, actions) are stored in Parquet files, which any data tool can read. Camera streams are stored as MP4, which any video tool can read. Timestamps tie the two together. Nothing in the layout needs a specialist library to inspect, which is a large part of why the format spread.

The library around the format is a separate thing from the format itself. It records, converts, trains and evaluates, and it ships reference policy implementations that consume a dataset in the format without modification. Data in another layout can be converted into it, and the format can also be read by tools that never touch the library.

What to specify when buying

The format belongs in the brief, at specification time. A dataset delivered in a documented, widely supported format can be handed to a new engineer without a briefing. A folder of files in a homegrown layout costs days before anyone trains anything, and that cost lands after the invoice. If the target is the LeRobot format, say so before collection starts.

Name the revision as well. The format has changed as the project has grown, and a dataset written for an older revision may need conversion before a current training script will read it. Delivery should include the metadata that describes every field, so the mapping from the raw sensors to the stored columns is not left to guesswork.

Where the data already exists in another format, conversion is usually the first piece of work, and it can be quoted on its own. The deliverable to ask for is not only the converted files but the check that a reference policy trains on them. A conversion that runs without error and produces a dataset the tools still refuse is not a finished job.