LeRobot v2 and v3 Format Differences
This page covers the format version of LeRobotDataset: the directory structure, metadata organization, and read path of a dataset. It does not cover the software release version of the lerobot codebase.
Definitions
| Category | Example | Meaning |
|---|---|---|
| LeRobot software version | v0.4.0, v0.5.0 | Release version of the Hugging Face lerobot codebase |
| LeRobotDataset format version | v2.0, v2.1, v3.0 | Version of the dataset directory structure, metadata organization, and read path |
LeRobotDataset v3.0 was introduced in lerobot v0.4.0. As of lerobot v0.5.0, codebase_version in meta/info.json is v3.0, the format mainline remains v3.0, and no new format generation has appeared.
Version Differences
Format Comparison
The core difference between the two formats is how episodes are organized in files: v2 writes each episode into its own set of files, while v3 merges episodes into shared files and restores the episode-level view from metadata.
| Dimension | v2.0 / v2.1 | v3.0 |
|---|---|---|
| Version marker | codebase_version is v2.0 or v2.1 | codebase_version is v3.0 |
| Tabular data path | data/chunk-XXX/episode_YYYYYY.parquet; the whole table is one episode | data/chunk-XXX/file-YYY.parquet; episodes are split by row range |
| Video path | videos/chunk-XXX/{feature_key}/episode_YYYYYY.mp4; one mp4 per episode | videos/{feature_key}/chunk-XXX/file-YYY.mp4; split by time range |
| Episode organization | One episode maps to a set of parquet and mp4 files | Episodes are merged into shared files; writes are split by chunks_size (default 1000) and file size, so the file count grows more slowly as episodes increase |
| Episode metadata | meta/episodes.jsonl, one episode per line | meta/episodes/chunk-XXX/file-YYY.parquet, in columnar storage |
| Task metadata | meta/tasks.jsonl | meta/tasks.parquet (official default) |
| Statistics metadata | Global meta/stats.json; per-episode stats stored separately in meta/episodes_stats.jsonl | Global meta/stats.json; per-episode stats written as stats/<feature>/... columns in meta/episodes/*.parquet |
| Path resolution | chunk_index is derived from episode_index and chunks_size (default 1000) and then applied to the path template | Determined by data/chunk_index, data/file_index, the row range, and the time range in the episode metadata |
| Path template | data_path and video_path are written to meta/info.json with the placeholders {episode_chunk}, {episode_index}, and {video_key} | data_path and video_path are written to meta/info.json with the placeholders {chunk_index}, {file_index}, and {video_key} |
| Episode locator fields | episode_index, length, tasks, task_index | episode_index, length, tasks, dataset_from_index, dataset_to_index, data/chunk_index, data/file_index, videos/<key>/from_timestamp, videos/<key>/to_timestamp |
| Video encoding metadata | The info of a video feature records video.codec, video.pix_fmt, video.fps, video.width, video.height, and video.channels | Same as v2, with the fields in the same positions |
| Large-scale scenarios | The file count grows linearly with episodes | The file count is small, which suits object storage and streaming reads from the Hub |
The official default video encoder is libsvtav1 (AV1). vcodec accepts h264, hevc, libsvtav1, or auto, and the video feature info in meta/info.json records the encoder actually used. When Embodiflow LeRobot Studio exports v3.0, it writes both meta/tasks.jsonl and meta/tasks.parquet.
Version Determination
codebase_version in meta/info.json is the only criterion for the format version. The loader uses this field to choose the v2 or v3 parser and reports VERSION_MISMATCH or VERSION_MISMATCH_V3 when the value does not match.
Path Resolution
Path resolution in v2 and v3:
In v3, dataset_from_index and dataset_to_index are global row indices after concatenation across datasets; when a single parquet file is read, they must be converted to local row indices within the file. Videos are read according to the time range in the episode metadata.
Verification Checklist
| Format version | Key files |
|---|---|
v2.1 | meta/info.json, meta/stats.json, meta/episodes.jsonl, meta/episodes_stats.jsonl, meta/tasks.jsonl, data/chunk-*/episode_*.parquet, videos/chunk-*/<key>/episode_*.mp4 |
v3.0 | meta/info.json, meta/stats.json, meta/episodes/chunk-*/file-*.parquet, data/chunk-*/file-*.parquet, videos/<key>/chunk-*/file-*.mp4, meta/tasks.parquet (or meta/tasks.jsonl) |
The episode count in meta/episodes.jsonl and meta/episodes/ should match total_episodes in meta/info.json; the health check raises a warning on a mismatch. The splits of v3 should contain the train key.
Basis and Sources
| Item | Value | Source |
|---|---|---|
| Supported format versions | v2.0, v2.1, v3.0 | lerobot/src/services/versioning/versionRegistry.ts |
| Training dataset versions supported by the platform | v2, v3 | train/app/services/training_capabilities.py |
v2 default data path template | data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet | lerobot v0.3.3 datasets/utils.py |
v2 default video path template | videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4 | lerobot v0.3.3 datasets/utils.py |
v3 default data path template | data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet | lerobot/src/services/versioning/v3FormatValidator.ts |
v3 default video path template | videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4 | lerobot/src/services/versioning/v3FormatValidator.ts |
v2 default chunks_size | 1000 | lerobot/src/services/versioning/v2Adapter.ts; lerobot v0.3.3 datasets/utils.py |
v3 required episode columns | episode_index, length, dataset_from_index, dataset_to_index | lerobot/src/services/versioning/v3FormatValidator.ts |
v3 default task path | meta/tasks.parquet | lerobot v0.5.0 datasets/utils.py |
v2 task path | meta/tasks.jsonl | lerobot v0.3.3 datasets/utils.py |
| Official default video encoder | libsvtav1 | lerobot v0.5.0 datasets/video_utils.py |
Migration and Compatibility
| Direction | Method | Constraint |
|---|---|---|
v2.1 → v3.0 | The official script lerobot.scripts.convert_dataset_v21_to_v30, which merges per-episode files into shared files and fills in the episode locator metadata | Applies to datasets already hosted on the Hugging Face Hub |
v2.1 and v3.0, both directions | Switch the target version when exporting in LeRobot Studio | Applies to local or private data; the dataset can be previewed before export |
| Training integration | The platform supports both training dataset versions v2 and v3 | For the dataset versions each model supports, see the corresponding model page |
Limitations
Format parsing and validation are bounded by the existing limits below.
| Item | Limit | Notes |
|---|---|---|
| Supported format versions | v2.0, v2.1, v3.0 | No parser can be selected when the codebase_version prefix is neither v2 nor v3 |
| Version criterion | Only codebase_version in meta/info.json | Validation reports VERSION_MISMATCH or VERSION_MISMATCH_V3 when it does not match the parser |
v3.0 required episode columns | episode_index, length, dataset_from_index, dataset_to_index | The health check warns when they are missing |
v3.0 splits | Must contain train | The health check warns when it is missing |
v3.0 path templates | Use the default data_path and video_path | The health check warns when a non-default template is used |
| feature dtype | Fixed allowlist | Validation fails for values outside the allowlist |
| feature shape | Array of positive integers | Validation fails for an empty array or one containing non-positive integers |
| feature names | Length equal to the last dimension of shape | A warning is raised on a mismatch |
| Subtask metadata | v3.0 only | v2.1 does not contain meta/subtasks.parquet |
v2.1 → v3.0 migration | The official script is limited to datasets hosted on the Hugging Face Hub | For local or private data, switch the target version at export time |
Related Pages
| Page | Purpose |
|---|---|
| LeRobot data visualization | In-browser preview and health check |
| LeRobot datasets and training | Export and training overview |
| Install LeRobot | Install the official training stack locally |