Training with HDF5 Datasets

HDF5 datasets exported by the platform are read by training frameworks outside the platform. Before connecting to training, complete data preparation, field reading, and structure verification.

Roles and Prerequisites

Roles and Concerns

Role	Concern
Algorithm engineer	Field shapes, frame alignment, training framework integration
Training operations	Dataset mounting, GPU memory and disk usage

Prerequisites

Item	Requirement
Dataset	HDF5 export completed and archive downloaded; for the structure, see HDF5 datasets
Files	After extraction, chunk_*.hdf5 files are present
Fields	Each episode contains at least action and observation.state
Runtime	A Python environment with h5py, numpy, and the target framework (one of PyTorch, TensorFlow, JAX)
Image decoding	JPEG decoding available (Pillow, OpenCV, or torchvision)
Resources	The training machine has enough disk for the extracted chunk files and checkpoints

Procedure

1. Extract the archive and confirm that all chunk files are in the same directory.
2. Open one chunk file, read the episode list under data and the shape of each dataset, and check that they match the field and shape tables in HDF5 datasets.

import h5py

with h5py.File("chunk_001.hdf5", "r") as f:
    for episode_name in f["data"]:
        episode = f[f"data/{episode_name}"]
        print(episode_name, episode.attrs["task"].decode())
        for key in episode:
            print(" ", key, episode[key].shape, episode[key].dtype)

3. Build an index of (chunk file path, episode name) to avoid rescanning files every epoch.
4. Split the training and validation sets by episode, keeping the same chunk file out of both sets to avoid leakage of samples from the same source.
5. On read, decode observation.images.* frame by frame as JPEG and align them with action, observation.state, and observation.gripper by frame index.
6. Wrap the episode as a dataset object of the target framework and output image tensors plus state and action vectors in batches.
7. Read the task, task_zh, and score attributes to filter samples by task or quality.

Minimal loader example:

import io
import h5py
import numpy as np
import torch
from PIL import Image
from torch.utils.data import Dataset

class Hdf5EpisodeDataset(Dataset):
    def __init__(self, files, transform=None):
        self.index = []
        self.transform = transform
        for path in files:
            with h5py.File(path, "r") as f:
                self.index += [(path, name) for name in f["data"]]

    def __len__(self):
        return len(self.index)

    def __getitem__(self, i):
        path, name = self.index[i]
        with h5py.File(path, "r") as f:
            ep = f[f"data/{name}"]
            images = [Image.open(io.BytesIO(frame.tobytes()))
                      for frame in ep["observation.images.camera_01"][:]]
            if self.transform:
                images = [self.transform(img) for img in images]
            return {
                "images": torch.stack(images),
                "state": torch.as_tensor(np.asarray(ep["observation.state"][:]), dtype=torch.float32),
                "action": torch.as_tensor(np.asarray(ep["action"][:]), dtype=torch.float32),
                "task": ep.attrs["task"].decode(),
            }

Verification

Check	Method	Pass criterion
File integrity	List the chunk files in the extracted directory	Names are continuous with no gaps
Field completeness	Iterate over each episode under data	Contains action and observation.state, and at least one observation.images.*
Consistent shapes	Read the shape of each dataset	The first dimension of each dataset equals the frame count of that episode
Decodable images	Sample elements of observation.images.*	JPEG decoding succeeds
Frame alignment	Compare the row counts of action and observation.state	The row counts are equal

Error Handling

Symptom	Possible cause	Action	Owner
File cannot be opened	Download interrupted or extraction incomplete	Download again and check the file size	User
observation.gripper missing	The export source has no gripper topic	Ignore the field during training, or drop it in the action dimension configuration	Algorithm engineer
Image decoding fails	The frame is a raw array rather than JPEG	Decode it as a uint8 array, or drop the frame	Algorithm engineer
Datasets have different frame counts	The sidecar JSON subtask intervals do not align with message times	Resample using the action timestamps as the reference	Algorithm engineer
Insufficient GPU memory	The batch loads an excessive number of images	Lower the batch size or reduce the image resolution	Algorithm engineer
Training metrics do not converge	The frame rate sampling is too low, or state and action are misaligned	Raise the export hz and check the topic mapping and frame alignment	Algorithm engineer

Related Pages

• HDF5 datasets
• Data export
• Model training