aloha_hd5 to LerobotDataset v2

[Raziel90]

What this does

Adds Script for the conversion of aloha_hd5 dataset format to LerobotDataset (v2)

• The scripts decouples the components dependant on the Original dataset format to those that create the dataset.
• The script is easy to generalise to other datasets as long as an equivalent of the AlohaHD5Extractor (check code) is provided

NB: Automated testing will be provided in a separate PR.

How it was tested

Launched the script to upload the dataset in multiple ways (e.g. images as images or as videos, different episode descriptions)

These datasets are uploaded with this
https://huggingface.co/datasets/ccop/aloha-hd5-conversion-test
https://huggingface.co/datasets/ccop/test-again

Examples:

• Added examples/port_datasets/aloha_hd5.py

How to checkout & try? (for the reviewer)

You can try on an hd5 dataset using the following. (change the arguments to fit your dataset).

python examples/port_datasets/aloha_hd5.py --fps=50 --raw-path=/home/ccop/code/aloha_data/aloha_stationary_pick_place_bottle/  --dataset-repo-id=ccop/test-again --video-encoding=false --push=True --description="Aloha HD5 dataset - Bottle Pick and Place"

[Michael-Equi]

Hi Raziel. Thanks for writing this! I think there is a bug in the implementation you are submitting that results in saving only one frame per episode. The fix should just be to tab in this line https://github.com/huggingface/lerobot/pull/586/files#diff-8a6e4cf809bbdbbde9c6d8d6887e9c3d1537f3dda6179c2bf819a04984bb9fd4R131

[Raziel90]

Hi @Michael-Equi do you mean line 135?

frames.append(frame)

I'm fixing that

[Raziel90]

Hi @Michael-Equi, I fixed it if you want to do a double check :)

[pprett]

does aloha use shape (c, h, w) when uncompressed? I remember it to be (h, w, c)

[Raziel90]

I will give a check, but if I have transposed there, there is a reason that I don't recall right now.
The result seems good here

[Takuzenn]

Hi @Raziel90 I tried but it is very slow, do you know why?

[Raziel90]

Hi @Takuzenn can you elaborate what you tried to run and why you think is slow?
In the script I am not doing any data heavy changes.

Have you tried to change set the number of workers? are you converting images to video or are you keeping images?