Code

[WIP] Fix SAC and port HIL SERL#644

Code

Open

AdilZouitine wants to merge 70 commits intouser/michel-aractingi/2024-11-27-port-hil-serlhuggingface/lerobot:user/michel-aractingi/2024-11-27-port-hil-serlfrom user/adil-zouitine/2025-1-7-port-hil-serl-newhuggingface/lerobot:user/adil-zouitine/2025-1-7-port-hil-serl-newCopy head branch name to clipboard

Commits

Commits on Feb 3, 2025

• [vizualizer] for LeRobodDataset V2 (#576)

  

  mishig25

  authored and

  michel-aractingi

  committed
  0a4e9e2

  Copy full SHA for 0a4e9e2

• Fix broken create_lerobot_dataset_card (#590)

  

  helper2424

  authored and

  michel-aractingi

  committed
  4a43c83

  Copy full SHA for 4a43c83

• Update README.md (#612)

  

  CharlesCNorton

  authored and

  michel-aractingi

  committed
  b1cfb6a

  Copy full SHA for b1cfb6a

• Fix Quality workflow (#622)

  

  aliberts

  authored and

  michel-aractingi

  committed
  31c34a4

  Copy full SHA for 31c34a4

• fix(docs): typos in benchmark readme.md (#614)

  Show description for d649815
  

  2 peopleauthored and

  michel-aractingi

  committed
  d649815

  Copy full SHA for d649815

• fix(visualise): use correct language description for each episode id (#604)

  Show description for a1b5d0f
  

  2 peopleauthored and

  michel-aractingi

  committed
  a1b5d0f

  Copy full SHA for a1b5d0f

• typo fix: batch_convert_dataset_v1_to_v2.py (#615)

  Show description for c2f7af3
  

  2 peopleauthored and

  michel-aractingi

  committed
  c2f7af3

  Copy full SHA for c2f7af3

• [viz] Fixes & updates to html visualizer (#617)

  

  mishig25

  authored and

  michel-aractingi

  committed
  100f54e

  Copy full SHA for 100f54e

• fixes to SO-100 readme (#600)

  Show description for df7310e
  

  3 peopleauthored and

  michel-aractingi

  committed
  df7310e

  Copy full SHA for df7310e

• Fix for the issue #638 (#639)

  

  PradeepKadubandi

  authored and

  michel-aractingi

  committed
  068efce

  Copy full SHA for 068efce

• [WIP] correct sac implementation

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  472a7f5

  Copy full SHA for 472a7f5

• remove breakpoint

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  c86dace

  Copy full SHA for c86dace

• SAC works

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  a0a50de

  Copy full SHA for a0a50de

• Add rlpd tricks

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  be96501

  Copy full SHA for be96501

• [WIP] correct sac implementation

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  956c547

  Copy full SHA for 956c547

• remove breakpoint

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  86df8a4

  Copy full SHA for 86df8a4

• SAC works

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  c1d4bf4

  Copy full SHA for c1d4bf4

• Add rlpd tricks

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  8105efb

  Copy full SHA for 8105efb

• Change SAC policy implementation with configuration and modeling classes

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  7d2970f

  Copy full SHA for 7d2970f

• Add type annotations and restructure SACConfig class fields

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  1fb03d4

  Copy full SHA for 1fb03d4

• Stable version of rlpd + drq

  

  AdilZouitine

  authored and

  michel-aractingi

  committed
  d75b44f

  Copy full SHA for d75b44f

• Added server directory in lerobot/scripts that contains scripts and the protobuf message types to split training into two processes, acting and learning. The actor rollouts the policy and collect…

  Show description for 322a78a
  michel-aractingi

  committed
  322a78a

  Copy full SHA for 322a78a

• FREEDOM, added back the optimization loop code in learner_server.py

  Show description for 36576c9
  

  michel-aractingi

  and

  AdilZouitine

  committed
  36576c9

  Copy full SHA for 36576c9

• - Added additional logging information in wandb around the timings of the policy loop and optimization loop.

  Show description for 42618f4
  

  michel-aractingi

  and

  AdilZouitine

  committed
  42618f4

  Copy full SHA for 42618f4

• Added missing config files env/maniskill_example.yaml and policy/sac_maniskill.yaml that are necessary to run the lerobot implementation of sac with the maniskill baselines.

  Show description for 9aabe21
  

  michel-aractingi

  and

  AdilZouitine

  committed
  9aabe21

  Copy full SHA for 9aabe21

• Removed unnecessary time.sleep in the streaming server on the learner side

  Show description for e856ffc
  

  michel-aractingi

  and

  AdilZouitine

  committed
  e856ffc

  Copy full SHA for e856ffc

• Added support for checkpointing the policy. We can save and load the policy state dict, optimizers state, optimization step and interaction step

  Show description for 367dfe5
  

  michel-aractingi

  and

  AdilZouitine

  committed
  367dfe5

  Copy full SHA for 367dfe5

• Cleaned learner_server.py. Added several block function to improve readability.

  Show description for 7c89bd1
  

  michel-aractingi

  and

  AdilZouitine

  committed
  7c89bd1

  Copy full SHA for 7c89bd1

• [Port HIL-SERL] Add HF vision encoder option in SAC (#651)

  Show description for f1c8bfe
  

  ChorntonYoel

  authored and

  michel-aractingi

  committed
  f1c8bfe

  Copy full SHA for f1c8bfe

• - Refactor observation encoder in modeling_sac.py

  Show description for 506821c
  

  michel-aractingi

  and

  AdilZouitine

  committed
  506821c

  Copy full SHA for 506821c

• - Added base gym env class for the real robot environment.

  Show description for 2211209
  

  michel-aractingi

  and

  AdilZouitine

  committed
  2211209

  Copy full SHA for 2211209

• Added crop_dataset_roi.py that allows you to load a lerobotdataset -> crop its images -> create a new lerobot dataset with the cropped and resized images.

  Show description for efb1982
  

  michel-aractingi

  and

  AdilZouitine

  committed
  efb1982

  Copy full SHA for efb1982

Commits on Feb 4, 2025

• Added additional wrappers for the environment: Action repeat, keyboard interface, reset wrapper

  Show description for e0527b4
  

  michel-aractingi

  and

  AdilZouitine

  committed
  e0527b4

  Copy full SHA for e0527b4

Commits on Feb 5, 2025

• fixed bug in crop_dataset_roi.py

  Show description for 7d5a953
  

  michel-aractingi

  and

  AdilZouitine

  committed
  7d5a953

  Copy full SHA for 7d5a953

Commits on Feb 6, 2025

• - Added lerobot/scripts/server/gym_manipulator.py that contains all the necessary wrappers to run a gym-style env around the real robot.

  Show description for 1252524
  

  michel-aractingi

  and

  AdilZouitine

  committed
  1252524

  Copy full SHA for 1252524

• [HIL-SERL port] Add Reward classifier benchmark tracking to chose best visual encoder (#688)

  helper2424

  authored
  b637386

  Copy full SHA for b637386

Commits on Feb 10, 2025

• Several fixes to move the actor_server and learner_server code from the maniskill environment to the real robot environment.

  Show description for d51374c
  

  michel-aractingi

  and

  AdilZouitine

  committed
  d51374c

  Copy full SHA for d51374c

• Added sac_real config file in the policym configs dir.

  Show description for b5f8943
  michel-aractingi

  committed
  b5f8943

  Copy full SHA for b5f8943

Commits on Feb 11, 2025

• - Added JointMaskingActionSpace wrapper in gym_manipulator in order to select which joints will be controlled. For example, we can disable the gripper actions for some tasks.

  Show description for a7db395
  

  michel-aractingi

  and

  AdilZouitine

  committed
  a7db395

  Copy full SHA for a7db395

• [Port HIL-SERL] Add resnet-10 as default encoder for HIL-SERL (#696)

  Show description for a1d16fb
  

  5 peopleauthored
  a1d16fb

  Copy full SHA for a1d16fb

• [PORT-Hilserl] classifier fixes (#695)

  Show description for 6868c88
  

  3 peopleauthored
  6868c88

  Copy full SHA for 6868c88

Commits on Feb 12, 2025

• Added possiblity to record and replay delta actions during teleoperation rather than absolute actions

  Show description for b9217b0
  

  michel-aractingi

  and

  AdilZouitine

  committed
  b9217b0

  Copy full SHA for b9217b0

Commits on Feb 13, 2025

• Added logging for interventions to monitor the rate of interventions through time

  Show description for dc086dc
  

  michel-aractingi

  and

  AdilZouitine

  committed
  dc086dc

  Copy full SHA for dc086dc

• fix log_alpha in modeling_sac: change to nn.parameter

  Show description for 459f22e
  

  michel-aractingi

  and

  AdilZouitine

  committed
  459f22e

  Copy full SHA for 459f22e

• Hardcoded some normalization parameters. TODO refactor

  Show description for c462a47
  

  michel-aractingi

  and

  AdilZouitine

  committed
  c462a47

  Copy full SHA for c462a47

• Changed bounds for a new so100 robot

  Show description for 0c32008
  

  michel-aractingi

  and

  AdilZouitine

  committed
  0c32008

  Copy full SHA for 0c32008

• Changed the init_final value to center the starting mean and std of the policy

  Show description for d9a7037
  

  michel-aractingi

  and

  AdilZouitine

  committed
  d9a7037

  Copy full SHA for d9a7037

• removed uncomment in actor server

  Show description for b07d95f
  

  michel-aractingi

  and

  AdilZouitine

  committed
  b07d95f

  Copy full SHA for b07d95f

• nit

  Show description for 95de8e2
  

  michel-aractingi

  and

  AdilZouitine

  committed
  95de8e2

  Copy full SHA for 95de8e2

• Optimized the replay buffer from the memory side to store data on cpu instead of a gpu device and send the batches to the gpu.

  Show description for c9e50bb
  

  michel-aractingi

  and

  AdilZouitine

  committed
  c9e50bb

  Copy full SHA for c9e50bb

Commits on Feb 14, 2025

• Modified crop_dataset_roi interface to automatically write the cropped parameters to a json file in the meta of the dataset

  Show description for 36711d7
  

  michel-aractingi

  and

  AdilZouitine

  committed
  36711d7

  Copy full SHA for 36711d7

• Fixed bug in the action scale of the intervention actions and offline dataset actions. (scale by inverse delta)

  Show description for 7ae368e
  

  michel-aractingi

  and

  Adil Zouitine

  committed
  7ae368e

  Copy full SHA for 7ae368e

• Add maniskill support.

  Show description for 2f3370e
  AdilZouitine

  committed
  2f3370e

  Copy full SHA for 2f3370e

Commits on Feb 17, 2025

• Improve wandb logging and custom step tracking in logger

  Show description for 446f434
  AdilZouitine

  committed
  446f434

  Copy full SHA for 446f434

• Re-enable parameter push thread in learner server

  Show description for befa1fe
  AdilZouitine

  committed
  befa1fe

  Copy full SHA for befa1fe

Commits on Feb 19, 2025

• - Fixed big issue in the loading of the policy parameters sent by the learner to the actor -- pass only the actor to the update_policy_parameters and remove strict=False

  Show description for ff47c0b
  

  michel-aractingi

  and

  AdilZouitine

  committed
  ff47c0b

  Copy full SHA for ff47c0b

Commits on Feb 20, 2025

• Refactor SAC policy with performance optimizations and multi-camera support

  Show description for ff82367
  AdilZouitine

  committed
  ff82367

  Copy full SHA for ff82367

Commits on Feb 21, 2025

• [Port HIL-SERL] Adjust Actor-Learner architecture & clean up dependency management for HIL-SERL (#722)

  helper2424

  authored
  3ffe0cf

  Copy full SHA for 3ffe0cf

• Added caching function in the learner_server and modeling sac in order to limit the number of forward passes through the pretrained encoder when its frozen.

  Show description for 5467191
  

  michel-aractingi

  and

  AdilZouitine

  committed
  5467191

  Copy full SHA for 5467191

Commits on Feb 24, 2025

• Update ManiSkill configuration and replay buffer to support truncation and dataset handling

  Show description for 42a0381
  AdilZouitine

  committed
  42a0381

  Copy full SHA for 42a0381

Commits on Feb 25, 2025

• Refactor ReplayBuffer with tensor-based storage and improved sampling efficiency

  Show description for ef8d943
  AdilZouitine

  committed
  ef8d943

  Copy full SHA for ef8d943

• Add storage device parameter to replay buffer initialization

  Show description for 5b4a7aa
  AdilZouitine

  committed
  5b4a7aa

  Copy full SHA for 5b4a7aa

• Add memory optimization option to ReplayBuffer

  Show description for 1df9ee4
  AdilZouitine

  committed
  1df9ee4

  Copy full SHA for 1df9ee4

Commits on Mar 4, 2025

• Add storage device configuration for SAC policy and replay buffer

  Show description for d8a1758
  AdilZouitine

  committed
  d8a1758

  Copy full SHA for d8a1758

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 584cad8
  pre-commit-ci[bot]

  committed
  584cad8

  Copy full SHA for 584cad8

Commits on Mar 5, 2025

• [HIL-SERL] Migrate threading to multiprocessing (#759)

  Show description for 700f00c
  

  helper2424

  and

  pre-commit-ci[bot]

  authored
  700f00c

  Copy full SHA for 700f00c

Commits on Mar 12, 2025

• [Port HIL-SERL] Balanced sampler function speed up and refactor to align with train.py (#715)

  Show description for d711e20
  

  s1lent4gnt

  and

  pre-commit-ci[bot]

  authored
  d711e20

  Copy full SHA for d711e20

• Remove torch.no_grad decorator and optimize next action prediction in SAC policy

  Show description for 25b88f3
  AdilZouitine

  committed
  25b88f3

  Copy full SHA for 25b88f3

• Add custom save and load methods for SAC policy

  Show description for 5081c14
  AdilZouitine

  committed
  5081c14

  Copy full SHA for 5081c14

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 41219fe
  pre-commit-ci[bot]

  committed
  41219fe

  Copy full SHA for 41219fe