-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug causes error uploading to huggingface, unicode issue on windows. #450
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Cadene
approved these changes
Nov 22, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
@resolver101757 If you find any other incompatibility issue, please let me know. I will be more reactive this time!!! |
ChorntonYoel
pushed a commit
to ChorntonYoel/lerobot
that referenced
this pull request
Nov 22, 2024
astroyat
pushed a commit
to astroyat/lerobot
that referenced
this pull request
Nov 23, 2024
astroyat
pushed a commit
to astroyat/lerobot
that referenced
this pull request
Nov 23, 2024
DomThePorcupine
pushed a commit
to DomThePorcupine/lerobot
that referenced
this pull request
Dec 2, 2024
Cadene
pushed a commit
that referenced
this pull request
Dec 28, 2024
villekuosmanen
added a commit
to villekuosmanen/lerobot
that referenced
this pull request
Dec 30, 2024
* feat: enable to use multiple rgb encoders per camera in diffusion policy (huggingface#484) Co-authored-by: Alexander Soare <alexander.soare159@gmail.com> * Fix config file (huggingface#495) * fix: broken images and a few minor typos in README (huggingface#499) Signed-off-by: ivelin <ivelin117@gmail.com> * Add support for Windows (huggingface#494) * bug causes error uploading to huggingface, unicode issue on windows. (huggingface#450) * Add distinction between two unallowed cases in name check "eval_" (huggingface#489) * Rename deprecated argument (temporal_ensemble_momentum) (huggingface#490) * Dataset v2.0 (huggingface#461) Co-authored-by: Remi <remi.cadene@huggingface.co> * Refactor OpenX (huggingface#505) * Fix missing local_files_only in record/replay (huggingface#540) Co-authored-by: Simon Alibert <alibert.sim@gmail.com> * Control simulated robot with real leader (huggingface#514) Co-authored-by: Remi <remi.cadene@huggingface.co> * Update 7_get_started_with_real_robot.md (huggingface#559) * LerobotDataset pushable to HF from any folder (huggingface#563) * Fix example 6 (huggingface#572) * fixing typo from 'teloperation' to 'teleoperation' (huggingface#566) * [vizualizer] for LeRobodDataset V2 (huggingface#576) * Fix broken `create_lerobot_dataset_card` (huggingface#590) * feat(act): support training end of episode token to ACT model * changes * feat(arx): add arx arm (#2) * feat(arx): support arx arm * changes * changes * changes * changes * pass pipes explicitly * changes * us ndarray over a pipe * changes * changes * replay basically works * patch arx sdk * changes * support cameras in arx5 * rename to arx5 * kind of works * changes * changes * changes * various changes * changes * revert a few changes * changes * changes * changes * changes * changes * changes * changes * changes * changes * remove TODO * allow multiple tasks --------- Signed-off-by: ivelin <ivelin117@gmail.com> Co-authored-by: Hirokazu Ishida <38597814+HiroIshida@users.noreply.github.com> Co-authored-by: Alexander Soare <alexander.soare159@gmail.com> Co-authored-by: Arsen Ohanyan <arsenohanyan@gmail.com> Co-authored-by: Ivelin Ivanov <ivelin117@gmail.com> Co-authored-by: Daniel Ritchie <daniel@brainwavecollective.ai> Co-authored-by: resolver101757 <kelster101757@hotmail.com> Co-authored-by: Jannik Grothusen <56967823+J4nn1K@users.noreply.github.com> Co-authored-by: KasparSLT <133706781+KasparSLT@users.noreply.github.com> Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> Co-authored-by: Remi <remi.cadene@huggingface.co> Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co> Co-authored-by: Simon Alibert <alibert.sim@gmail.com> Co-authored-by: berjaoui <berjaoui@gmail.com> Co-authored-by: Claudio Coppola <Claudiocoppola90@gmail.com> Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com> Co-authored-by: Mishig <dmishig@gmail.com> Co-authored-by: Eugene Mironov <helper2424@gmail.com>
villekuosmanen
added a commit
to villekuosmanen/lerobot
that referenced
this pull request
Jan 10, 2025
* feat: enable to use multiple rgb encoders per camera in diffusion policy (huggingface#484) Co-authored-by: Alexander Soare <alexander.soare159@gmail.com> * Fix config file (huggingface#495) * fix: broken images and a few minor typos in README (huggingface#499) Signed-off-by: ivelin <ivelin117@gmail.com> * Add support for Windows (huggingface#494) * bug causes error uploading to huggingface, unicode issue on windows. (huggingface#450) * Add distinction between two unallowed cases in name check "eval_" (huggingface#489) * Rename deprecated argument (temporal_ensemble_momentum) (huggingface#490) * Dataset v2.0 (huggingface#461) Co-authored-by: Remi <remi.cadene@huggingface.co> * Refactor OpenX (huggingface#505) * Fix missing local_files_only in record/replay (huggingface#540) Co-authored-by: Simon Alibert <alibert.sim@gmail.com> * Control simulated robot with real leader (huggingface#514) Co-authored-by: Remi <remi.cadene@huggingface.co> * Update 7_get_started_with_real_robot.md (huggingface#559) * LerobotDataset pushable to HF from any folder (huggingface#563) * Fix example 6 (huggingface#572) * fixing typo from 'teloperation' to 'teleoperation' (huggingface#566) * [vizualizer] for LeRobodDataset V2 (huggingface#576) * Fix broken `create_lerobot_dataset_card` (huggingface#590) * Update README.md (huggingface#612) * Fix Quality workflow (huggingface#622) * fix(docs): typos in benchmark readme.md (huggingface#614) Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> * fix(visualise): use correct language description for each episode id (huggingface#604) Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> * typo fix: batch_convert_dataset_v1_to_v2.py (huggingface#615) Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> * [viz] Fixes & updates to html visualizer (huggingface#617) * fixes to SO-100 readme (huggingface#600) Co-authored-by: Philip Fung <no@one> Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> --------- Signed-off-by: ivelin <ivelin117@gmail.com> Co-authored-by: Hirokazu Ishida <38597814+HiroIshida@users.noreply.github.com> Co-authored-by: Alexander Soare <alexander.soare159@gmail.com> Co-authored-by: Arsen Ohanyan <arsenohanyan@gmail.com> Co-authored-by: Ivelin Ivanov <ivelin117@gmail.com> Co-authored-by: Daniel Ritchie <daniel@brainwavecollective.ai> Co-authored-by: resolver101757 <kelster101757@hotmail.com> Co-authored-by: Jannik Grothusen <56967823+J4nn1K@users.noreply.github.com> Co-authored-by: KasparSLT <133706781+KasparSLT@users.noreply.github.com> Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> Co-authored-by: Remi <remi.cadene@huggingface.co> Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co> Co-authored-by: Simon Alibert <alibert.sim@gmail.com> Co-authored-by: berjaoui <berjaoui@gmail.com> Co-authored-by: Claudio Coppola <Claudiocoppola90@gmail.com> Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com> Co-authored-by: Mishig <dmishig@gmail.com> Co-authored-by: Eugene Mironov <helper2424@gmail.com> Co-authored-by: CharlesCNorton <135471798+CharlesCNorton@users.noreply.github.com> Co-authored-by: Philip Fung <1054593+philfung@users.noreply.github.com> Co-authored-by: Philip Fung <no@one>
aliberts
pushed a commit
that referenced
this pull request
Feb 9, 2025
[Fix] Move back to manual calibration (#488) feat: enable to use multiple rgb encoders per camera in diffusion policy (#484) Co-authored-by: Alexander Soare <alexander.soare159@gmail.com> Fix config file (#495) fix: broken images and a few minor typos in README (#499) Signed-off-by: ivelin <ivelin117@gmail.com> Add support for Windows (#494) bug causes error uploading to huggingface, unicode issue on windows. (#450) Add distinction between two unallowed cases in name check "eval_" (#489) WIP Fix autocalib moss (#486) [Fix] Move back to manual calibration (#488) feat: enable to use multiple rgb encoders per camera in diffusion policy (#484) Co-authored-by: Alexander Soare <alexander.soare159@gmail.com> Fix config file (#495) fix: broken images and a few minor typos in README (#499) Signed-off-by: ivelin <ivelin117@gmail.com> Add support for Windows (#494) bug causes error uploading to huggingface, unicode issue on windows. (#450) Add distinction between two unallowed cases in name check "eval_" (#489) Rename deprecated argument (temporal_ensemble_momentum) (#490) Dataset v2.0 (#461) Co-authored-by: Remi <remi.cadene@huggingface.co> Refactor OpenX (#505) Fix missing local_files_only in record/replay (#540) Co-authored-by: Simon Alibert <alibert.sim@gmail.com> Control simulated robot with real leader (#514) Co-authored-by: Remi <remi.cadene@huggingface.co> Update 7_get_started_with_real_robot.md (#559) LerobotDataset pushable to HF from any folder (#563) Fix example 6 (#572) fixing typo from 'teloperation' to 'teleoperation' (#566) [vizualizer] for LeRobodDataset V2 (#576) Fix broken `create_lerobot_dataset_card` (#590) Update README.md (#612) Add draccus, create MainConfig WIP refactor train.py and ACT Add policies training presets Update diffusion policy Add pusht and xarm env configs Update tdmpc Update vqbet Fix poetry relax Add feature types to envs Add EvalPipelineConfig, parse features from envs Add custom parser Update pretrained loading mechanisms Add dependency fixes & lock update Fix pretrained_path Refactor envs, remove RealEnv Fix typo Enable end-to-end tests Fix Makefile Log eval config Fix end-to-end tests Fix Quality workflow (#622) Remove amp & add resume test Speed-up tests Fix poetry relax Remove config yaml for robot devices (#594) Co-authored-by: Simon Alibert <simon.alibert@huggingface.co> fix(docs): typos in benchmark readme.md (#614) Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> fix(visualise): use correct language description for each episode id (#604) Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> typo fix: batch_convert_dataset_v1_to_v2.py (#615) Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> [viz] Fixes & updates to html visualizer (#617) Fix logger Remove hydra-core Add aggregate_stats Add estimate_num_samples for images, Add test image Remove NoneSchedulerConfig Add push_pretrained Remove eval.episode_length Fix wandb_video Fix typo Add features back into policy configs (#643) fixes to SO-100 readme (#600) Co-authored-by: Philip Fung <no@one> Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> Fix for the issue #638 (#639) Fix env_to_policy_features call Fix wandb init remove omegaconf Add branch arg Move deprecated Move training config Remove pathable_args Implement custom HubMixin Fixes Implement PreTrainedPolicy base class Add HubMixin to TrainPipelineConfig Udpate example 2 & 3 Update push_pretrained Bump`rerun-sdk` dependency to `0.21.0` (#618) Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> Fix config_class Fix from_pretrained kwargs Remove policy_protocol Camelize PretrainedConfig Additional fix while retraining policies (#629) Co-authored-by: Simon Alibert <simon.alibert@huggingface.co> Actually reactivate tdmpc online test Update example 4 Remove advanced example 1 Remove example 5 Move example 6 to advanced Use HubMixin.save_pretrained Enable config_path to be a repo_id Dry has_method Update example 4 Update README Cleanup pyproject.toml Update eval docstring Update README Clean example 4 Update README Make 'last' checkpoint symlink relative Fix cluster image (#653) Simplify example 4 fix stats per episodes and aggregate stats and casting to tensor
JIy3AHKO
pushed a commit
to vertix/lerobot
that referenced
this pull request
Feb 27, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this does
This PR resolves a bug in the LeRobot repository that causes an error when uploading a dataset to Huggingface on Windows due to a Unicode encoding issue. The problem arises because Windows, by default, uses cp1252 encoding, which cannot handle certain Unicode characters, such as emojis. The error is caused by the dataset card containing an emoji.
Error trace:
The error produced is 'Traceback (most recent call last):
File "D:\dev\lerobot\lerobot\scripts\control_robot.py", line 885, in
record(robot, **kwargs)
File "D:\dev\lerobot\lerobot\scripts\control_robot.py", line 681, in record
push_dataset_card_to_hub(repo_id, revision="main", tags=tags)
File "D:\dev\lerobot\lerobot\scripts\push_dataset_to_hub.py", line 124, in push_dataset_card_to_hub
card.push_to_hub(repo_id=repo_id, repo_type="dataset", revision=revision)
File "C:\Users\tblocal.conda\envs\lerobot\lib\site-packages\huggingface_hub\repocard.py", line 277, in push_to_hub
tmp_path.write_text(str(self))
File "C:\Users\tblocal.conda\envs\lerobot\lib\pathlib.py", line 1155, in write_text
return f.write(data)
File "C:\Users\tblocal.conda\envs\lerobot\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f917' in position 102: character maps to
This fix removes the emoji 🤗 which prevents UnicodeEncodeError.
How it was tested
python lerobot/scripts/control_robot.py record --robot-path lerobot/configs/robot/koch_original.yaml --fps 30 --root data --repo-id RESOLVER101757/koch_test_2 --tags tutorial --warmup-time-s 5 --episode-time-s 30 --reset-time-s 30 --num-episodes 1
How to checkout & try? (for the reviewer)
python lerobot/scripts/control_robot.py record --robot-path lerobot/configs/robot/koch_original.yaml --fps 30 --root data --repo-id RESOLVER101757/koch_test_2 --tags tutorial --warmup-time-s 5 --episode-time-s 30 --reset-time-s 30 --num-episodes 1