Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for HubertForCTC #347

Merged
merged 5 commits into from
Apr 24, 2023
Merged

Add support for HubertForCTC #347

merged 5 commits into from
Apr 24, 2023

Conversation

jimypbr
Copy link
Contributor

@jimypbr jimypbr commented Apr 20, 2023

What does this PR do?

Adds support for HubertForCTC model.

Can fine-tune it on librispeech with the command:

python examples/speech-recognition/run_speech_recognition_ctc.py \
    --dataset_name "librispeech_asr" \
    --dataset_config_name "clean" \
    --train_split_name "train.100" \
    --eval_split_name "validation" \
    --model_name_or_path facebook/hubert-base-ls960 \
    --ipu_config_name Graphcore/wav2vec2-ctc-base-ipu \
    --ipu_config_overrides "device_iterations=16,inference_device_iterations=16" \
    --mask_time_prob 0.0 \
    --mask_feature_prob 0.0 \
    --output_dir "./hubert-base-960h" \
    --overwrite_output_dir \
    --length_column_name "input_length" \
    --num_train_epochs 5 \
    --learning_rate "3e-4" \
    --warmup_steps 400 \
    --evaluation_strategy "steps" \
    --text_column_name "text" \
    --save_steps 400 \
    --eval_steps 400 \
    --logging_steps 10 \
    --save_total_limit 1 \
    --freeze_feature_encoder \
    --do_train \
    --do_eval \
    --layerdrop 0.0 \
    --per_device_train_batch_size 1 \
    --per_device_eval_batch_size 2 \
    --pod_type pod8 \
    --adam_beta1 0.9 \
    --adam_beta2 0.98 \
    --adam_epsilon 0.0001 \
    --report_to wandb \
    --dataloader_drop_last \
    --dataloader_mode "async_rebatched" \
    --dataloader_num_workers 8

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Apr 20, 2023

The documentation is not available anymore as the PR was closed or merged.

Undo the changes to the model done by `parallelize`.
"""
super().deparallelize()
self.change_hubert_encoder_class(True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in parallelize the feature encoder gets frozen; does any "unfreezing" need to happen here?

@jimypbr jimypbr merged commit 869cd06 into main Apr 24, 2023
@jimypbr jimypbr deleted the hubert-ctc branch April 24, 2023 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants