Add onnx config for RoFormer

[skrsna]

What does this PR do?

Before submitting

• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
  • ONNXConfig: Add a configuration for all available models #16308
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• [ ] Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[chainyo]

Hey nice PR! Did you try to convert one RoFormer model with your add ?
If so you could add the converted model to the ONNXConfig for all organization, it would be awesome!

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[skrsna]

Hey nice PR! Did you try to convert one RoFormer model with your add ? If so you could add the converted model to the ONNXConfig for all organization, it would be awesome!

python -m transformers.onnx -m junnyu/roformer_chinese_sim_char_ft_small onnx/model.onnx
Validating ONNX model...
	-[✓] ONNX model output names match reference model ({'last_hidden_state'})
	- Validating ONNX Model output "last_hidden_state":
		-[✓] (2, 8, 384) matches (2, 8, 384)
		-[✓] all values close (atol: 1e-05)
All good, model saved at: onnx/model.onnx

:)

[chainyo]

It seems that there is 2x sequence-classification for features

All good, model saved at: onnx/model.onnx

Nice news!

[lewtun]

Thanks for adding RoFormer @skrsna and thank you to @chainyo for the review!

I agree with @chainyo's suggestions and would also like to see this included in the slow tests if possible. Could you add a roformer checkpoint to text_onnx_v2.py please, e.g. maybe this works well: https://huggingface.co/junnyu/roformer_chinese_base

Then please run

RUN_SLOW=1 pytest tests/onnx/test_onnx_v2.py -s -k "roformer"

[skrsna]

Thanks for adding RoFormer @skrsna and thank you to @chainyo for the review!

I agree with @chainyo's suggestions and would also like to see this included in the slow tests if possible. Could you add a roformer checkpoint to text_onnx_v2.py please, e.g. maybe this works well: https://huggingface.co/junnyu/roformer_chinese_base

Then please run

RUN_SLOW=1 pytest tests/onnx/test_onnx_v2.py -s -k "roformer"

Thanks for the review @chainyo and @lewtun, I added roformer to the onnx test case

tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_65_roformer_causal_lm If you want to use `RoFormerForCausalLM` as a standalone, add `is_decoder=True.`
PASSED
tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_66_roformer_default PASSED
tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_67_roformer_masked_lm PASSED
tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_68_roformer_multiple_choice PASSED
tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_69_roformer_question_answering PASSED
tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_70_roformer_sequence_classification PASSED
tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_71_roformer_token_classification PASSED

[lewtun]

Great, thanks for adding the slow test and checking that it works!

This LGTM so gently pinging @sgugger for final approval

[sgugger]

Thanks for your PR!