Add compression for UIE #3496

LiuChiachi · 2022-10-17T12:26:36Z

PR types

New features

PR changes

Models

Description

Add Trainer implementation
Add compression for UIE example(QAT)

wawltor · 2022-10-25T13:45:42Z

docs/compression.md

   * [模型评估与部署](#模型评估与部署)
   * [FAQ](#FAQ)
   * [参考文献](#References)


-<a name="模型压缩API功能介绍"></a>
+<a name="模型压缩API功能简介"></a>


在文档里面还是建议说清楚一些技术概念，可以通过图片的方式解释

同时压缩API的使用流程，裁剪和量化的区别；静态量化和动态量化在使用的流程区别在什么地方需要说清楚

需要在下一个PR中对压缩API的文档进行进一步更新。

wawltor · 2022-10-25T13:48:01Z

docs/compression.md

+```
+
+示例代码中压缩使用的是 datasets 内置的数据集，若想要使用自定义数据集压缩，可参考 [datasets 加载自定义数据集文档](https://huggingface.co/docs/datasets/loading)。
+


这里还有另外一个问题，cpu和gpu量化对机器要求不一样，这里是否可以概括一下

已经把对机器的要求包括在4.7模型部署中的使用建议中

wawltor · 2022-10-25T13:48:49Z

model_zoo/uie/README.md

- `logging_steps`: 日志打印的间隔steps数，默认10。
- `valid_steps`: evaluate的间隔steps数，默认100。
- `device`: 选用什么设备进行训练，可选cpu或gpu。
+可配置的压缩相关的参数：


这里有些担忧，do_compress是不是要关闭最好，因为开源用户的机器可能无法支持INT8量化

同时动态量化模型导出的模型是静态图，而之前我们是有export_model.py脚本，如何整合了？

do_compress已关闭，感谢指出

wawltor · 2022-10-25T13:49:34Z

model_zoo/uie/deploy/python/uie_predictor.py

-        self.inference_backend = InferBackend(args.model_path_prefix,
-                                              device=args.device,
-                                              use_fp16=args.use_fp16,
-                                              device_id=args.device_id)


INT8量化模型和use_fp16这里要不要加一个逻辑，同时FP16和INT8如何使用能不能有个最佳指南

最佳指南在文档中已给出

这里还是不太明确怎么使用INT8的模型

wawltor · 2022-10-26T02:56:15Z

model_zoo/uie/run_trainer.py

+    # for key, val in resource_file_urls.items():
+    #     file_path = os.path.join(model_args.model_name_or_path, key)
+    #     if not os.path.exists(file_path):
+    #         get_path_from_url(val, model_args.model_name_or_path)


删除一下

Done Thanks

wawltor · 2022-11-01T03:41:50Z

model_zoo/uie/README.md

-<a name="模型评估"></a>
-
-#### 4.4 模型评估
+export finetuned_model=uie_finetuned


这里保存模型地址和之前下面的目录有冲突

感谢提醒，目录和下面进行了统一

wawltor · 2022-11-01T03:42:13Z

model_zoo/uie/README.md

+    --overwrite_output_dir \
+    --disable_tqdm True \
+    --metric_for_best_model eval_f1 \
+    --load_best_model_at_end  True \


多卡的示例是不是要保留

多卡的脚本已经补充出来了，感谢提醒，应该和之前体验保持一致

wawltor · 2022-11-01T03:42:42Z

model_zoo/uie/deploy/python/uie_predictor.py

-        self.inference_backend = InferBackend(args.model_path_prefix,
-                                              device=args.device,
-                                              use_fp16=args.use_fp16,
-                                              device_id=args.device_id)


这里还是不太明确怎么使用INT8的模型

wawltor · 2022-11-01T03:44:06Z

model_zoo/uie/README.md


-通过运行以下命令进行模型评估：
-


这里的evaluate的逻辑还包括了debug逻辑不建议删除

已经恢复了evaluate.py

linjieccc · 2022-11-01T12:50:40Z

model_zoo/uie/README.md

+    --learning_rate 1e-5 \
+    --do_train \
+    --do_eval \
+    --do_export \


我看最新的finetune脚本这里已经默认开启了do_export，后面的模型导出章节是否可以删掉和模型压缩的流程保持一致

已经删掉了，只用了一句话介绍了下导出模型

linjieccc · 2022-11-02T02:42:50Z

model_zoo/uie/README.md

+* `device`: 训练设备，可选择 'cpu'、'gpu' 其中的一种；默认为 GPU 训练。
+* `per_device_train_batch_size`：训练集训练过程批处理大小，请结合显存情况进行调整，若出现显存不足，请适当调低这一参数；默认为 32。
+* `per_device_eval_batch_size`：开发集评测过程批处理大小，请结合显存情况进行调整，若出现显存不足，请适当调低这一参数；默认为 32。
+* `learning_rate`：训练最大学习率；默认为3e-5。


UIE微调学习率默认值建议用1e-5

好的，因为这里用的Trainer API，可能默认值不好直接改，但是我这里对文档进行了补充，：训练最大学习率，UIE 推荐设置为 1e-5；默认值为3e-5。

linjieccc · 2022-11-02T02:44:42Z

model_zoo/uie/README.md

+* `logging_steps`: 训练过程中日志打印的间隔 steps 数，默认100。
+* `save_steps`: 训练过程中保存模型 checkpoint 的间隔 steps 数，默认100。
+* `seed`：全局随机种子，默认为 42。
+* `label_names`：训练数据标签的key的名称，例如 'labels'、'start_postions' 或者 'end_positions' 等。自定义数据集需要设置，否则可能会造成识别错误；默认为 None。


这里如果UIE默认需要用'start_postions' + 'end_positions'是否可以不作为可配置项暴露出来

通过对Trainer API进行修改，修改了UIE 模型的默认label_names值，不需要让用户再学习使用这个参数。Trainer 这里也请@ZHUI 帮忙review下

linjieccc · 2022-11-02T02:50:06Z

model_zoo/uie/README.md

@@ -625,57 +626,96 @@ python labelstudio2doccano.py --labelstudio_file label-studio.json

 #### 4.3 模型微调

+推荐使用 [Trainer API ](../../docs/trainer.md) 对模型进行微调。只需输入模型、数据集等就可以使用 Trainer API 高效快速地进行预训练、微调和模型压缩等任务，可以一键启动多卡训练、混合精度训练、梯度累积、断点重启、日志显示等功能，Trainer API 还针对训练过程的通用训练配置做了封装，比如：优化器、学习率调度等。
+
+使用下面的命令，使用 `uie-base` 作为预训练模型进行模型微调，将微调后的模型保存至`$finetuned_model`：


想问下这里改为trainer后保存路径下的文件及命名是否和之前一致，后续Taskflow加载定制模型需要用到

最好的动态图模型、导出的静态图模型、量化后模型都在 ./checkpoint/model_best下面，只是用 $finetuned_model 变量来表示，但是这样是不是不能确保用户一定会存在 ./checkpoint/model_best下面

linjieccc

LGTM for UIE

wawltor

LGTM

wanshichenguang · 2024-03-09T05:47:16Z

给个环境配置吧，uie训练加上do_compress参数一直压缩失败。
通过paddleslim也报同样的错.
[2024-03-09 13:39:44,804] [ INFO] - Exporting inference model to ./checkpoint/model_best/model
/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddle/fluid/variable_index.py:591: UserWarning: Warning: In Tensor 'getitem', if the number of scalar elements in the index is equal to the rank of the Tensor, the output should be 0-D. In order to be consistent with the behavior of previous versions, it will be processed to 1-D. But it is not correct and will be removed in release 2.6. If 1-D is still wanted, please modify the index element from scalar to slice (e.g. 'x[i]' => 'x[i:i+1]').
I0309 13:39:45.839694 25187 interpretercore.cc:237] New Executor is Running.
[2024-03-09 13:39:50,783] [ INFO] - Inference model exported.
[2024-03-09 13:39:50,863] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/tokenizer_config.json
[2024-03-09 13:39:50,864] [ INFO] - Special tokens file saved in ./checkpoint/model_best/special_tokens_map.json
Sat Mar 09 13:39:50 Dynamic-to-Static INFO: (Level -1) Not recommend to deepcopy 'UIE' decorated with @to_static, it has side effect that will rollback into original state before @to_static. Please deepcopy 'UIE' before applying @to_static.
/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils_base.py:2538: FutureWarning: The max_seq_len argument is deprecated and will be removed in a future version, please use max_length instead.
/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils_base.py:1938: FutureWarning: The pad_to_max_length argument is deprecated and will be removed in a future version, use padding=True or padding='longest' to pad to the longest sequence in the batch, or use padding='max_length' to pad to a max length. In this case, you can give a specific length with max_length (e.g. max_length=45) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).
Traceback (most recent call last):
File "/root/source/nlp/PaddleNLP-develop/model_zoo/uie/finetune.py", line 262, in
main()
File "/root/source/nlp/PaddleNLP-develop/model_zoo/uie/finetune.py", line 258, in main
trainer.compress(custom_evaluate=custom_evaluate)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddlenlp/trainer/trainer_compress.py", line 73, in compress
_dynabert(self, self.model)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddlenlp/trainer/trainer_compress.py", line 158, in _dynabert
ofa_model, teacher_model = _dynabert_init(self, model, eval_dataloader)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddlenlp/trainer/trainer_compress.py", line 300, in _dynabert_init
head_importance, neuron_importance = compute_neuron_head_importance(
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddlenlp/transformers/ofa_utils.py", line 307, in compute_neuron_head_importance
logits = model(**batch)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1254, in call
return self.forward(*inputs, **kwargs)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddlenlp/transformers/ernie/modeling.py", line 1269, in forward
sequence_output, _ = self.ernie(
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1256, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1235, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddlenlp/trainer/trainer_compress.py", line 907, in auto_model_dynabert_forward
embedding_output = self.embeddings(**embedding_kwargs)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1256, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1235, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddlenlp/transformers/ernie/modeling.py", line 127, in forward
embeddings = self.layer_norm(embeddings)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1254, in call
return self.forward(*inputs, **kwargs)
File "/root/miniconda3/envs/paddlenlp/lib/python3.9/site-packages/paddleslim/nas/ofa/layers.py", line 1301, in forward
out, _, _ = paddle._C_ops.layer_norm(
ValueError: too many values to unpack (expected 3)
(paddlenlp) [root@localhost uie]# pip list|grep paddlenlp
paddlenlp 2.7.2
WARNING: There was an error checking the latest version of pip.
(paddlenlp) [root@localhost uie]# pip list|grep paddle
paddle-bfloat 0.1.7
paddle2onnx 1.1.0
paddlefsl 1.1.0
paddlenlp 2.7.2
paddleocr 2.6.1.3
paddlepaddle-gpu 2.5.1
paddleslim 2.6.0

add compression for uie

788c401

LiuChiachi self-assigned this Oct 17, 2022

LiuChiachi added the model-compression label Oct 17, 2022

LiuChiachi force-pushed the add-compress-for-uie branch 2 times, most recently from 5dd6387 to b28d7cd Compare October 24, 2022 03:23

add compression for uie

76c05e2

LiuChiachi force-pushed the add-compress-for-uie branch from b28d7cd to 76c05e2 Compare October 24, 2022 03:47

wawltor reviewed Oct 25, 2022

View reviewed changes

wawltor reviewed Oct 26, 2022

View reviewed changes

LiuChiachi force-pushed the add-compress-for-uie branch from 8ca31a8 to 759d620 Compare October 31, 2022 12:29

update uie readme

e546afd

LiuChiachi force-pushed the add-compress-for-uie branch from 759d620 to e546afd Compare October 31, 2022 13:51

wawltor reviewed Nov 1, 2022

View reviewed changes

wawltor mentioned this pull request Nov 1, 2022

[Question]: 如何提高UIE使用CPU部署的model.onnx模型的推理性能？ #3636

Closed

LiuChiachi added 7 commits November 1, 2022 07:51

update reamde, recover evaluate.py

d53f6e8

solve conficts

63b12ee

fix resource check

e378ff3

fix qat ernie-m

98fa7eb

requirement

43b50f2

update

6fa1b6f

update

ebd8f0b

linjieccc reviewed Nov 1, 2022

View reviewed changes

LiuChiachi added 2 commits November 1, 2022 13:34

delete export_model.py

ddc9449

delete model export in readme

54ac5ae

linjieccc reviewed Nov 2, 2022

View reviewed changes

set start_positions and end_positions to default for UIE

207e090

linjieccc approved these changes Nov 2, 2022

View reviewed changes

wawltor approved these changes Nov 2, 2022

View reviewed changes

LiuChiachi merged commit e94af79 into PaddlePaddle:develop Nov 2, 2022

LiuChiachi mentioned this pull request Nov 17, 2022

PaddleNLP 2.4.3 Release Note Candidate #3774

Closed

linjieccc mentioned this pull request Jan 12, 2023

PaddleNLP 2.5.0 Release Note Candidate #4439

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add compression for UIE #3496

Add compression for UIE #3496

LiuChiachi commented Oct 17, 2022

wawltor Oct 25, 2022

LiuChiachi Oct 31, 2022

wawltor Oct 25, 2022

LiuChiachi Oct 31, 2022

wawltor Oct 25, 2022

LiuChiachi Oct 31, 2022

wawltor Oct 25, 2022

LiuChiachi Oct 31, 2022

wawltor Nov 1, 2022

wawltor Oct 26, 2022

LiuChiachi Oct 31, 2022

wawltor Nov 1, 2022

LiuChiachi Nov 1, 2022

wawltor Nov 1, 2022

LiuChiachi Nov 1, 2022

wawltor Nov 1, 2022

wawltor Nov 1, 2022

LiuChiachi Nov 1, 2022

linjieccc Nov 1, 2022 •

edited

Loading

LiuChiachi Nov 1, 2022

linjieccc Nov 2, 2022

LiuChiachi Nov 2, 2022

linjieccc Nov 2, 2022

LiuChiachi Nov 2, 2022

linjieccc Nov 2, 2022

LiuChiachi Nov 2, 2022

linjieccc left a comment

wawltor left a comment

wanshichenguang commented Mar 9, 2024

		```

		示例代码中压缩使用的是 datasets 内置的数据集，若想要使用自定义数据集压缩，可参考 [datasets 加载自定义数据集文档](https://huggingface.co/docs/datasets/loading)。


		通过运行以下命令进行模型评估：

Add compression for UIE #3496

Add compression for UIE #3496

Conversation

LiuChiachi commented Oct 17, 2022

PR types

PR changes

Description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linjieccc Nov 1, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linjieccc left a comment

Choose a reason for hiding this comment

wawltor left a comment

Choose a reason for hiding this comment

wanshichenguang commented Mar 9, 2024

linjieccc Nov 1, 2022 •

edited

Loading