Support more model outputs for BERT/ERNIE/RoBERTa

[guoshengCS]

PR types

New features

PR changes

APIs

Description

Support more model outputs for BERT. The default behavior is the same as before for compatibility.

Import more output structures refer to HF.

    BaseModelOutputWithPastAndCrossAttentions,
    BaseModelOutputWithPoolingAndCrossAttentions,
    SequenceClassifierOutput,
    TokenClassifierOutput,
    QuestionAnsweringModelOutput,
    MultipleChoiceModelOutput,
    MaskedLMOutput,
    ModelOutput,

Usage:

import paddle
import paddlenlp
from paddlenlp.transformers import AutoModel, AutoTokenizer, AutoModelForTokenClassification, AutoModelForPretraining

paddle.seed(123)
model_name = "bert-base-uncased"
model_name = "ernie-1.0-base-zh"
model_name = "hfl/roberta-wwm-ext"
#model = AutoModel.from_pretrained(model_name)
model = AutoModelForTokenClassification.from_pretrained(model_name)
#model = AutoModelForPretraining.from_pretrained(model_name)
model.eval()
tokenizer = AutoTokenizer.from_pretrained(model_name)

inputs = tokenizer("欢迎使用百度飞桨!")
inputs = {k:paddle.to_tensor([v]) for (k, v) in inputs.items()}
print(inputs)
outputs = model(**inputs, return_dict=True, output_hidden_states=True, output_attentions=False)
print(outputs)

[yingyibiao]

这里的 import 是否应该挪到文件开头位置

[guoshengCS]

Done

[yingyibiao]

HF 中的 all_hidden_states 包含了 embedding_output，这里的实现相比会少这个 Tensor。

[guoshengCS]

Done. Thanks

[yingyibiao]

LGTM