【Hackathon 5th No.103】move skip_layernorm/fused_bias_dropout_residual_layer_norm to phi #58396

zeroRains · 2023-10-26T01:53:17Z

PR types

Others

PR changes

Others

Description

move skip_layernorm/fused_bias_dropout_residual_layer_norm to phi
#57262

…unctor.h

paddle-bot · 2023-10-26T01:53:22Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

zeroRains · 2023-10-27T11:01:15Z

额，请问这个应该怎么转化成phi中的函数参数变量名呢？

zeroRains · 2023-10-28T07:23:02Z

额，Cmake的时候说反向输出的数量比正向输出的数量要多，所以失败了，我数了一下.cc文件里正向有5个输出，反向有6个输出，但是反向的3个输出是要被3个可选参数控制的（没有可选参数就没有这三个输出）。请问这种情况应该怎么解决呢？还是我哪里理解错了吗？

yuanlehome · 2023-11-03T08:59:16Z

额，请问这个应该怎么转化成phi中的函数参数变量名呢？

op_compat.yaml 放在extra里

yuanlehome · 2023-11-03T09:01:33Z

额，Cmake的时候说反向输出的数量比正向输出的数量要多，所以失败了，我数了一下.cc文件里正向有5个输出，反向有6个输出，但是反向的3个输出是要被3个可选参数控制的（没有可选参数就没有这三个输出）。请问这种情况应该怎么解决呢？还是我哪里理解错了吗？

这应该就是输入输出数量没对应好，可以再检查一下～

zeroRains · 2023-11-04T06:03:30Z

paddle/fluid/operators/fused/fused_bias_dropout_residual_layer_norm_op.cc

-    AddOutput("BiasDropoutResidualOut", "Output of bias + dropout + residual.")
-        .AsIntermediate();
-    AddOutput("DropoutMaskOut", "The random sampled dropout mask.")
-        .AsIntermediate();
-    AddOutput("LnMean", "Mean of the current mini batch.").AsIntermediate();
-    AddOutput("LnVariance", "Variance of the current mini batch.")
-        .AsIntermediate();
-    AddOutput("Y", "Result.");


在forward中确实设置了5个output，但在backword设置了6个output呀_(:з」∠)_

zeroRains · 2023-11-04T06:04:19Z

paddle/fluid/operators/fused/fused_bias_dropout_residual_layer_norm_op.cc

-    if (this->HasInput("Bias")) {
-      op->SetInput("Bias", this->Input("Bias"));
-      op->SetOutput(framework::GradVarName("Bias"), this->InputGrad("Bias"));
-    }
-    if (this->HasInput("LnScale")) {
-      op->SetInput("LnScale", this->Input("LnScale"));
-      op->SetOutput(framework::GradVarName("LnScale"),
-                    this->InputGrad("LnScale"));
-    }
-    if (this->HasInput("LnBias")) {
-      op->SetInput("LnBias", this->Input("LnBias"));
-      op->SetOutput(framework::GradVarName("LnBias"),
-                    this->InputGrad("LnBias"));
-    }
-    if (this->HasOutput("LnMean")) {
-      op->SetInput("LnMean", this->Output("LnMean"));
-    }
-    if (this->HasOutput("LnVariance")) {
-      op->SetInput("LnVariance", this->Output("LnVariance"));
-    }
-    if (this->HasOutput("BiasDropoutResidualOut")) {
-      op->SetInput("BiasDropoutResidualOut",
-                   this->Output("BiasDropoutResidualOut"));
-    }
-    op->SetInput("DropoutMaskOut", this->Output("DropoutMaskOut"));
-    op->SetOutput(framework::GradVarName("X"), this->InputGrad("X"));
-    op->SetOutput(framework::GradVarName("Residual"),
-                  this->InputGrad("Residual"));
-    op->SetOutput(framework::GradVarName("BiasDropoutResidualOut"),
-                  this->OutputGrad("BiasDropoutResidualOut"));


这个backward确实SetOutput6次，因为前3个Output是受可选参数控制的我还把他们加在了optional里，但是仍然报错，(:з」∠)

现在主要问题还是这个地方

zeroRains · 2023-11-04T12:56:01Z

额，请问这个应该怎么转化成phi中的函数参数变量名呢？

op_compat.yaml 放在extra里

额，我不太明白是吧什么放extra里。是这样吗？

paddle-ci-bot · 2023-11-05T03:16:18Z

Sorry to inform you that 8092f50's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

yuanlehome · 2023-11-06T05:37:18Z

额，请问这个应该怎么转化成phi中的函数参数变量名呢？

op_compat.yaml 放在extra里

额，我不太明白是吧什么放extra里。是这样吗？

对的，试一下

zeroRains · 2023-11-06T06:02:20Z

额，请问这个应该怎么转化成phi中的函数参数变量名呢？

op_compat.yaml 放在extra里

额，我不太明白是吧什么放extra里。是这样吗？

对的，试一下

确实可以

yuanlehome · 2023-11-07T06:45:31Z

这个pr可以尽快推进下哈～有工作依赖fc的迁移

zeroRains · 2023-11-07T06:50:58Z

这个pr可以尽快推进下哈～有工作依赖fc的迁移

卡在前向和反向的输出数量上了。。实在看不出来哪里有问题_(:з)∠)_

luotao1 · 2023-11-07T06:54:37Z

先把skip_layernorm和fused_bias_dropout_residual_layer_norm两个交了？

yuanlehome · 2023-11-07T06:54:42Z

这个pr可以尽快推进下哈～有工作依赖fc的迁移

卡在前向和反向的输出数量上了。。实在看不出来哪里有问题_(:з)∠)_

分开，可以先把fc搞了

yuanlehome · 2023-11-07T06:58:53Z

这个pr可以尽快推进下哈～有工作依赖fc的迁移

卡在前向和反向的输出数量上了。。实在看不出来哪里有问题_(:з)∠)_

分开，可以先把fc搞了

fc需要确保开启那个flag也能跑通哈

zeroRains · 2023-11-07T07:05:54Z

这个pr可以尽快推进下哈～有工作依赖fc的迁移

卡在前向和反向的输出数量上了。。实在看不出来哪里有问题_(:з)∠)_

分开，可以先把fc搞了

fc需要确保开启那个flag也能跑通哈

好的，晚上我看看

zeroRains · 2023-11-07T12:29:11Z

这个pr可以尽快推进下哈～有工作依赖fc的迁移

卡在前向和反向的输出数量上了。。实在看不出来哪里有问题_(:з)∠)_

分开，可以先把fc搞了

fc需要确保开启那个flag也能跑通哈

fc的pr在#58777 这里，但是有点小问题，写在fc的pr里了

CLAassistant · 2023-11-07T12:53:36Z

All committers have signed the CLA.

luotao1 · 2023-11-15T09:43:47Z

@zeroRains 可以更新下这个PR了

paddle-ci-bot · 2023-11-16T03:14:28Z

Sorry to inform you that b3ea0d5's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

… 103

zeroRains · 2023-11-16T14:34:09Z

感觉这个有点多余了，我试试把这个参数在backward的参数列表中去掉，然后在backward的kernel中新建一个DenseTensor代替他，这样就能解决反向输出比前向输入多的问题了

zeroRains · 2023-11-16T14:57:24Z

但是遇到了个奇怪的问题。。。这个vjp是啥
@yuanlehome

… 103

zeroRains · 2023-11-22T02:28:49Z

但是遇到了个奇怪的问题。。。这个vjp是啥 @yuanlehome

将算子加入到vjp_interface_black_list解决

zeroRains · 2023-11-23T04:41:14Z

两个算子的单测，均已通过~

@yuanlehome

yuanlehome · 2023-11-23T11:09:21Z

paddle/phi/api/yaml/fused_ops.yaml

+  infer_meta :
+    func : SkipLayerNormInferMeta
+  kernel :
+    func : skip_layer


data_type : x

yuanlehome · 2023-11-23T11:11:02Z

两个算子的单测，均已通过~

@yuanlehome

👍，开启那个flags也是可以通过的吗？

zeroRains · 2023-11-23T11:26:52Z

两个算子的单测，均已通过~
@yuanlehome

👍，开启那个flags也是可以通过的吗？

是的~

zyfncg · 2023-11-23T11:18:39Z

paddle/phi/api/yaml/fused_backward.yaml

+- backward_op : fused_bias_dropout_residual_layer_norm_grad
+  forward: fused_bias_dropout_residual_layer_norm (Tensor x, Tensor residual, Tensor bias, Tensor ln_scale, Tensor ln_bias, float dropout_rate, bool is_test, bool dropout_fix_seed, int dropout_seed, str dropout_implementation, float ln_epsilon) -> Tensor(bias_dropout_residual_out), Tensor(dropout_mask_out), Tensor(ln_mean), Tensor(ln_variance), Tensor(y)
+  args : (Tensor y_grad, Tensor x, Tensor residual, Tensor bias, Tensor ln_scale, Tensor ln_bias, Tensor ln_mean, Tensor ln_variance, Tensor bias_dropout_residual_out, Tensor dropout_mask_out, float dropout_rate = 0.5f, bool is_test = false, bool dropout_fix_seed = true, int dropout_seed = true, str dropout_implementation = "downgrade_in_infer", float ln_epsilon = 1e-5)
+  output : Tensor(bias_grad), Tensor(ln_scale_grad), Tensor(ln_bias_grad), Tensor(x_grad), Tensor(residual_grad)


grad的顺序需要按照forward里的tensor顺序排

zyfncg · 2023-11-23T11:22:17Z

paddle/phi/api/yaml/fused_ops.yaml

+- op : fused_bias_dropout_residual_layer_norm
+  args : (Tensor x, Tensor residual, Tensor bias, Tensor ln_scale, Tensor ln_bias, float dropout_rate = 0.5f, bool is_test = false, bool dropout_fix_seed = true, int dropout_seed = true, str dropout_implementation = "downgrade_in_infer", float ln_epsilon = 1e-5)
+  optional : bias, ln_scale, ln_bias
+  output : Tensor(bias_dropout_residual_out), Tensor(dropout_mask_out), Tensor(ln_mean), Tensor(ln_variance), Tensor(y)


标记为intermediate的tensor放在后面，没有标记intermediate的在前面

zyfncg · 2023-11-23T11:35:27Z

paddle/phi/infermeta/fusion.h

+void SkipLayerNormInferMeta(const MetaTensor& x,
+                            const MetaTensor& y,
+                            const MetaTensor& scale,
+                            const MetaTensor& bias,
+                            const float epsilon,
+                            const int begin_norm_axis,
+                            MetaTensor* out);
+
+void FusedBiasDropoutResidualLnInferMeta(


InferMeta函数按照字母序放置

zyfncg · 2023-11-23T11:43:28Z

paddle/phi/kernels/funcs/skip_layernorm_functor.h

这个SkipLayerNormFunctor看着只有skip_layernorm的kernel在用，直接放到skip_layernorm_kernel.cu中吧

直接复制到skip_layernorm_kernel中会有一些奇怪的匹配问题，得再看看

这个能不能不改呀，o(╥﹏╥)o，好多奇怪的不匹配的bug

我一眼看过去参数匹配没问题啊

不改也行

zyfncg · 2023-11-23T11:45:21Z

paddle/phi/kernels/fusion/gpu/fused_bias_dropout_residual_layer_norm_kernel.cu

+  kernel->OutputAt(1).SetDataType(phi::DataType::UINT8);
+  kernel->OutputAt(2).SetDataType(kernel_key.dtype());
+  kernel->OutputAt(3).SetDataType(kernel_key.dtype());
+  kernel->OutputAt(3).SetDataType(kernel_key.dtype());


和kernel_key dtype相同的就不需要设置了，可以去掉

zyfncg · 2023-11-23T11:45:59Z

paddle/phi/kernels/fusion/gpu/skip_layernorm_kernel.cu

+#include "paddle/phi/core/errors.h"
+#include "paddle/phi/core/kernel_registry.h"
+#include "paddle/phi/core/tensor_utils.h"
+#include "paddle/phi/kernels/funcs/blas/blas.h"


blas.h有使用到吗？

确实没有，已删除

yuanlehome

LGTM

…_layer_norm to phi (PaddlePaddle#58396) * temp commit * move skip_layernorm to phi, but have abug in include skip_layernorm_functor.h * temp save * move skip_layernorm to phi * move fused_bias_dropout_layer_norm to phi * roback move fc to file but have a bug in onednn roback * change the register name * fix the forward input len smaller than backward output len * roback * add in vjp_interface_black_list * fix parse * fix the bug * Update fused_ops.yaml * update

yuanlehome · 2023-11-28T11:14:44Z

@zeroRains 提个PR修复一下吧

zeroRains · 2023-11-28T12:49:39Z

@zeroRains 提个PR修复一下吧

已提交，#59461 ，sorry

zeroRains added 2 commits October 25, 2023 14:24

temp commit

e04c905

move skip_layernorm to phi, but have abug in include skip_layernorm_f…

7c7284f

…unctor.h

paddle-bot bot added the contributor External developers label Oct 26, 2023

zeroRains added 3 commits October 26, 2023 04:22

temp save

3a23af3

move skip_layernorm to phi

8e00268

fix conflict

61c6f47

move fused_bias_dropout_layer_norm to phi

8092f50

luotao1 mentioned this pull request Oct 29, 2023

【PaddlePaddle Hackathon 5th】开源贡献个人挑战赛 #57262

Closed

luotao1 added the PaddlePaddle Hackathon label Nov 3, 2023

luotao1 assigned luotao1 and yuanlehome Nov 3, 2023

zeroRains commented Nov 4, 2023

View reviewed changes

zeroRains changed the title ~~【Hackathon 5th No.103】move skip_layernorm/fc/fused_bias_dropout_residual_layer_norm to phi~~ 【Hackathon 5th No.103】move skip_layernorm/fused_bias_dropout_residual_layer_norm to phi Nov 7, 2023

zeroRains added 3 commits November 16, 2023 13:08

change the register name

8cb3aa2

fix the forward input len smaller than backward output len

adeeb6f

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

f37def5

… 103

zeroRains added 3 commits November 21, 2023 10:52

roback

f75537a

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

9d97f6c

… 103

add in vjp_interface_black_list

8a8a9e4

zeroRains added 2 commits November 22, 2023 02:43

fix parse

ce80dfd

fix the bug

d687f1b

yuanlehome reviewed Nov 23, 2023

View reviewed changes

Update fused_ops.yaml

0bb3a4e

zyfncg reviewed Nov 23, 2023

View reviewed changes

update

df5c9be

yuanlehome approved these changes Nov 24, 2023

View reviewed changes

luotao1 merged commit e57e312 into PaddlePaddle:develop Nov 24, 2023

zeroRains deleted the 103 branch November 24, 2023 14:33

zeroRains restored the 103 branch November 28, 2023 11:59

zeroRains mentioned this pull request Nov 28, 2023

fix the bug in moving skip_layernorm op to phi #59461

Merged

luotao1 changed the title ~~【Hackathon 5th No.103】move skip_layernorm/fused_bias_dropout_residual_layer_norm to phi~~ 【Hackathon 5th No.103】move skip_layernorm/fused_bias_dropout_residual_layer_norm to phi -part Dec 22, 2023

luotao1 changed the title ~~【Hackathon 5th No.103】move skip_layernorm/fused_bias_dropout_residual_layer_norm to phi -part~~ 【Hackathon 5th No.103】move skip_layernorm/fused_bias_dropout_residual_layer_norm to phi Jan 29, 2024

【Hackathon 5th No.103】move skip_layernorm/fused_bias_dropout_residual_layer_norm to phi #58396

【Hackathon 5th No.103】move skip_layernorm/fused_bias_dropout_residual_layer_norm to phi #58396

Conversation

zeroRains commented Oct 26, 2023 • edited Loading

PR types

PR changes

Description

paddle-bot bot commented Oct 26, 2023

zeroRains commented Oct 27, 2023 • edited Loading

zeroRains commented Oct 28, 2023

yuanlehome commented Nov 3, 2023

yuanlehome commented Nov 3, 2023

Choose a reason for hiding this comment

zeroRains Nov 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zeroRains commented Nov 4, 2023

paddle-ci-bot bot commented Nov 5, 2023

yuanlehome commented Nov 6, 2023

zeroRains commented Nov 6, 2023

yuanlehome commented Nov 7, 2023

zeroRains commented Nov 7, 2023 • edited Loading

luotao1 commented Nov 7, 2023

yuanlehome commented Nov 7, 2023

yuanlehome commented Nov 7, 2023

zeroRains commented Nov 7, 2023

zeroRains commented Nov 7, 2023

CLAassistant commented Nov 7, 2023 • edited Loading

luotao1 commented Nov 15, 2023

paddle-ci-bot bot commented Nov 16, 2023

zeroRains commented Nov 16, 2023 • edited Loading

zeroRains commented Nov 16, 2023 • edited Loading

zeroRains commented Nov 22, 2023

zeroRains commented Nov 23, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yuanlehome commented Nov 23, 2023

zeroRains commented Nov 23, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zeroRains Nov 23, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yuanlehome left a comment

Choose a reason for hiding this comment

yuanlehome commented Nov 28, 2023 • edited Loading

zeroRains commented Nov 28, 2023 • edited Loading

zeroRains commented Oct 26, 2023 •

edited

Loading

zeroRains commented Oct 27, 2023 •

edited

Loading

zeroRains Nov 4, 2023 •

edited

Loading

zeroRains commented Nov 7, 2023 •

edited

Loading

CLAassistant commented Nov 7, 2023 •

edited

Loading

zeroRains commented Nov 16, 2023 •

edited

Loading

zeroRains commented Nov 16, 2023 •

edited

Loading

zeroRains Nov 23, 2023 •

edited

Loading

yuanlehome commented Nov 28, 2023 •

edited

Loading

zeroRains commented Nov 28, 2023 •

edited

Loading