【SCU】【Paddle Tensor 第二期 API支持 0-size Tensor】（paddle.sum、paddle.mean、paddle.prod、 paddle.var、 paddle.std）支持 0-size tensor #70146

mori0umi · 2024-12-11T11:10:59Z

PR Category

User Experience

PR Types

Bug fixes

Description

解决 sum, mean, prod, var, std 五个 reduce_op 输入 Tensor 为 0-size Tensor 时出现的报错。

paddle-bot · 2024-12-11T11:11:04Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

… var api

HydrogenSulfate · 2024-12-12T02:50:45Z

paddle/phi/kernels/funcs/reduce_function.h

+  }
+};
+
+// Prod操作的特化


中文注释删掉

HydrogenSulfate · 2024-12-12T02:52:17Z

paddle/phi/kernels/cpu/elementwise_subtract_kernel.cc

  dev_ctx.template Alloc<T>(out);
+


多余换行删除

HydrogenSulfate · 2024-12-12T02:53:01Z

paddle/phi/kernels/funcs/reduce_function.h

+// Sum操作的特化
+template <typename Context, typename T, typename OutT>
+struct SpecialCaseReduceOp<phi::funcs::SumFunctor, Context, T, OutT> {
+  static void HandleEmptyTensor(const Context& dev_ctx,
+                                const phi::DenseTensor& input,
+                                phi::DenseTensor* output) {
+    auto out = dev_ctx.template Alloc<OutT>(output);
+    *out = static_cast<OutT>(0);
+  }
+};
+


对于Sum的处理，是不是改这里就行了，

确实，我感觉是的，当时没有想到，我修改一下，感谢reveiw！另外请问GPU部分的实现是不是也是在phi/kps/reduce_kernel.cu里面改比较合适？

确实，我感觉是的，当时没有想到，我修改一下，感谢reveiw！另外请问GPU部分的实现是不是也是在phi/kps/reduce_kernel.cu里面改比较合适？

好像是的，改SumRawKernel更合适，因为SumKernel只是一个转发的

HydrogenSulfate · 2024-12-12T02:53:17Z

paddle/phi/kernels/funcs/reduce_function.h

+
+// Mean操作的特化
+template <typename Context, typename T, typename OutT>
+struct SpecialCaseReduceOp<phi::funcs::MeanFunctor, Context, T, OutT> {


HydrogenSulfate · 2024-12-12T02:53:37Z

paddle/phi/kernels/funcs/reduce_function.h

+// Prod操作的特化
+template <typename Context, typename T, typename OutT>
+struct SpecialCaseReduceOp<phi::funcs::ProdFunctor, Context, T, OutT> {
+  static void HandleEmptyTensor(const Context& dev_ctx,


Empty同理

HydrogenSulfate · 2024-12-12T02:54:42Z

python/paddle/tensor/stat.py

@@ -201,7 +201,7 @@ def var(
    )
    n = n.astype(dtype)
    if unbiased:
-        one_const = paddle.ones([], x.dtype)


这里有必要加上to吗？paddle默认创建的Tensor都在GPU上

如果是cpu创建的数据过一下var就被放在gpu上了，感觉不太符合逻辑

如果是cpu创建的数据过一下var就被放在gpu上了，感觉不太符合逻辑

这个是Paddle设备切换的逻辑问题，在 #68635 中尝试修复过，跟其他同事讨论的结果是，Paddle算子运行设备由当前全局设备控制，而不是算子输入的设备控制，所以这里不用管这个问题（有一个非公开的设备上下文管理器，但是不可能给每个API都加上）。这块跟pytorch存在一定差异

明白了，下个commit去掉，感谢解答！

HydrogenSulfate

输出Tensor的形状需要考虑各种情况

HydrogenSulfate · 2024-12-13T03:10:18Z

paddle/phi/kernels/cpu/elementwise_subtract_kernel.cc

+  if (x.numel() == 0 && y.numel() != 0) {
+    out->Resize(y.dims());
+    dev_ctx.template Alloc<T>(out);
+    ActivationImpl<T, T, Context, phi::funcs::NegativeFunctor<T>>(
+        dev_ctx, y, out, phi::funcs::NegativeFunctor<T>());
+    return;
+  }
+  if (y.numel() == 0 && x.numel() != 0) {
+    out->Resize(x.dims());
+    dev_ctx.template Alloc<T>(out);
+    phi::Copy(dev_ctx, x, dev_ctx.GetPlace(), false, out);
+    return;
+  }


是不是缺少当x和y都为0-size的逻辑？

我想了一下，这里做特判好像本身就不合适。因为 numpy 和 torch 的表现是即使有一方是 0-size ，也要先处理广播的逻辑。比如（0，3）和（2，）的 tensor 相减， numpy 报错是 ValueError: operands could not be broadcast together with shapes (0,3) (2,) 。

我想了一下，这里做特判好像本身就不合适。因为 numpy 和 torch 的表现是即使有一方是 0-size ，也要先处理广播的逻辑。比如（0，3）和（2，）的 tensor 相减， numpy 报错是 ValueError: operands could not be broadcast together with shapes (0,3) (2,) 。

对的，广播是有条件限制的，并不是任意两种形状都能广播

HydrogenSulfate · 2024-12-13T03:11:54Z

paddle/phi/kernels/cpu/prod_kernel.cc

+  if (x.numel() == 0) {
+    DataType dtype = phi::CppTypeToDataType<T>::Type();
+    int value = 1;
+    FullKernel<T, Context>(
+        dev_ctx, std::vector<int64_t>({}), value, dtype, out);
+    return;
+  }


这里可能有问题，prod是允许在某个维度上进行reduce，而不是全部reduce的，即输出的形状需要考虑dims

HydrogenSulfate · 2024-12-13T03:12:26Z

paddle/phi/kernels/cpu/reduce_mean_kernel.cc

+  if (x.numel() == 0) {
+    DataType dtype = phi::CppTypeToDataType<T>::Type();
+    double nanValue = std::numeric_limits<double>::quiet_NaN();
+    FullKernel<T, Context>(
+        dev_ctx, std::vector<int64_t>({}), nanValue, dtype, out);
+    return;
+  }


同理，输出形状需要考虑dims和keep_dim以及reduce_all三个变量

HydrogenSulfate · 2024-12-13T03:12:34Z

paddle/phi/kernels/cpu/reduce_sum_kernel.cc

+  if (x.numel() == 0) {
+    DataType dtype = phi::CppTypeToDataType<T>::Type();
+    int value = 0;
+    FullKernel<T, Context>(
+        dev_ctx, std::vector<int64_t>({}), value, dtype, out);
+    return;
+  }


HydrogenSulfate · 2024-12-13T03:12:57Z

paddle/phi/kernels/kps/elementwise_kernel.cu

+  if (x.numel() == 0 && y.numel() != 0) {
+    out->Resize(y.dims());
+    dev_ctx.template Alloc<T>(out);
+    ActivationImpl<T, T, Context, phi::funcs::NegativeFunctor<T>>(
+        dev_ctx, y, out, phi::funcs::NegativeFunctor<T>());
+    return;
+  }
+  if (y.numel() == 0 && x.numel() != 0) {
+    out->Resize(x.dims());
+    dev_ctx.template Alloc<T>(out);
+    phi::Copy(dev_ctx, x, dev_ctx.GetPlace(), false, out);
+    return;
+  }


同理，0-size减去0-size会如何

HydrogenSulfate

以及添加形状足够丰富的单测来验证代码的正确性

HydrogenSulfate · 2024-12-19T03:33:31Z

@mori0umi 好像有编译上的问题，

paddle-ci-bot · 2024-12-20T03:10:12Z

Sorry to inform you that cf92bb1's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

mori0umi added 2 commits December 11, 2024 11:01

add 0-size support for sum & prod & mean

20d6974

fix codestyle

1197496

paddle-bot bot added the contributor External developers label Dec 11, 2024

modify subtract kernel to support var & std reduce op, and fix bug of…

d138888

… var api

luotao1 added the HappyOpenSource Pro 进阶版快乐开源活动，更具挑战性的任务 label Dec 12, 2024

luotao1 assigned luotao1 and HydrogenSulfate Dec 12, 2024

HydrogenSulfate reviewed Dec 12, 2024

View reviewed changes

mori0umi added 4 commits December 12, 2024 06:50

add support for xpu and fix bug

839a986

fix bug

03a92c5

fix bug

0def592

fix codestyle

cf92bb1

HydrogenSulfate reviewed Dec 13, 2024

View reviewed changes

mori0umi mentioned this pull request Dec 15, 2024

[WeeklyReports] 2024.11.29~2024.12.13周报收集 PFCCLab/Starter#463

Open

74 tasks

PolaKuma mentioned this pull request Dec 16, 2024

【SCU】【Paddle Tensor 第二期 API 支持 0-size TensorNo.25、47】paddle.tensordot 支持 0-size Tensor #70238

Merged

luotao1 closed this Jan 8, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【SCU】【Paddle Tensor 第二期 API支持 0-size Tensor】（paddle.sum、paddle.mean、paddle.prod、 paddle.var、 paddle.std）支持 0-size tensor #70146

【SCU】【Paddle Tensor 第二期 API支持 0-size Tensor】（paddle.sum、paddle.mean、paddle.prod、 paddle.var、 paddle.std）支持 0-size tensor #70146

mori0umi commented Dec 11, 2024 •

edited

Loading

paddle-bot bot commented Dec 11, 2024

HydrogenSulfate Dec 12, 2024

HydrogenSulfate Dec 12, 2024

HydrogenSulfate Dec 12, 2024

mori0umi Dec 12, 2024

HydrogenSulfate Dec 12, 2024

HydrogenSulfate Dec 12, 2024

HydrogenSulfate Dec 12, 2024

HydrogenSulfate Dec 12, 2024

mori0umi Dec 12, 2024

HydrogenSulfate Dec 12, 2024 •

edited

Loading

mori0umi Dec 12, 2024

HydrogenSulfate left a comment

HydrogenSulfate Dec 13, 2024

mori0umi Dec 13, 2024

HydrogenSulfate Dec 13, 2024

HydrogenSulfate Dec 13, 2024

HydrogenSulfate Dec 13, 2024

HydrogenSulfate Dec 13, 2024

HydrogenSulfate Dec 13, 2024

HydrogenSulfate left a comment

HydrogenSulfate commented Dec 19, 2024

paddle-ci-bot bot commented Dec 20, 2024

【SCU】【Paddle Tensor 第二期 API支持 0-size Tensor】（paddle.sum、paddle.mean、paddle.prod、 paddle.var、 paddle.std）支持 0-size tensor #70146

【SCU】【Paddle Tensor 第二期 API支持 0-size Tensor】（paddle.sum、paddle.mean、paddle.prod、 paddle.var、 paddle.std）支持 0-size tensor #70146

Conversation

mori0umi commented Dec 11, 2024 • edited Loading

PR Category

PR Types

Description

paddle-bot bot commented Dec 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HydrogenSulfate Dec 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HydrogenSulfate left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HydrogenSulfate left a comment

Choose a reason for hiding this comment

HydrogenSulfate commented Dec 19, 2024

paddle-ci-bot bot commented Dec 20, 2024

mori0umi commented Dec 11, 2024 •

edited

Loading

HydrogenSulfate Dec 12, 2024 •

edited

Loading