-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
【Paddle Tensor 第二期 复数类型支持问题No.9、10】为paddle.not_equal
、paddle.equal
添加复数类型支持。
#69968
【Paddle Tensor 第二期 复数类型支持问题No.9、10】为paddle.not_equal
、paddle.equal
添加复数类型支持。
#69968
Conversation
你的PR提交成功,感谢你对开源项目的贡献! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
…_equal` and `equal`
compare_kernel
添加复数类型支持。not_equal
、paddle.equal
添加复数类型支持。
not_equal
、paddle.equal
添加复数类型支持。paddle.not_equal
、paddle.equal
添加复数类型支持。
paddle.not_equal
、paddle.equal
添加复数类型支持。paddle.not_equal
、paddle.equal
添加复数类型支持。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
除了模板合并之外,其他感觉没问题了
template <typename OutT> | ||
struct EqualFunctor<phi::dtype::complex<float>, OutT> { | ||
HOSTDEVICE OutT operator()(const phi::dtype::complex<float>& a, | ||
const phi::dtype::complex<float>& b) const { | ||
if (isinf(a.real) || isinf(a.imag) || isinf(b.real) || isinf(b.imag)) { | ||
return a == b; | ||
} | ||
if (isnan(a.real) || isnan(a.imag) || isnan(b.real) || isnan(b.imag)) { | ||
return false; | ||
} | ||
float epsilon = 1e-8f; | ||
return std::abs(a.real - b.real) < epsilon && | ||
std::abs(a.imag - b.imag) < epsilon; | ||
} | ||
}; | ||
|
||
template <typename OutT> | ||
struct EqualFunctor<phi::dtype::complex<double>, OutT> { | ||
HOSTDEVICE OutT operator()(const phi::dtype::complex<double>& a, | ||
const phi::dtype::complex<double>& b) const { | ||
if (isinf(a.real) || isinf(a.imag) || isinf(b.real) || isinf(b.imag)) { | ||
return a == b; | ||
} | ||
if (isnan(a.real) || isnan(a.imag) || isnan(b.real) || isnan(b.imag)) { | ||
return false; | ||
} | ||
double epsilon = 1e-8; | ||
return std::abs(a.real - b.real) < epsilon && | ||
std::abs(a.imag - b.imag) < epsilon; | ||
} | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这俩能合并为以下代码吗(可以参考: Paddle/paddle/phi/kernels/funcs/matrix_inverse.h
)?
template <typename OutT> | |
struct EqualFunctor<phi::dtype::complex<float>, OutT> { | |
HOSTDEVICE OutT operator()(const phi::dtype::complex<float>& a, | |
const phi::dtype::complex<float>& b) const { | |
if (isinf(a.real) || isinf(a.imag) || isinf(b.real) || isinf(b.imag)) { | |
return a == b; | |
} | |
if (isnan(a.real) || isnan(a.imag) || isnan(b.real) || isnan(b.imag)) { | |
return false; | |
} | |
float epsilon = 1e-8f; | |
return std::abs(a.real - b.real) < epsilon && | |
std::abs(a.imag - b.imag) < epsilon; | |
} | |
}; | |
template <typename OutT> | |
struct EqualFunctor<phi::dtype::complex<double>, OutT> { | |
HOSTDEVICE OutT operator()(const phi::dtype::complex<double>& a, | |
const phi::dtype::complex<double>& b) const { | |
if (isinf(a.real) || isinf(a.imag) || isinf(b.real) || isinf(b.imag)) { | |
return a == b; | |
} | |
if (isnan(a.real) || isnan(a.imag) || isnan(b.real) || isnan(b.imag)) { | |
return false; | |
} | |
double epsilon = 1e-8; | |
return std::abs(a.real - b.real) < epsilon && | |
std::abs(a.imag - b.imag) < epsilon; | |
} | |
}; | |
template <typename InT, typename OutT> | |
struct EqualFunctor<phi::dtype::complex<InT>, OutT> { | |
HOSTDEVICE OutT operator()(const phi::dtype::complex<InT>& a, | |
const phi::dtype::complex<InT>& b) const { | |
if (isinf(a.real) || isinf(a.imag) || isinf(b.real) || isinf(b.imag)) { | |
return a == b; | |
} | |
if (isnan(a.real) || isnan(a.imag) || isnan(b.real) || isnan(b.imag)) { | |
return false; | |
} | |
InT epsilon = 1e-8; | |
return std::abs(a.real - b.real) < epsilon && | |
std::abs(a.imag - b.imag) < epsilon; | |
} | |
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
原来还能这么用!
@MrXnneHang 包的编译好像有点问题,可以本地测试下编译和安装 |
我正在给我家里的电脑配环境,我笔记本一直都是编译cpu版本... |
struct EqualFunctor<phi::dtype::complex<T>, OutT> { | ||
HOSTDEVICE OutT operator()(const phi::dtype::complex<float>& a, | ||
const phi::dtype::complex<float>& b) const { | ||
if (isinf(a.real) || isinf(a.imag) || isinf(b.real) || isinf(b.imag)) { | ||
return a == b; | ||
} | ||
if (isnan(a.real) || isnan(a.imag) || isnan(b.real) || isnan(b.imag)) { | ||
return false; | ||
} | ||
float epsilon = 1e-8f; | ||
return std::abs(a.real - b.real) < epsilon && | ||
std::abs(a.imag - b.imag) < epsilon; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
我对这里的isnan有疑惑,cpu可以用std::isnan,而GPU应该使用isnan,这两者能通用吗?参考代码:Paddle/paddle/phi/kernels/impl/isfinite_kernel_impl.h
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
可能isnan
通用而std::isnan
只能用于cpu?
我注意到原本的EqualFunctor
在cpu和gpu上的注册方式完全一致。并且也只采用了isnan
。
我编译gpu版本的时候都换成std::isnan
测试下看看
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
目前我全部使用isnan。
但是我找不到equal
在哪里注册的<GPU,AllLayout,int,float....>那些。
我注意到我在gpu版本中,对于常数的==
是可以正常运行的,但是如果对于复数,会报错在未注册GPU上complex64
,complex128
而且我注意到似乎gpu版本的常数==
不会进入EqualFunctor主模板或者特化模板中。
参考:#69968 (comment)
好的 |
bf5ac03
to
6af2648
Compare
7be9d62
to
c5413b0
Compare
b071f4e
to
8eb19ee
Compare
template <typename T> | ||
struct EqualFunctor<phi::dtype::complex<T>> { | ||
HOSTDEVICE bool operator()(const phi::dtype::complex<T>& a, | ||
const phi::dtype::complex<T>& b) const { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
特化模板应该期望得到引用或者是一个指向变量的指针。
而我五天前因为包编译错误把引用给去了。导致期望得到引用或者变量指针
匹配不到传入临时变量类型
,这很大概率是CI一直C++ Coverage为0的原因。
ba8cbe8
如果直接引用,会产生这个冲突:reinterpret_cast casts away qualifiers
我尝试用static_cast来转,但是引用输入是不期望被修改的。编译会出错。
不能在operater中引用变量->不能使用特化模板->合并模板。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
让我疑惑的是,我在本地printf
可以打印出特化模板的内容。
可能的解释是下午我加入printf
后出现的static check
的警告:
You are using GPU version Paddle, but your CUDA device is not set properly. CPU device will be used by default.
加入printf
后,GPU陷入一种薛定谔状态,它也不知道自己是CPU还是GPU。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
让我疑惑的是,我在本地
printf
可以打印出特化模板的内容。 可能的解释是下午我加入printf
后出现的static check
的警告:You are using GPU version Paddle, but your CUDA device is not set properly. CPU device will be used by default.
加入
printf
后,GPU陷入一种薛定谔状态,它也不知道自己是CPU还是GPU。
Coverage的覆盖率测试可能是存在问题的,如果仅仅是coverage没通过,而本地自测能够打印出coverage中报告未覆盖(红色)的内容,就先不用管。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
似乎确实是CI的问题
不过也确定了引用类型可以匹配上变量类型。
} | ||
if (isinf(static_cast<float>(a)) || isinf(static_cast<float>(b))) { | ||
return static_cast<OutT>(a == b); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里我把泛化模板的isnan
的检查提前到isinf
之前。
这样子在碰到inf==nan
的情况不用多算一次a==b
。结果是一样的。逻辑是只要碰到nan直接抛出false
@MrXnneHang 覆盖率没过 |
我本地编译和测试的。 |
即使没有用特化模板它的覆盖率也是0,这是上次合并模板后的CI结果 |
好的 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
有一个问题可以看一下
return static_cast<OutT>(a == b); | ||
if (isnan(static_cast<float>(a)) || isnan(static_cast<float>(b))) | ||
if (isnan(static_cast<float>(a)) || isnan(static_cast<float>(b))) { | ||
return static_cast<OutT>(false); | ||
} | ||
if (isinf(static_cast<float>(a)) || isinf(static_cast<float>(b))) { | ||
return static_cast<OutT>(a == b); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
std::is_floating_point分支下,为什么要再转成float呢?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/* IsnanFunctor */
template <typename T>
__global__ void IsnanCUDAKernel(
const T* in_data,
int num,
bool* out_data,
typename std::enable_if<std::is_floating_point<T>::value>::type* = 0) {
unsigned int idx = threadIdx.x + blockIdx.x * blockDim.x;
for (int i = idx; i < num; i += blockDim.x * gridDim.x) {
const T& a = in_data[i];
out_data[i] = isnan(a);
}
}
可能是处于并发检查的性能和内存考虑的?
因为对于每个数都要检查一遍,如果是double的话,长度是float的两倍。而且如果都是相同的数据类型的话对于GPU本身指令特化的结构也会更有利一些。
而且isnan
和isinf
的话确实也没必要保持double。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
我去改成float
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/* IsnanFunctor */ template <typename T> __global__ void IsnanCUDAKernel( const T* in_data, int num, bool* out_data, typename std::enable_if<std::is_floating_point<T>::value>::type* = 0) { unsigned int idx = threadIdx.x + blockIdx.x * blockDim.x; for (int i = idx; i < num; i += blockDim.x * gridDim.x) { const T& a = in_data[i]; out_data[i] = isnan(a); } }可能是处于并发检查的性能和内存考虑的? 因为对于每个数都要检查一遍,如果是double的话,长度是float的两倍。而且如果都是相同的数据类型的话对于GPU本身指令特化的结构也会更有利一些。 而且
isnan
和isinf
的话确实也没必要保持double。
这会不会导致原本double的数被转为float后结果就不正确了呢
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
哦如果原本是这样写的话,那先保持原状吧
template <typename T> | ||
struct NotEqualFunctor<phi::dtype::complex<T>> { | ||
HOSTDEVICE bool operator()(const phi::dtype::complex<T> a, | ||
const phi::dtype::complex<T> b) const { | ||
return !EqualFunctor<phi::dtype::complex<T>>()(a, b); | ||
} | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里能不改动原本的模板参数吗?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个我需要测试一下,我记得NotEqual没有特化也是正常工作的。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
删除掉NotEqual特化模板没有影响,依然可以进Equal特化和泛化。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR Category
User Experience
PR Types
Others
Description
为
compare kernel
添加复数支持:偏特化
EqualFunctor
加入对<phi::dtype::complex<double>
和<phi::dtype::complex<float>
的计算逻辑。error 1 (solved)
TypeError: (InvalidType) Type promotion only support calculations between floating-point numbers and between complex and real numbers. But got different data type x: int16, y: int8. (at /paddle/paddle/phi/common/type_promotion.h:229
这个问题是不支持int的自动类型提升。
https://github.com/HydrogenSulfate/array-api-tests/blob/paddle/array_api_tests/test_operators_and_elementwise_functions.py#1119
我把这一行换到手动类型转换之后就
pass
了。error 2 (solved)
问题二似乎和提前的截断有关。测试里有个bug。
https://github.com/HydrogenSulfate/array-api-tests/blob/paddle/array_api_tests/test_operators_and_elementwise_functions.py#L1117
手动类型转换前
left
和right
就把data截断了。后续对left
和right
做类型转换也太晚了。应该利用
promoted_dtype
进行初始化而不是类型转换。这样子似乎可以解决:
涉及手动类型提升的测试似乎都应该这么改。
error 3 (solved) :
Paddle
Tensor == Tensor
,Tensor != Tensor
结果的shape和equal,not_equal不同。手动调用Tensor.equal: