[PIR save_load] Params save/load v1.0 #63389
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
| @@ -18,6 +18,7 @@ limitations under the License. */ | |||
| #include "paddle/fluid/framework/lod_tensor.h" | |||
| #include "paddle/fluid/framework/selected_rows_utils.h" | |||
| #include "paddle/fluid/pir/serialize_deserialize/include/interface.h" | |||
| #include "paddle/fluid/pir/serialize_deserialize/include/save_load_parameters.h" | |||
There was a problem hiding this comment.
这个可以考虑都加到interface.h 中，改一下interface 的名称，pir_io.h或者其他
There was a problem hiding this comment.
TODO：下一个pr整合一下。
| elif op.name() == "pd_op.data" and "persistable" in op.attrs(): | ||
| if op.attrs()['persistable'] == [True]: | ||
| name = op.attrs()["name"] | ||
| opts.append(name) |
There was a problem hiding this comment.
这个可以确保opt 的一定是data产生的吗
There was a problem hiding this comment.
目前打印program看到的是这样，记个TODO后面查一下pir下opt代码
|
|
||
| ### TODO(chenzhiyang): save program | ||
|
|
||
|
|
There was a problem hiding this comment.
补充的内容是不是可以考虑新开一个文件，后面直接删除原文件退场
There was a problem hiding this comment.
TODO：和前面整理头文件的内容一起在下个pr整合
|
|
||
| param_dict = {} | ||
| opt_dict = {} | ||
| for op in prog.global_block().ops: |
There was a problem hiding this comment.
这段代码不是有库函数吗，为什么需要再写一次
There was a problem hiding this comment.
这里是未适配api时直接测试C++function时写的测试，需要先获取参数再调用接口，后面可以考虑把获取参数的接口引入进来直接调用，待单测完善后这部分测试可以直接删掉。
| y = linear(x) | ||
|
|
||
| place = base.CPUPlace() | ||
| x_tensor = base.create_lod_tensor( |
There was a problem hiding this comment.
最好不使用pir 下不推荐使用的api ,增加退场负担
There was a problem hiding this comment.
同上，后面会考虑直接删除这部分单测
PR Category
Others
PR Types
New features
Description
pcard-67164
本PR实现了PIR下参数save load初始版本，并适配了部分API接口，具体细节如下：
C++侧
PIR下C++侧参数存储不再迁移save(combine) op 和 load(combine) op，新增函数
SaveFunctionSaveCombineFunctionLoadFunctionLoadCombineFunction来实现对应的kernel功能，新增文件save_load_parameters.h和save_load_parameters.cc，放置在PIR save_load 根目录Paddle/paddle/fluid/pir/serialize_deserialize下。新增的函数通过pybind提供给Python端调用。Python侧
迁移适配了部分API功能，包括
paddle.static.save和paddle.static.load中的参数存储加载部分，并调用新增的C++函数迁移适配了paddle.static.save_vars和paddle.static.load_vars，新增单测文件进行验证。TODO：
paddle.static.load和paddle.static.load_vars功能进一步完善。