[PTen] Add variable transform to/from ptenTensor and add cast kernel #36916

MingMingShangTian · 2021-11-01T13:26:03Z

PR types

New features

PR changes

Others

Describe

本PR 主要改动点如下：

重构了Pten 的Cast Kernel，并完成了原Op Kernel的替换。
新增了PD_VISIT_ALL_TYPES 宏定义，用于匹配对应类型，执行自定的操作。
DenseTensor 的Mutable_data 支持在data_type 未定义情况下，根据特化类型设置data_type的操作。
新增了在原框架中Pten kernel执行完成后，回写到原Variable的逻辑。
修复了原pten中以下一些bug.

Pten tensor 写回到原Variable时，部分场景下dtype 没设置的bug
mutable_data对 request byte 为0时，不执行Malloc的bug
reshape kernel的Inplace 操作导致xshape类型错误的bug
生成pten tensor 的place和原来Variable的place 不一致的bug
ReMakePtenDenseTensor 函数中out的place和原来Variable的place 不一致的bug
Revert 了之前pten tensor 的resize 引入的lod 设置的改动
ResetHolderAndType因为reset holder 时会执行size check ，在新的type比原来小的情况下失败的bug

paddle-bot-old · 2021-11-01T13:26:06Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

Fix cast kernel refactor bugs

chenwhql · 2021-11-18T08:03:16Z

paddle/fluid/framework/tensor.cc

@@ -209,5 +209,7 @@ void Tensor::ResetHolderWithType(std::shared_ptr<memory::Allocation> holder,
  type_ = type;
 }

+void Tensor::setType(const proto::VarType::Type type) { type_ = type; }


setType -> set_type

chenwhql · 2021-11-18T08:08:41Z

paddle/fluid/operators/reshape_op.cc

@@ -552,14 +552,13 @@ class Reshape2Op : public ReshapeOp {
      const framework::ExecutionContext &ctx) const override {
    auto multi_inputs = ctx.MultiInput<framework::Tensor>("ShapeTensor");
    if (multi_inputs.size() > 0) {
-      return framework::KernelSignature(
-          "reshape2.mulhost.mid", {"X", "ShapeTensor"}, {}, {"XShape", "Out"});
+      return framework::KernelSignature("reshape2.mulhost",


为什么XShape输出可以删除？这里mul和host后缀最好拆分一下

和负责的rd 沟通过，先保持现状

chenwhql · 2021-11-18T08:09:05Z

paddle/pten/api/include/manipulation.h

@@ -21,6 +21,8 @@ namespace experimental {

 PD_DLL_DECL Tensor flatten(const Tensor& x, int start_axis, int stop_axis);

+Tensor cast(const Tensor& x, DataType out_dtype);


需要增加PD_DLL_DECL声明

chenwhql · 2021-11-18T08:12:41Z

paddle/pten/common/data_type.h

+#define PTEN_PRIVATE_CASE_TYPE(NAME, enum_type, type, ...) \
+  PTEN_PRIVATE_CASE_TYPE_USING_HINT(NAME, enum_type, type, data_t, __VA_ARGS__)
+
+#define PTEN_DISPATCH_ALL_TYPES(TYPE, NAME, ...)                              \


这里建议直接复用ext/dispatch.h中的实现，或者在ext/dispatch.h新增宏

chenwhql · 2021-11-18T08:14:02Z

paddle/pten/infermeta/unary.cc

@@ -74,6 +74,12 @@ DenseTensorMeta FlattenInferShape(const DenseTensorMeta& x_meta,
  return return_meta;
 }

+DenseTensorMeta CastInferShape(const DenseTensorMeta& x_meta,


建议直接命名为InferMeta吧，后面也需要统一改

YuanRisheng · 2021-11-18T08:25:27Z

paddle/fluid/framework/operator.cc

+    size_t start_idx =
+        (i == 0 ? 0 : pt_kernel_context_->InputRangeAt(i - 1).second);
+    size_t end_idx = start_idx + ins_vector.size();
+
+    if (pt_kernel_context_->InputsSize() == start_idx) {
      paddle::SmallVector<std::shared_ptr<pten::TensorBase>> tmp_inputs;
      for (auto* var : ins_vector) {
        tmp_inputs.emplace_back(
            experimental::MakePtenTensorBaseFromVar(*var, in_def));
      }
      pt_kernel_context_->EmplaceBackInputs(std::move(tmp_inputs));
-    } else {
+    } else if (pt_kernel_context_->InputsSize() > start_idx) {
      size_t input_size = pt_kernel_context_->InputsSize();
      for (size_t j = 0; j < ins_vector.size(); ++j) {
-        if (input_size > i + j) {
+        if (input_size > start_idx + j) {
          experimental::ReMakePtenDenseTensorFromVar(
              *ins_vector[j], in_def,
-              pt_kernel_context_->MutableInputAt<pten::DenseTensor>(i + j));
+              pt_kernel_context_->MutableInputAt<pten::DenseTensor>(start_idx +
+                                                                    j));
+        } else {
+          pt_kernel_context_->EmplaceBackInputWithoutSetRange(
+              experimental::MakePtenTensorBaseFromVar(*ins_vector[j], in_def));
        }
-        // TODO(chenweihang): adapt multi-input case later
      }
      pt_kernel_context_->MutableInputRangeAt(i) =


这部分逻辑建议加上详细注释，方便代码阅读理解

已经添加注释

YuanRisheng · 2021-11-18T08:27:49Z

paddle/pten/api/lib/manipulation.cc

@@ -60,6 +60,40 @@ PD_DLL_DECL Tensor flatten(const Tensor& x, int start_axis, int stop_axis) {
  return out;
 }

+Tensor cast(const Tensor& x, DataType out_dtype) {


需要增加PD_DLL_DECL声明

paddle/pten/kernels/cpu/manipulation.cc

zyfncg · 2021-11-18T08:35:16Z

paddle/fluid/framework/operator.cc

+          "error start index when trying to set new tensor to inputs, start "
+          "index is `%d`, but current pt_kernel_context_.inputs.size() is "
+          "`%d` ",


报错信息首字母大写，结尾加.

zyfncg · 2021-11-18T08:37:18Z

paddle/fluid/framework/operator.cc

+          "error start index when trying to set new tensor to inputs, start "
+          "index is `%d`, but current pt_kernel_context_.outputs.size() is "
+          "`%d` ",


zyfncg · 2021-11-18T08:41:44Z

paddle/fluid/imperative/prepared_operator.cc

+          "error start index when trying to set new tensor to inputs, start "
+          "index is `%d`, but current pt_kernel_context_.inputs.size() is "
+          "`%d` ",


报错信息首字母大写，结尾加.

zyfncg · 2021-11-18T08:42:16Z

paddle/fluid/imperative/prepared_operator.cc

+          "error start index when trying to set new tensor to inputs, start "
+          "index is `%d`, but current pt_kernel_context_.outputs.size() is "
+          "`%d` ",


报错信息首字母大写，结尾加.

zyfncg · 2021-11-18T10:49:32Z

paddle/pten/api/lib/utils/tensor_utils.cc

+    auto* tensor = variable->GetMutable<framework::SelectedRows>();
+    auto dtype = pten::TransToProtoVarType(src->dtype());
+
+    if (tensor->value().IsInitialized()) {


if内部没有处理逻辑是否可以和else合并？

已经合并

zyfncg · 2021-11-18T10:56:13Z

paddle/pten/kernels/cuda/manipulation.cu

-  ReshapeFromVectorValImpl(dev_ctx, x, shape, out, false);
+  auto out_meta = InferShapeFromVecValue(x.meta(), shape);
+  if (&x == out) {
+    LOG(INFO) << "out_meta dims:" << out_meta.dims;


LOG(INFO）是调试加的代码吗？

chenwhql · 2021-11-18T11:17:59Z

paddle/fluid/framework/tensor.cc

@@ -209,5 +209,7 @@ void Tensor::ResetHolderWithType(std::shared_ptr<memory::Allocation> holder,
  type_ = type;
 }

+void Tensor::set_type(const proto::VarType::Type type) { type_ = type; }


const proto::VarType::Type -> const proto::VarType::Type&

chenwhql

LGTM

chenwhql · 2021-11-22T02:06:51Z

paddle/fluid/framework/operator.cc

@@ -1901,5 +1961,26 @@ void OperatorWithKernel::BuildPtenKernelContext(
  }
 }

+void OperatorWithKernel::WriteBackToOutputs(RuntimeContext* ctx) const {
+  // auto& input_names = std::get<0>(pt_kernel_signature_->args);


如果是无用的注释，建议在下个PR移除

add cast kernel

0ee0d15

MingMingShangTian and others added 28 commits November 1, 2021 13:39

add cast cuda kernel

1ae3fe2

Merge branch 'develop' into cast_kernel

dc81e06

add cast kernel

7cd7966

merge develop branch

a244a0f

make cast kernel output dtype undefined

0eaf913

Merge branch 'develop' into cast_kernel

dc9b42a

get cast dtype from vardesc

83415da

merge develop branch

62cfb21

move cast to manipulation and add test case

b6c3c05

add castinfershape

17913da

avoid reinitilaze variable

6fbd94d

InitializeVariable support datatype

70d4069

merge develop branch

c434610

merge develop branch

a5c234e

fix merge bug

821b6e0

revert modify initializeVariable

4538134

revert modify on InitializeVariable

bbc83c4

revert modify on InitializeVariable

1feb022

mutable support reset dtype

6d55883

enable make pten tensor from variable when def_arg.type is undefined

7eea4dd

fix build pten ctx start_idx error

51dc272

copy pten out tensor to variable

a927e6f

merge develop branch

f893438

merge develop branch

9ba67db

fix non pten kernel cast failed

d8ce4c3

add reset allocation place for remake tensor

dce29b1

Merge pull request #1 from chenwhql/tianyu_cast_kernel

b567179

Fix cast kernel refactor bugs

fix inplace realloc error

4b70d76

fix ReMakePtenDenseTensor place mismatch bug

6199455

chenwhql reviewed Nov 18, 2021

View reviewed changes

Shixiaowei02 and others added 2 commits November 18, 2021 08:37

reverts the set_lod interface, test=develop

c469ffd

Merge branch 'xiaowei fix code' into cast_kernel

6d712d0

YuanRisheng reviewed Nov 18, 2021

View reviewed changes

MingMingShangTian added 2 commits November 18, 2021 08:58

Merge branch 'develop' into cast_kernel

539ce20

modify by the review options

30e057a

MingMingShangTian changed the title ~~[PTen]add cast kernel~~ [PTen] Add variable transform to/from ptenTensor and add cast kernel Nov 18, 2021

zyfncg reviewed Nov 18, 2021

View reviewed changes

modify error message

8ca880b

chenwhql reviewed Nov 18, 2021

View reviewed changes

add & for const input arguments

55f7cb6

PaddlePaddle locked and limited conversation to collaborators Nov 18, 2021

PaddlePaddle unlocked this conversation Nov 18, 2021

YuanRisheng and others added 8 commits November 18, 2021 11:59

add reference in params

d8873ff

elementwise_sub add mutable_data

7860e78

fix ResetHolderWithType check size bug

9df1dc4

Merge branch 'develop' into cast_kernel

37a805d

add dependence pten_tensor to test_cast_api object

9a49c51

merge develop branch to fix conflict

9dbfbee

remove unused code to pass ci coverage

2a27ce3

merge pr:37293 Fix bug of paddle::experimental::MovesStorageOD

68aa002

PaddlePaddle locked and limited conversation to collaborators Nov 19, 2021

PaddlePaddle unlocked this conversation Nov 19, 2021

Merge branch 'develop' into cast_kernel

a112c6d

chenwhql approved these changes Nov 22, 2021

View reviewed changes

raindrops2sea approved these changes Nov 22, 2021

View reviewed changes

MingMingShangTian merged commit 5caa6fc into PaddlePaddle:develop Nov 22, 2021

MingMingShangTian deleted the cast_kernel branch November 22, 2021 06:26

zhiqiu mentioned this pull request Dec 9, 2021

revert flags_benchmark #38005

Merged

		@@ -21,6 +21,8 @@ namespace experimental {

		PD_DLL_DECL Tensor flatten(const Tensor& x, int start_axis, int stop_axis);

		Tensor cast(const Tensor& x, DataType out_dtype);

[PTen] Add variable transform to/from ptenTensor and add cast kernel #36916

[PTen] Add variable transform to/from ptenTensor and add cast kernel #36916

Uh oh!

Conversation

MingMingShangTian commented Nov 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR types

PR changes

Describe

Uh oh!

paddle-bot-old bot commented Nov 1, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chenwhql left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MingMingShangTian commented Nov 1, 2021 •

edited

Loading