Skip to content

Conversation

@NHZlX
Copy link
Contributor

@NHZlX NHZlX commented Mar 20, 2019

zero copy 使用方式

   std::vector<PaddleTensor> inputs;

    std::vector<std::vector<float>> input(1);
    std::vector<std::vector<float>> output;

    input[0].resize(input_num);
    float sum_i = 0;
      for (int i = 0; i < input_num; i++) {
       //input[0][i] = Random(1, 128.);
        input[0][i] = 1;
       sum_i += input[0][i];
    }

    // set input
    auto input_names = predictor->GetInputNames();
    for (auto& name : input_names) {
      auto input_t = predictor->GetInputTensor(name);
      input_t->Reshape({batch_size, channels, height, width});
      input_t->copy_from_cpu(input[0].data());
    }

    // predict
    CHECK(predictor->ZeroCopyRun());
    // get output
    auto output_names = predictor->GetOutputNames();
    for (auto& name : output_names) {
      std::cout << name << std::endl;
      std::vector<float> temp_data;
      auto output_t = predictor->GetOutputTensor(name);
      PaddleDType output_type = output_t->type();
      std::cout << "Type: " << output_type << std::endl;
      std::vector<int> temp_shape = output_t->shape();
      int num = std::accumulate(temp_shape.begin(), temp_shape.end(), 1, std::multiplies<int>());
      temp_data.resize(num);

      output_t->copy_to_cpu(temp_data.data());
      output.push_back(temp_data);
    }

2. add data type for zero copy
if (type == framework::proto::VarType::FP32) {
return PaddleDType::FLOAT32;
} else if (type == framework::proto::VarType::INT64) {
return PaddleDType::INT64;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We support INT32 as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. At present, int32 will not be used of the VIS model and this branch will then merge to develop.

@NHZlX NHZlX merged commit 1946d67 into PaddlePaddle:feature/anakin-engine Mar 20, 2019
NHZlX added a commit to NHZlX/Paddle that referenced this pull request Mar 20, 2019
…addlePaddle#16313

1. refine anakin engine
2. add data type for zero copy

align dev branch and PaddlePaddle:feature/anakin-engine brach
the cudnn workspace modify was not included for now, because we use a hard code way
in feature/anakin-engine branch. There should be a better way to implement it,
and subsequent submissions will be made.

test=develop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants