Add data type reorder for DNNL OP #1
Conversation
|
@pinzhenx @jiayisunx , @hongzhen1, could you help review on the reorder code for BF16? |
| /** | ||
| * Reorder the input tensor to the specified scalar type. It is an optimized version for | ||
| * DNNL OP. It means that if DNNL supports current OP, you should call this API. Otherwise, you | ||
| * should call @sa @ref reorderTensorToScalaraType |
There was a problem hiding this comment.
Doxygen semantic. "sa" means "see", "ref" means a link.
torch_ipex/csrc/aten_ipex_bridge.cpp
Outdated
|
|
||
| void reorderTensorToScalarTypeForDNNL(const at::Tensor& ipexTensor, at::ScalarType dstScalarType) { | ||
| if (ipexTensor.device().type() == at::DeviceType::CPU) { | ||
| return reorderTensorToScalaraType(ipexTensor, dstScalarType); |
There was a problem hiding this comment.
this branch handles the conversion before OP returns?
if so, can use last branch (w/o shade) to handle this?
There was a problem hiding this comment.
The API is used before the dispatch to DNNL op. reorderTensorToScalarTypeForDNNL => Dispatch to DNNL.
There was a problem hiding this comment.
if this API is called at the beginning of OPs, Is there any chance the device will be CPU?
There was a problem hiding this comment.
Yes. "to" should be the case. CPU tensor converts to IPEX tensor.
There was a problem hiding this comment.
Sorry, misunderstood your comment point. I will fix it.
torch_ipex/csrc/aten_ipex_bridge.cpp
Outdated
| return; | ||
|
|
||
| if (check_data_is_part_of_storage(ipexTensor)) | ||
| if (!check_tensor_own_whole_storage(ipexTensor)) |
There was a problem hiding this comment.
where is the gatekeeper for this branch?
There was a problem hiding this comment.
I will refine it. I intend not to support the case if the tensor data is just a part of its storage.
enable automix bf16 interaction, emb fw
No description provided.