Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

安装失败 #12

Closed
elvinpoon opened this issue Aug 31, 2016 · 11 comments
Closed

安装失败 #12

elvinpoon opened this issue Aug 31, 2016 · 11 comments
Assignees
@reyoung
Labels
duplicate

Comments

@elvinpoon
Copy link

环境：ubuntu16.04 + cuda8.0RC + cuDNN5.1.5

CMake Error at paddle_cuda_generated_hl_table_apply.cu.o.cmake:262 (message):
  Error generating file
  /home/pcy/paddle/paddle/cuda/CMakeFiles/paddle_cuda.dir/src/./paddle_cuda_generated_hl_table_apply.cu.o


paddle/cuda/CMakeFiles/paddle_cuda.dir/build.make:2365: recipe for target 'paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_table_apply.cu.o' failed
make[2]: *** [paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_table_apply.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
Done processing /home/pcy/paddle/paddle/math/Vector.cpp
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/Allocator.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/BaseMatrix.h
Total errors found: 0
/home/pcy/paddle/paddle/cuda/include/hl_device_functions.cuh:39:13: error: 'double hppl::atomicAdd(double*, double)' conflicts with a previous declaration
 using hppl::atomicAdd;
             ^
/usr/local/cuda/include/sm_60_atomic_functions.h:308:46: note: previous declaration 'double atomicAdd(double*, double)'
 __SM_60_ATOMIC_FUNCTIONS_DECL__ double atomicAdd(double *address, double val) __DEF_IF_HOST
                                              ^
Done processing /home/pcy/paddle/paddle/math/Bits.h
Total errors found: 0
CMake Error at paddle_cuda_generated_hl_cuda_sequence.cu.o.cmake:262 (message):
  Error generating file
  /home/pcy/paddle/paddle/cuda/CMakeFiles/paddle_cuda.dir/src/./paddle_cuda_generated_hl_cuda_sequence.cu.o


paddle/cuda/CMakeFiles/paddle_cuda.dir/build.make:2140: recipe for target 'paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_cuda_sequence.cu.o' failed
make[2]: *** [paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_cuda_sequence.cu.o] Error 1
Done processing /home/pcy/paddle/paddle/math/CpuSparseMatrix.h
Total errors found: 0
/home/pcy/paddle/paddle/cuda/src/hl_cuda_aggregate.cu: In function 'void hl_vector_sum(paddle::real*, paddle::real*, int)':
/home/pcy/paddle/paddle/cuda/src/hl_cuda_aggregate.cu:268:21: error: comparison of constant '34' with boolean expression is always false [-Werror=bool-compare]
   } while (isNotReady == cudaErrorNotReady);
                     ^
/home/pcy/paddle/paddle/cuda/src/hl_cuda_aggregate.cu: In function 'void hl_vector_abs_sum(paddle::real*, paddle::real*, int)':
/home/pcy/paddle/paddle/cuda/src/hl_cuda_aggregate.cu:324:21: error: comparison of constant '34' with boolean expression is always false [-Werror=bool-compare]
   } while (isNotReady == cudaErrorNotReady);
                     ^
Done processing /home/pcy/paddle/paddle/math/ExecViaCpu.h
Total errors found: 0
cc1plus: all warnings being treated as errors
CMake Error at paddle_cuda_generated_hl_cuda_aggregate.cu.o.cmake:262 (message):
  Error generating file
  /home/pcy/paddle/paddle/cuda/CMakeFiles/paddle_cuda.dir/src/./paddle_cuda_generated_hl_cuda_aggregate.cu.o


paddle/cuda/CMakeFiles/paddle_cuda.dir/build.make:523: recipe for target 'paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_cuda_aggregate.cu.o' failed
make[2]: *** [paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_cuda_aggregate.cu.o] Error 1
Done processing /home/pcy/paddle/paddle/math/MathFunctions.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/MathUtils.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/Matrix.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/MemoryHandle.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/PoolAllocator.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/SIMDFunctions.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/SparseMatrix.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/SparseRowMatrix.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/Storage.h
Total errors found: 0
Done processing /home/pcy/paddle/paddle/math/Vector.h
Total errors found: 0
/home/pcy/paddle/paddle/cuda/include/hl_device_functions.cuh:39:13: error: 'double hppl::atomicAdd(double*, double)' conflicts with a previous declaration
 using hppl::atomicAdd;
             ^
/usr/local/cuda/include/sm_60_atomic_functions.h:308:46: note: previous declaration 'double atomicAdd(double*, double)'
 __SM_60_ATOMIC_FUNCTIONS_DECL__ double atomicAdd(double *address, double val) __DEF_IF_HOST
                                              ^
[ 48%] Built target paddle_math
CMake Error at paddle_gserver_generated_GruCompute.cu.o.cmake:262 (message):
  Error generating file
  /home/pcy/paddle/paddle/gserver/CMakeFiles/paddle_gserver.dir/layers/./paddle_gserver_generated_GruCompute.cu.o


paddle/gserver/CMakeFiles/paddle_gserver.dir/build.make:329: recipe for target 'paddle/gserver/CMakeFiles/paddle_gserver.dir/layers/paddle_gserver_generated_GruCompute.cu.o' failed
make[2]: *** [paddle/gserver/CMakeFiles/paddle_gserver.dir/layers/paddle_gserver_generated_GruCompute.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/home/pcy/paddle/paddle/cuda/include/hl_device_functions.cuh:39:13: error: 'double hppl::atomicAdd(double*, double)' conflicts with a previous declaration
 using hppl::atomicAdd;
             ^
/usr/local/cuda/include/sm_60_atomic_functions.h:308:46: note: previous declaration 'double atomicAdd(double*, double)'
 __SM_60_ATOMIC_FUNCTIONS_DECL__ double atomicAdd(double *address, double val) __DEF_IF_HOST
                                              ^
CMake Error at paddle_gserver_generated_LstmCompute.cu.o.cmake:262 (message):
  Error generating file
  /home/pcy/paddle/paddle/gserver/CMakeFiles/paddle_gserver.dir/layers/./paddle_gserver_generated_LstmCompute.cu.o


paddle/gserver/CMakeFiles/paddle_gserver.dir/build.make:602: recipe for target 'paddle/gserver/CMakeFiles/paddle_gserver.dir/layers/paddle_gserver_generated_LstmCompute.cu.o' failed
make[2]: *** [paddle/gserver/CMakeFiles/paddle_gserver.dir/layers/paddle_gserver_generated_LstmCompute.cu.o] Error 1
CMakeFiles/Makefile2:519: recipe for target 'paddle/gserver/CMakeFiles/paddle_gserver.dir/all' failed
make[1]: *** [paddle/gserver/CMakeFiles/paddle_gserver.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
/home/pcy/paddle/paddle/cuda/include/hl_device_functions.cuh:39:13: error: 'double hppl::atomicAdd(double*, double)' conflicts with a previous declaration
 using hppl::atomicAdd;
             ^
/usr/local/cuda/include/sm_60_atomic_functions.h:308:46: note: previous declaration 'double atomicAdd(double*, double)'
 __SM_60_ATOMIC_FUNCTIONS_DECL__ double atomicAdd(double *address, double val) __DEF_IF_HOST
                                              ^
CMake Error at paddle_cuda_generated_hl_cuda_sparse.cu.o.cmake:262 (message):
  Error generating file
  /home/pcy/paddle/paddle/cuda/CMakeFiles/paddle_cuda.dir/src/./paddle_cuda_generated_hl_cuda_sparse.cu.o


paddle/cuda/CMakeFiles/paddle_cuda.dir/build.make:1009: recipe for target 'paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_cuda_sparse.cu.o' failed
make[2]: *** [paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_cuda_sparse.cu.o] Error 1
/home/pcy/paddle/paddle/cuda/include/hl_device_functions.cuh:39:13: error: 'double hppl::atomicAdd(double*, double)' conflicts with a previous declaration
 using hppl::atomicAdd;
             ^
/usr/local/cuda/include/sm_60_atomic_functions.h:308:46: note: previous declaration 'double atomicAdd(double*, double)'
 __SM_60_ATOMIC_FUNCTIONS_DECL__ double atomicAdd(double *address, double val) __DEF_IF_HOST
                                              ^
CMake Error at paddle_cuda_generated_hl_cuda_lstm.cu.o.cmake:262 (message):
  Error generating file
  /home/pcy/paddle/paddle/cuda/CMakeFiles/paddle_cuda.dir/src/./paddle_cuda_generated_hl_cuda_lstm.cu.o


paddle/cuda/CMakeFiles/paddle_cuda.dir/build.make:1464: recipe for target 'paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_cuda_lstm.cu.o' failed
make[2]: *** [paddle/cuda/CMakeFiles/paddle_cuda.dir/src/paddle_cuda_generated_hl_cuda_lstm.cu.o] Error 1
CMakeFiles/Makefile2:299: recipe for target 'paddle/cuda/CMakeFiles/paddle_cuda.dir/all' failed
make[1]: *** [paddle/cuda/CMakeFiles/paddle_cuda.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2
@gangliao
Copy link
Contributor

we do not test PaddlePaddle on cuda 8.0 and cudnn v5.1, we will test these version later.

@elvinpoon
Copy link
Author

建议发布cuda8.0+cuDNN 5的binary release

@reyoung
Copy link
Collaborator

reyoung commented Aug 31, 2016

This issue is duplicated, please see #3 for the progress of supporting cuda 8.0

@reyoung reyoung closed this as completed Aug 31, 2016
@stoneyang
Copy link
Contributor

stoneyang commented Sep 2, 2016 •
edited
Loading

@elvinpoon see my PR #15 , it is merged into the master branch. You can check it out and rebuild the whole repo :)

But, GPU version's still not functioning well, see #18 for more details on description of it.

@kitianFresh
Copy link

是啊，我的也是ubuntu16.04，出现错误，什么时候修复啊，等着用呢

@gangliao
Copy link
Contributor

gangliao commented Oct 24, 2016 •
edited
Loading

@kitianFresh Can you paste more detail error info?

@kitianFresh
Copy link

kitianFresh commented Oct 26, 2016 •
edited
Loading

@gangliao My enviroment is ubuntu16.04, First, i use sudo dpkg -i paddle-cpu-0.8.0b1-Linux.deb to install , but failed with error: dependency problems; following is my error message:
Selecting previously unselected package paddle.
(Reading database ... 427947 files and directories currently installed.)
Preparing to unpack paddle-cpu-0.8.0b1-Linux.deb ...
Unpacking paddle (0.8.0b1) ...
dpkg: dependency problems prevent configuration of paddle:
paddle depends on libgflags2; however:
Package libgflags2 is not installed.
paddle depends on libgoogle-glog0; however:
Package libgoogle-glog0 is not installed.
paddle depends on libprotobuf8; however:
Package libprotobuf8 is not installed.

dpkg: error processing package paddle (--install):
dependency problems - leaving unconfigured
Errors were encountered while processing:
paddle

then, i use sudo apt-get install -f, after that , i install again, but it failed agian! so i manually install dependency , sudo apt-get install libgflags2 libgoogle-glog0 libprotobuf8, but it seems no these lib, and error like this:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package libgflags2 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
libgflags2v5:i386 libgflags2v5

Package libgoogle-glog0 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
libgoogle-glog0v5:i386 libgoogle-glog0v5

Package libprotobuf8 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'libgflags2' has no installation candidate
E: Package 'libgoogle-glog0' has no installation candidate
E: Package 'libprotobuf8' has no installation candidate

so i use sudo apt-get install libgflags2v5 libgoogle-glog0v5 libprotobuf8, but it failed!!

@kitianFresh
Copy link

@gangliao Does the bug fix in ubuntu16.04?

@gangliao
Copy link
Contributor

gangliao commented Oct 27, 2016 •
edited
Loading

@kitianFresh Can you follow below commands and try it again? Thanks.

sudo apt-get update
sudo apt-get install libgoogle-glog-dev
sudo apt-get install libgflags-dev
sudo apt-get install libgtest-dev
pushd /usr/src/gtest
cmake .
make
sudo cp *.a /usr/lib
popd

@kitianFresh
Copy link

kitianFresh commented Oct 27, 2016 •
edited
Loading

@gangliao I tried again step by step! but failed again! following is the details.

sudo apt-get update
[sudo] password for kinny: 
Hit:1 http://mirrors.ustc.edu.cn/ubuntu xenial InRelease
Get:2 http://mirrors.ustc.edu.cn/ubuntu xenial-updates InRelease [95.7 kB]
Get:3 http://mirrors.ustc.edu.cn/ubuntu xenial-backports InRelease [92.2 kB]
Get:4 http://mirrors.ustc.edu.cn/ubuntu xenial-security InRelease [94.5 kB]
Get:5 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/multiverse Sources [3,216 B]
Get:6 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/main Sources [201 kB]
Get:7 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/universe Sources [107 kB] 
Get:8 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/main amd64 Packages [413 kB]
Get:9 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/main i386 Packages [408 kB]
Get:10 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/main Translation-en [158 kB]
Get:11 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/universe amd64 Packages [356 kB]
Get:12 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/universe i386 Packages [352 kB]
Get:13 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/universe Translation-en [128 kB]
Get:14 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/universe amd64 DEP-11 Metadata [107 kB]
Get:15 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/universe DEP-11 64x64 Icons [119 kB]
Get:16 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/multiverse amd64 Packages [5,492 B]
Get:17 http://mirrors.ustc.edu.cn/ubuntu xenial-updates/multiverse i386 Packages [4,284 B]
Get:18 http://mirrors.ustc.edu.cn/ubuntu xenial-backports/main Sources [1,972 B]
Get:19 http://mirrors.ustc.edu.cn/ubuntu xenial-backports/main amd64 Packages [2,552 B]
Get:20 http://mirrors.ustc.edu.cn/ubuntu xenial-backports/main i386 Packages [2,556 B]
Get:21 http://mirrors.ustc.edu.cn/ubuntu xenial-backports/main amd64 DEP-11 Metadata [196 B]
Get:22 http://mirrors.ustc.edu.cn/ubuntu xenial-security/multiverse Sources [728 B]
Get:23 http://mirrors.ustc.edu.cn/ubuntu xenial-security/multiverse amd64 Packages [1,176 B]
Get:24 http://mirrors.ustc.edu.cn/ubuntu xenial-security/multiverse i386 Packages [1,344 B]
Get:25 https://apt.dockerproject.org/repo ubuntu-xenial InRelease [30.2 kB]
Get:26 https://apt.dockerproject.org/repo ubuntu-xenial/main amd64 Packages [2,553 B]
Fetched 2,688 kB in 3s (817 kB/s)      
Reading package lists... Done

kinny@kinny-Lenovo-XiaoXin:~$ sudo apt-get install libgoogle-glog-dev 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  authbind gstreamer0.10-gnomevfs gthumb-data libatlas3-base libblas-common libcommons-collections3-java libcommons-dbcp-java
  libcommons-pool-java libecj-java libgfortran3 libgstreamer0.10-0 liblept5 libopenjp2-7 libqgsttools-p1 libqt5clucene5
  libqt5designer5 libqt5help5 libqt5multimedia5-plugins libqt5multimediawidgets5 libtesseract3 libtomcat8-java
  linux-headers-4.4.0-21 linux-headers-4.4.0-21-generic linux-headers-4.4.0-28 linux-headers-4.4.0-28-generic
  linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-headers-4.4.0-34 linux-headers-4.4.0-34-generic
  linux-headers-4.4.0-36 linux-headers-4.4.0-36-generic linux-headers-4.4.0-38 linux-headers-4.4.0-38-generic
  linux-image-4.4.0-21-generic linux-image-4.4.0-28-generic linux-image-4.4.0-31-generic linux-image-4.4.0-34-generic
  linux-image-4.4.0-36-generic linux-image-4.4.0-38-generic linux-image-extra-4.4.0-21-generic
  linux-image-extra-4.4.0-28-generic linux-image-extra-4.4.0-31-generic linux-image-extra-4.4.0-34-generic
  linux-image-extra-4.4.0-36-generic linux-image-extra-4.4.0-38-generic linux-signed-image-4.4.0-21-generic
  linux-signed-image-4.4.0-28-generic linux-signed-image-4.4.0-31-generic linux-signed-image-4.4.0-34-generic
  linux-signed-image-4.4.0-36-generic linux-signed-image-4.4.0-38-generic python-numpy python-protobuf python3-pyqt5
  python3-pyqt5.qtmultimedia python3-pyqt5.qtquick python3-pyqt5.qtwebkit python3-sip python3-xlib qml-module-qtquick-controls
  qtdeclarative5-controls-plugin tesseract-ocr tesseract-ocr-chi-sim tesseract-ocr-chi-tra tesseract-ocr-eng tesseract-ocr-equ
  tesseract-ocr-osd tomcat8-common
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libgflags-dev libgflags2v5 libgoogle-glog0v5
The following NEW packages will be installed:
  libgflags-dev libgflags2v5 libgoogle-glog-dev libgoogle-glog0v5
0 upgraded, 4 newly installed, 0 to remove and 71 not upgraded.
Need to get 240 kB of archives.
After this operation, 1,340 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://mirrors.ustc.edu.cn/ubuntu xenial/universe amd64 libgflags2v5 amd64 2.1.2-3 [54.4 kB]
Get:2 http://mirrors.ustc.edu.cn/ubuntu xenial/universe amd64 libgflags-dev amd64 2.1.2-3 [64.8 kB]
Get:3 http://mirrors.ustc.edu.cn/ubuntu xenial/universe amd64 libgoogle-glog0v5 amd64 0.3.4-0.1 [49.3 kB]
Get:4 http://mirrors.ustc.edu.cn/ubuntu xenial/universe amd64 libgoogle-glog-dev amd64 0.3.4-0.1 [71.3 kB]
Fetched 240 kB in 0s (332 kB/s)               
Selecting previously unselected package libgflags2v5.
(Reading database ... 427964 files and directories currently installed.)
Preparing to unpack .../libgflags2v5_2.1.2-3_amd64.deb ...
Unpacking libgflags2v5 (2.1.2-3) ...
Selecting previously unselected package libgflags-dev.
Preparing to unpack .../libgflags-dev_2.1.2-3_amd64.deb ...
Unpacking libgflags-dev (2.1.2-3) ...
Selecting previously unselected package libgoogle-glog0v5.
Preparing to unpack .../libgoogle-glog0v5_0.3.4-0.1_amd64.deb ...
Unpacking libgoogle-glog0v5 (0.3.4-0.1) ...
Selecting previously unselected package libgoogle-glog-dev.
Preparing to unpack .../libgoogle-glog-dev_0.3.4-0.1_amd64.deb ...
Unpacking libgoogle-glog-dev (0.3.4-0.1) ...
Setting up libgflags2v5 (2.1.2-3) ...
Setting up libgflags-dev (2.1.2-3) ...
Setting up libgoogle-glog0v5 (0.3.4-0.1) ...
Setting up libgoogle-glog-dev (0.3.4-0.1) ...
Processing triggers for libc-bin (2.23-0ubuntu3) ...

kinny@kinny-Lenovo-XiaoXin:~$ sudo apt-get install libgflags-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libgflags-dev is already the newest version (2.1.2-3).
libgflags-dev set to manually installed.
The following packages were automatically installed and are no longer required:
  authbind gstreamer0.10-gnomevfs gthumb-data libatlas3-base libblas-common libcommons-collections3-java libcommons-dbcp-java
  libcommons-pool-java libecj-java libgfortran3 libgstreamer0.10-0 liblept5 libopenjp2-7 libqgsttools-p1 libqt5clucene5
  libqt5designer5 libqt5help5 libqt5multimedia5-plugins libqt5multimediawidgets5 libtesseract3 libtomcat8-java
  linux-headers-4.4.0-21 linux-headers-4.4.0-21-generic linux-headers-4.4.0-28 linux-headers-4.4.0-28-generic
  linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-headers-4.4.0-34 linux-headers-4.4.0-34-generic
  linux-headers-4.4.0-36 linux-headers-4.4.0-36-generic linux-headers-4.4.0-38 linux-headers-4.4.0-38-generic
  linux-image-4.4.0-21-generic linux-image-4.4.0-28-generic linux-image-4.4.0-31-generic linux-image-4.4.0-34-generic
  linux-image-4.4.0-36-generic linux-image-4.4.0-38-generic linux-image-extra-4.4.0-21-generic
  linux-image-extra-4.4.0-28-generic linux-image-extra-4.4.0-31-generic linux-image-extra-4.4.0-34-generic
  linux-image-extra-4.4.0-36-generic linux-image-extra-4.4.0-38-generic linux-signed-image-4.4.0-21-generic
  linux-signed-image-4.4.0-28-generic linux-signed-image-4.4.0-31-generic linux-signed-image-4.4.0-34-generic
  linux-signed-image-4.4.0-36-generic linux-signed-image-4.4.0-38-generic python-numpy python-protobuf python3-pyqt5
  python3-pyqt5.qtmultimedia python3-pyqt5.qtquick python3-pyqt5.qtwebkit python3-sip python3-xlib qml-module-qtquick-controls
  qtdeclarative5-controls-plugin tesseract-ocr tesseract-ocr-chi-sim tesseract-ocr-chi-tra tesseract-ocr-eng tesseract-ocr-equ
  tesseract-ocr-osd tomcat8-common
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 71 not upgraded.


kinny@kinny-Lenovo-XiaoXin:~$ sudo apt-get install libgtest-dev 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  authbind gstreamer0.10-gnomevfs gthumb-data libatlas3-base libblas-common libcommons-collections3-java libcommons-dbcp-java
  libcommons-pool-java libecj-java libgfortran3 libgstreamer0.10-0 liblept5 libopenjp2-7 libqgsttools-p1 libqt5clucene5
  libqt5designer5 libqt5help5 libqt5multimedia5-plugins libqt5multimediawidgets5 libtesseract3 libtomcat8-java
  linux-headers-4.4.0-21 linux-headers-4.4.0-21-generic linux-headers-4.4.0-28 linux-headers-4.4.0-28-generic
  linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-headers-4.4.0-34 linux-headers-4.4.0-34-generic
  linux-headers-4.4.0-36 linux-headers-4.4.0-36-generic linux-headers-4.4.0-38 linux-headers-4.4.0-38-generic
  linux-image-4.4.0-21-generic linux-image-4.4.0-28-generic linux-image-4.4.0-31-generic linux-image-4.4.0-34-generic
  linux-image-4.4.0-36-generic linux-image-4.4.0-38-generic linux-image-extra-4.4.0-21-generic
  linux-image-extra-4.4.0-28-generic linux-image-extra-4.4.0-31-generic linux-image-extra-4.4.0-34-generic
  linux-image-extra-4.4.0-36-generic linux-image-extra-4.4.0-38-generic linux-signed-image-4.4.0-21-generic
  linux-signed-image-4.4.0-28-generic linux-signed-image-4.4.0-31-generic linux-signed-image-4.4.0-34-generic
  linux-signed-image-4.4.0-36-generic linux-signed-image-4.4.0-38-generic python-numpy python-protobuf python3-pyqt5
  python3-pyqt5.qtmultimedia python3-pyqt5.qtquick python3-pyqt5.qtwebkit python3-sip python3-xlib qml-module-qtquick-controls
  qtdeclarative5-controls-plugin tesseract-ocr tesseract-ocr-chi-sim tesseract-ocr-chi-tra tesseract-ocr-eng tesseract-ocr-equ
  tesseract-ocr-osd tomcat8-common
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
  libgtest-dev
0 upgraded, 1 newly installed, 0 to remove and 71 not upgraded.
Need to get 198 kB of archives.
After this operation, 1,361 kB of additional disk space will be used.
Get:1 http://mirrors.ustc.edu.cn/ubuntu xenial/universe amd64 libgtest-dev amd64 1.7.0-4ubuntu1 [198 kB]
Fetched 198 kB in 0s (406 kB/s)  
Selecting previously unselected package libgtest-dev:amd64.
(Reading database ... 428002 files and directories currently installed.)
Preparing to unpack .../libgtest-dev_1.7.0-4ubuntu1_amd64.deb ...
Unpacking libgtest-dev:amd64 (1.7.0-4ubuntu1) ...
Setting up libgtest-dev:amd64 (1.7.0-4ubuntu1) ...

kinny@kinny-Lenovo-XiaoXin:/usr/src/gtest$ sudo cmake .
-- The CXX compiler identification is GNU 5.4.0
-- The C compiler identification is GNU 5.4.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found PythonInterp: /usr/bin/python (found version "2.7.12") 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/src/gtest
kinny@kinny-Lenovo-XiaoXin:/usr/src/gtest$ sudo make
Scanning dependencies of target gtest
[ 25%] Building CXX object CMakeFiles/gtest.dir/src/gtest-all.cc.o
[ 50%] Linking CXX static library libgtest.a
[ 50%] Built target gtest
Scanning dependencies of target gtest_main
[ 75%] Building CXX object CMakeFiles/gtest_main.dir/src/gtest_main.cc.o
[100%] Linking CXX static library libgtest_main.a
[100%] Built target gtest_main
kinny@kinny-Lenovo-XiaoXin:/usr/src/gtest$ sudo cp *.a /usr/lib

kinny@kinny-Lenovo-XiaoXin:~$ sudo dpkg -i paddle-cpu-0.8.0b1-Linux.deb 
Selecting previously unselected package paddle.
(Reading database ... 430127 files and directories currently installed.)
Preparing to unpack paddle-cpu-0.8.0b1-Linux.deb ...
Unpacking paddle (0.8.0b1) ...
dpkg: dependency problems prevent configuration of paddle:
 paddle depends on libgflags2; however:
  Package libgflags2 is not installed.
 paddle depends on libgoogle-glog0; however:
  Package libgoogle-glog0 is not installed.
 paddle depends on libprotobuf8; however:
  Package libprotobuf8 is not installed.

dpkg: error processing package paddle (--install):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 paddle

I am sorry. I am not familiar with cmake and linux dependency.So i don't understand what is the real reason for failure in my laptop. Does the dependency change its name in ubuntu16.04? It sames that paddlepaddle want to use name libgflags2 libgoogle-glog0 libprotobuf8 , but its name changed in ubuntu16.04.
Is it possible that my ubuntu16.04 was different from others? I want to know other guys who use ubuntu16.04 encounter this problem.

@reyoung
Copy link
Collaborator

reyoung commented Oct 27, 2016 •
edited
Loading

ubuntu 16.04 install package is not tested, and maybe sightly different from ubuntu 14.04.

It is recommend to use Docker to run Paddle, see here.

And ubuntu 16.04 install package will be supported soon. Please add another issue for ubuntu 16.04 support.

@sdujq sdujq mentioned this issue May 5, 2017
Closed
@xiehongweiscut xiehongweiscut mentioned this issue Aug 3, 2017
Closed
reyoung pushed a commit to reyoung/Paddle that referenced this issue Aug 12, 2017
@wangkuiyi
Merge pull request PaddlePaddle#12 from qingqing01/framework_proto
5422776 
make unit test of backward_test pass.
@fsfszongming256 fsfszongming256 mentioned this issue Sep 8, 2017
Closed
@fty8788 fty8788 mentioned this issue Jan 23, 2018
Closed
@likeqinqin likeqinqin mentioned this issue Mar 28, 2018
Closed
@heroes999 heroes999 mentioned this issue Mar 30, 2018
Closed
@likeqinqin likeqinqin mentioned this issue Apr 11, 2018
Closed
@lyp2github lyp2github mentioned this issue Apr 18, 2018
Closed
@wangshuohuan wangshuohuan mentioned this issue Apr 25, 2018
Closed
@yttbgf yttbgf mentioned this issue May 22, 2018
Closed
@xuezhong xuezhong mentioned this issue Jun 12, 2018
Closed
@lyp2github lyp2github mentioned this issue Jun 13, 2018
Closed
velconia pushed a commit that referenced this issue Mar 22, 2019
@junjun315
Merge pull request #12 from PaddlePaddle/develop
5bb04ea 
merge to local
@jjlucus jjlucus mentioned this issue Apr 29, 2019
Closed
@kangzhangqi kangzhangqi mentioned this issue Jun 26, 2019
Closed
chenwhql added a commit that referenced this issue Nov 1, 2021
@chenwhql @MingMingShangTian
@zyfncg @YuanRisheng @Shixiaowei02
Paddle Tensor Operation Library initial implementation (#34425)
b9fdd3b 
* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

* polish kernel signature details

* fix a bug about offsets of the tensor, test=develop (#31)

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

Co-authored-by: chentianyu03 <ctychentianyu@gmail.com>
Co-authored-by: zyfncg <1370305206@qq.com>
Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
zyfncg added a commit that referenced this issue Nov 2, 2021
@zyfncg @chenwhql
@MingMingShangTian @YuanRisheng @Shixiaowei02
Add matmul_v2 kernel in pten (#36844)
e11ecfc 
* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

* polish kernel signature details

* fix a bug about offsets of the tensor, test=develop (#31)

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* add matmul kernel in pten

* add unittest for new matmul_v2 kernel

* fix bug of CI compile

* fix bug of CI compile

* merge conflict

* remove useless file

Co-authored-by: Chen Weihang <chenweihang@baidu.com>
Co-authored-by: chentianyu03 <ctychentianyu@gmail.com>
Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
chenwhql added a commit that referenced this issue Nov 2, 2021
@YuanRisheng @chenwhql
@MingMingShangTian @zyfncg @Shixiaowei02
Add Intermediate Kernel API for refactor Tensor Lib (#36914)
4a7f1a0 
* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* Add Intermediate API layer

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* intermediate api adapt to new dense tensor

* add some TODO and delete include header

Co-authored-by: Chen Weihang <chenweihang@baidu.com>
Co-authored-by: chentianyu03 <ctychentianyu@gmail.com>
Co-authored-by: zyfncg <1370305206@qq.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
MingMingShangTian added a commit that referenced this issue Nov 2, 2021
@MingMingShangTian @chenwhql
@zyfncg @YuanRisheng @Shixiaowei02
support multiply inputs and outputs (#36851)
e4a134a 
* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

* polish kernel signature details

* fix a bug about offsets of the tensor, test=develop (#31)

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* support multiply inputs and outputs

* rm attrs {}

* fix multioutputs bug

* merge develop

* remove unsed header file

* add missing & in const reference

* modify inputAt, outputAt to inputBetween, outputBetween

Co-authored-by: Chen Weihang <chenweihang@baidu.com>
Co-authored-by: zyfncg <1370305206@qq.com>
Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
ghost pushed a commit to piotrekobi/Paddle that referenced this issue Nov 3, 2021
@chenwhql @MingMingShangTian
@zyfncg @YuanRisheng @Shixiaowei02 @piotrekobi
Paddle Tensor Operation Library initial implementation (PaddlePaddle#…
b2e0661 
…34425)

* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (PaddlePaddle#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (PaddlePaddle#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (PaddlePaddle#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (PaddlePaddle#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (PaddlePaddle#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (PaddlePaddle#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (PaddlePaddle#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (PaddlePaddle#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (PaddlePaddle#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (PaddlePaddle#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (PaddlePaddle#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (PaddlePaddle#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (PaddlePaddle#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (PaddlePaddle#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (PaddlePaddle#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (PaddlePaddle#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (PaddlePaddle#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

* polish kernel signature details

* fix a bug about offsets of the tensor, test=develop (PaddlePaddle#31)

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

Co-authored-by: chentianyu03 <ctychentianyu@gmail.com>
Co-authored-by: zyfncg <1370305206@qq.com>
Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
ghost pushed a commit to piotrekobi/Paddle that referenced this issue Nov 3, 2021
@zyfncg @chenwhql
@MingMingShangTian @YuanRisheng @Shixiaowei02 @piotrekobi
Add matmul_v2 kernel in pten (PaddlePaddle#36844)
356a64e 
* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (PaddlePaddle#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (PaddlePaddle#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (PaddlePaddle#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (PaddlePaddle#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (PaddlePaddle#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (PaddlePaddle#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (PaddlePaddle#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (PaddlePaddle#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (PaddlePaddle#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (PaddlePaddle#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (PaddlePaddle#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (PaddlePaddle#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (PaddlePaddle#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (PaddlePaddle#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (PaddlePaddle#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (PaddlePaddle#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (PaddlePaddle#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

* polish kernel signature details

* fix a bug about offsets of the tensor, test=develop (PaddlePaddle#31)

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* add matmul kernel in pten

* add unittest for new matmul_v2 kernel

* fix bug of CI compile

* fix bug of CI compile

* merge conflict

* remove useless file

Co-authored-by: Chen Weihang <chenweihang@baidu.com>
Co-authored-by: chentianyu03 <ctychentianyu@gmail.com>
Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
ghost pushed a commit to piotrekobi/Paddle that referenced this issue Nov 3, 2021
@YuanRisheng @chenwhql
@MingMingShangTian @zyfncg @Shixiaowei02 @piotrekobi
Add Intermediate Kernel API for refactor Tensor Lib (PaddlePaddle#36914)
1ff9cf1 
* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (PaddlePaddle#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (PaddlePaddle#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (PaddlePaddle#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (PaddlePaddle#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (PaddlePaddle#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (PaddlePaddle#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (PaddlePaddle#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (PaddlePaddle#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (PaddlePaddle#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (PaddlePaddle#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (PaddlePaddle#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (PaddlePaddle#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (PaddlePaddle#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (PaddlePaddle#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (PaddlePaddle#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (PaddlePaddle#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* Add Intermediate API layer

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (PaddlePaddle#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* intermediate api adapt to new dense tensor

* add some TODO and delete include header

Co-authored-by: Chen Weihang <chenweihang@baidu.com>
Co-authored-by: chentianyu03 <ctychentianyu@gmail.com>
Co-authored-by: zyfncg <1370305206@qq.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
ghost pushed a commit to piotrekobi/Paddle that referenced this issue Nov 3, 2021
@MingMingShangTian @chenwhql
@zyfncg @YuanRisheng @Shixiaowei02 @piotrekobi
support multiply inputs and outputs (PaddlePaddle#36851)
e1547b2 
* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (PaddlePaddle#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (PaddlePaddle#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (PaddlePaddle#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (PaddlePaddle#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (PaddlePaddle#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (PaddlePaddle#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (PaddlePaddle#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (PaddlePaddle#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (PaddlePaddle#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (PaddlePaddle#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (PaddlePaddle#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (PaddlePaddle#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (PaddlePaddle#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (PaddlePaddle#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (PaddlePaddle#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (PaddlePaddle#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (PaddlePaddle#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

* polish kernel signature details

* fix a bug about offsets of the tensor, test=develop (PaddlePaddle#31)

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* support multiply inputs and outputs

* rm attrs {}

* fix multioutputs bug

* merge develop

* remove unsed header file

* add missing & in const reference

* modify inputAt, outputAt to inputBetween, outputBetween

Co-authored-by: Chen Weihang <chenweihang@baidu.com>
Co-authored-by: zyfncg <1370305206@qq.com>
Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
chenwhql added a commit that referenced this issue Nov 5, 2021
@YuanRisheng @chenwhql
@MingMingShangTian @zyfncg @Shixiaowei02
[PTen]Support XPU for Flatten Kernel (#36957)
09d407b 
* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* Add Intermediate API layer

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* intermediate api adapt to new dense tensor

* add some TODO and delete include header

* Support XPU for Flatten Kernel

* fix bugs when run kunlun ci

* fix compile bugs

* fix bugs for kunlun ci

* fix compile bugs when run kunlun

* fix compile bugs in kunlun

* fix compile bugs in kunlun

* fix bugs when compile

* fix bugs when compile

* fix compile bug

* delete useless annotation

Co-authored-by: Chen Weihang <chenweihang@baidu.com>
Co-authored-by: chentianyu03 <ctychentianyu@gmail.com>
Co-authored-by: zyfncg <1370305206@qq.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
zyfncg added a commit that referenced this issue Nov 8, 2021
@zyfncg @chenwhql
@MingMingShangTian @YuanRisheng @Shixiaowei02
[PTen] Add full kernel in pten (incomplete) (#36930)
655f4e3 
* initial tensor design & sign kernel demo

* add move constructor for meta & add lodtensor

* add dirs & sign xpu kernel

* add mean cpu&cuda kernel impl

* move sign & mean xpu & npu kernel

* add selected_rows basic impl

* refactor design, BaseTensor to DenseTensor, etc.

* add scale mkldnn kernel

* polish xpu & npu impl details

* fix mkldnn reuse compile failed

* change tensor operation lib name

* rename util filename

* add more comments

* change TensorImplInterface to TensorInterface

* add kernel key and factory

* remove MKLDNNTensorMeta, add MKLDNNDenseTensor

* change XXDeviceContext to XXContext

* add base kernel registrar utils & test on sign

* replace boost::any by paddle::any

* fix several ci failed

* fix npu compile error

* add ordered map util

* fix multiple ordered_map compile errors

* move dev into include dir

* support sign op in static op run

* fix static op run error

* fix new executor compile failed

* add dygraph branch & remove sign_op.h

* fix test_infer_no_need_buffer_slots

* fix rocm compile link error

* fix unitybuild error & clear glog

* fix npu compile failed

* skip quant trans test

* fix part windows compile problem

* fix xpu enforce error

* fix inference test failed

* remove ordered_map to solve quant failed

* fix part of rcom compile faild

* add more register kernels

* revert scale kernel temporarily

* fix code format error

* add new kernel registrar marco

* rename top to tcmpt

* revert xpu, npu, mkldnn impl & remove op def

* add kernel args parse functor to auto parse args

* revert some change & add scale kernels

* add op proto in dygraph kernelcontext building

* polish kernel dispatch logic & nameing rule

* fix scale kernel match error

* fix scale test failed

* add mean API and unittest

* test mean api success

* add branch to solve compiled error

* skip clang format error

* add mean skip rule in op_library

* add dot kernel, api and unittest (#6)

* remove old kernel and add symbol link

* fix dot compiled failed

* add merco for module declare

* fix npu and xpu compile error

* revert sign, mean, scale, dot kernel removing

* add comment for keeping old kernel impl

* fix mutable_data error

* fix bfloat16 conflit

* fix inference undef error

* adapt to msvc compile rules

* polish comment for template inst

* add cmake template instantiation for win

* fix backend to place device id bug

* fix ifdef error

* Op2functor (#7)

* add kernel args maker class

* make args maker non-const

* remove debug log

* modify codes by review options

* split constructPrKernelContext function

* fix output name bug

* fix test_mean_op test_sign_op failed

* fill_any_like kernel refactor (#10)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* skip dtype for fill_any_like

* add attrs for kernel key constrcut

* add use_pt_kernel Flags to control whether to use pt kernel (#13)

* add use_pt_kernel Flags to control whether to use pt kernel

* change the default value to true for cheking pt kernels

* fix mutable_data cuda place error

* move high level apis into hapi

* remove selectedrows adapting temporarily

* Support Scalar in Tensor Compute Library (#14)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* remove mkldnn tensor & polish details

* use flat_hash_map and small_vector in kernel factory

* Refactor flatten kernel (#12)

* refactor flatten kernel

* update infershape function

* fix compile bugs

* fix bugs when merge

* fix compiler bugs

* fix bugs when run test_flatten_api

* fix bugs when run test

* Revert "use flat_hash_map and small_vector in kernel factory"

This reverts commit 2309149.

* Move cpu, cuda and other device code into kernels (#15)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Perfect unitests (#16)

* perfect unittest

* update license

* replace with flat_hash_map, small_vector (#19)

* fix small_vector build error on windows platform

* replace with flat_hash_map, small_vector

* remove todo

* Perfect unitests (#20)

* perfect unittest

* update license

* fix bug when run tcmpt_utils_test

* refactor execution adapting impl

* fix insert conflit

* Fix CI bug of test_yolov3 (#21)

* fill_any_like kernel refactor

* remove useless code of full_like c++ api

* Support Scalar in Tensor Compute Library

* add scalar in dygraph and static graph mode

* keep the basic type for attr, instead of using scalar for all

* merge the code

* start refactor matmul

* move cpu, cuda and other device modules into kernels

* merge code

* polish code in operator.cc

* Fix CI bug of test_yolov3

* add the tensor base class, test=develop (#17)

* update the tensor base class, test=develop

* remove two funcs, test=develop

* update the error msg, test=develop

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* [no-verify] commit backend and tensor signature changes

* Rename tcmpt to pten (#23)

* rename tcmpt to pten

* update omitted files for rename to pten

* update omitted file for rename to pten

* remove k of all enum var

* remove kernel_instantiate (#26)

* remove symbols and spatial_tensor

* change common to functions

* readd share tensor impl methods

* add a candidate dense tensor class, test=develop (#28)

* change all Pt to Pten

* resolve conflit with xiaowei

* Op2functor opt1 (#27)

* replace to small vector and change to const &

* add std::move

Co-authored-by: Chen Weihang <chenweihang@baidu.com>

* polish kernel factory and kernel registry

* fix operator test error msg mismatch

* remove tensor signature and backend set member

* move scalar and polish enforce

* revert dtype layout change to fix error

* fix enum operator override error

* add several base unittests

* add pten utils tests

* polish some details

* Dev/op2func refactor 3 (#30)

* add a candidate dense tensor class, test=develop

* remove TensorBase::backend(), test=develop

* remove some ops, test=develop

* cherry-pick the pr of tensor meta, test=develop

* moves the dense tensor and some ops, test=develop

* update the linalg operator, test=develop

* update other operators, test=develop

* fix errors, test=develop

* fix bugs, test=develop

* try to resolve the problem of windows ci, test=develop

* updates codes, test=develop

* fix the tensor_utils.cc, test=develop

* modify the dense tensor, test=develop

* fix the data type, test=develop

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

* polish kernel signature details

* fix a bug about offsets of the tensor, test=develop (#31)

Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* polish some details

* add fill_constant kernel in pten

* fix bug of full api (c++)

* remove the support for SelectRows in new fill_constant kernel

* fix bug of setting fill_any_like kernel key

* merge code confilct

* modify fill_constant GetExpectedKernelType

* fix fill_constant KernelType bug

* polish code of build pten KernelContext

* refactor code of fill_constant in pten

Co-authored-by: Chen Weihang <chenweihang@baidu.com>
Co-authored-by: chentianyu03 <ctychentianyu@gmail.com>
Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>
Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
@AshburnLee AshburnLee mentioned this issue Dec 8, 2021
Merged
LielinJiang pushed a commit to LielinJiang/Paddle that referenced this issue Jan 27, 2022
@heavengate
Merge pull request PaddlePaddle#12 from LielinJiang/add_pipeline
bf7cb1c 
Add pipeline
@wuwuwuxxx wuwuwuxxx mentioned this issue Mar 10, 2022
Closed
zhoutianzi666 pushed a commit to zhoutianzi666/Paddle that referenced this issue May 23, 2022
@jiweibo
Merge pull request PaddlePaddle#12 from jiweibo/lic_demo
b105710 
add lic demo.
Thunderbrook pushed a commit to Thunderbrook/Paddle that referenced this issue Jun 2, 2022
@seemingwang
Merge pull request PaddlePaddle#12 from seemingwang/gpu_graph_engine3
2457680 
changing int64 key to uint64 for graph engine
zmxdream added a commit to zmxdream/Paddle that referenced this issue Jun 9, 2022
@zmxdream @yaoxuefeng6 @Thunderbrook
Fix eigvals_op (PaddlePaddle#12)
ef5d85a 
* dymf tmp

* add dymf tmp

* local test change

* pull thread pool

* fix conflict

* delete unuse log

* local change for mirrow 0

* fix dymf

* code clean

* fix code clean

* code clean

* code clean

* fix dymf

* fix dymf

* add endpass optimize

* clean code

* fix endpass optimize

* fix

* fix

* fix eigvals_op

* merge pre-stable

* merge pre-stable

Co-authored-by: yaoxuefeng6 <yaoxuefeng@baidu.com>
Co-authored-by: Thunderbrook <a754913769@163.com>
zmxdream added a commit to zmxdream/Paddle that referenced this issue Jun 24, 2022
@zmxdream @yaoxuefeng6 @Thunderbrook
Fix eigvals_op (PaddlePaddle#12)
7134e26 
* dymf tmp

* add dymf tmp

* local test change

* pull thread pool

* fix conflict

* delete unuse log

* local change for mirrow 0

* fix dymf

* code clean

* fix code clean

* code clean

* code clean

* fix dymf

* fix dymf

* add endpass optimize

* clean code

* fix endpass optimize

* fix

* fix

* fix eigvals_op

* merge pre-stable

* merge pre-stable

Co-authored-by: yaoxuefeng6 <yaoxuefeng@baidu.com>
Co-authored-by: Thunderbrook <a754913769@163.com>
zmxdream added a commit to zmxdream/Paddle that referenced this issue Jun 30, 2022
@SmallBirdLiao @xionglei1234
@rensilin @zmxdream @yaoxuefeng6 @Thunderbrook @liaoxiaochao-bb
Base remote/lxch pre stable (PaddlePaddle#30)
e603334 
* parquet parser

* fix IsThreadLocalCapturing

* run cuda kernel: CalcAucKernel with 512 threads

* fix_afs_api_download_dnn_plugin

* fix_fleet_last_base

* parquet parser

* add ps core so

* chg cmake

Co-authored-by: rensilin <rensilin@baidu.com>
Co-authored-by: root <root@yq01-sys-hic-k8s-v100-box-a225-0693.yq01.baidu.com>

* parquet

* fix IsThreadLocalCapturing

* run cuda kernel: CalcAucKernel with 512 threads

* fix_afs_api_download_dnn_plugin

* fix_fleet_last_base

* parquet parser

* add ps core so

* chg cmake

* fix libjvm lost

Co-authored-by: rensilin <rensilin@baidu.com>
Co-authored-by: root <root@yq01-sys-hic-k8s-v100-box-a225-0693.yq01.baidu.com>

* add dymf (PaddlePaddle#10)

* dymf tmp

* add dymf tmp

* local test change

* pull thread pool

* fix conflict

* delete unuse log

* local change for mirrow 0

* fix dymf

* code clean

* fix code clean

* code clean

* code clean

* fix dymf

* fix dymf

* add endpass optimize

* clean code

* fix endpass optimize

* fix

* fix

Co-authored-by: yaoxuefeng6 <yaoxuefeng@baidu.com>
Co-authored-by: Thunderbrook <a754913769@163.com>

* pipeline build (#9)

* Fix eigvals_op (PaddlePaddle#12)

* dymf tmp

* add dymf tmp

* local test change

* pull thread pool

* fix conflict

* delete unuse log

* local change for mirrow 0

* fix dymf

* code clean

* fix code clean

* code clean

* code clean

* fix dymf

* fix dymf

* add endpass optimize

* clean code

* fix endpass optimize

* fix

* fix

* fix eigvals_op

* merge pre-stable

* merge pre-stable

Co-authored-by: yaoxuefeng6 <yaoxuefeng@baidu.com>
Co-authored-by: Thunderbrook <a754913769@163.com>

* test

* passid memory && Generalization

* fix code style

Co-authored-by: xionglei1234 <105704175+xionglei1234@users.noreply.github.com>
Co-authored-by: rensilin <rensilin@baidu.com>
Co-authored-by: root <root@yq01-sys-hic-k8s-v100-box-a225-0693.yq01.baidu.com>
Co-authored-by: zmxdream <zhangminxu01@baidu.com>
Co-authored-by: yaoxuefeng6 <yaoxuefeng@baidu.com>
Co-authored-by: Thunderbrook <a754913769@163.com>
Co-authored-by: Thunderbrook <52529258+Thunderbrook@users.noreply.github.com>
Co-authored-by: liaoxiaochao <liaoxiaochao@baidu.com>
jack603047588 referenced this issue in jack603047588/Paddle Nov 9, 2022
@qingshui
Merge pull request #12 from songyuwen0808/paddlebox
c53658f 
adapt to feabox MPI mode
jack603047588 referenced this issue in jack603047588/Paddle Nov 9, 2022
@qingshui
Merge pull request #12 from qingshui/paddlebox
026d37d 
add enable_dense_nccl_barrier flags and add cuda_check remove threadpool catch exception
jack603047588 referenced this issue in jack603047588/Paddle Nov 9, 2022
@qingshui
Merge pull request #27 from jiaoxuewu/paddlebox
5fed723 
Merge pull request #12 from qingshui/paddlebox
@marsbzp marsbzp mentioned this issue Jan 11, 2023
Open
qizhaoaoe pushed a commit to qizhaoaoe/Paddle that referenced this issue Mar 3, 2023
remove blanks in statements (PaddlePaddle#12)
fbb20a3
@chlyzzo chlyzzo mentioned this issue Mar 29, 2023
Closed
feifei-111 referenced this issue in feifei-111/Paddle Jun 26, 2023
@2742195759
Merge pull request #12 from NotHaozi/pr54202
948788b 
Fix unitests
lizexu123 pushed a commit to lizexu123/Paddle that referenced this issue Feb 23, 2024
@qingqing01
Add algorithm docs (PaddlePaddle#12)
939afdc 
* Move algorithm doc into PaddleSlim
* Update docs/mkdocs.yml
NKNaN pushed a commit to NKNaN/Paddle that referenced this issue Mar 3, 2024
@ekelsen
Merge pull request PaddlePaddle#12 from ekelsen/master
32af11f 
fix overzealous link
Fridge003 pushed a commit to Fridge003/Paddle that referenced this issue Apr 28, 2024
@feifei-111
Merge pull request PaddlePaddle#12 from Fridge003/multi-down
4eda8bc
wwbitejotunn pushed a commit to wwbitejotunn/Paddle that referenced this issue Jun 21, 2024
@Xreki
Merge pull request PaddlePaddle#12 from umiswing/bump_to_fa2
c585eb3
lizexu123 added a commit to lizexu123/Paddle that referenced this issue Aug 5, 2024
@lizexu123
Merge pull request PaddlePaddle#12 from lizexu123/add_trt
33548c5 
实现了pd_op.concat,并修改了converter.py对builtin.combine的处理
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@reyoung reyoung

Labels
duplicate
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@reyoung @stoneyang @gangliao @kitianFresh @elvinpoon