Merge branch 'main' of https://github.com/alibaba/DeepRec into featur…

…es/gpu_embedding_fusion
DeepRec-AI · nvzhou · Jan 10, 2022 · Jan 10, 2022 · Jan 10, 2022 · Jan 14, 2022
commit 4dfc639113fb8fc84b98cea738a663068812bd17
diff --git a/README.md b/README.md
@@ -43,24 +43,12 @@ DeepRec has super large-scale distributed training capability, supporting model
 
 **CPU Platform**
 
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-dev-cpu-py36-ubuntu18.04
-```
-
-Docker Hub repository
-
 ``````
 alideeprec/deeprec-build:deeprec-dev-cpu-py36-ubuntu18.04
 ``````
 
 **GPU Platform**
 
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-dev-gpu-py36-cu110-ubuntu18.04
-```
-
-Docker Hub repository
-
 ```
 alideeprec/deeprec-build:deeprec-dev-gpu-py36-cu110-ubuntu18.04
 ```
@@ -100,19 +88,11 @@ $ pip3 install /tmp/tensorflow_pkg/tensorflow-1.15.5+${version}-cp36-cp36m-linux
 
 #### Image for CPU
 
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec2206-cpu-py36-ubuntu18.04
-```
-Docker Hub repository
 ```
 alideeprec/deeprec-release:deeprec2206-cpu-py36-ubuntu18.04
 ```
 
 #### Image for GPU CUDA11.0
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec2206-gpu-py36-cu110-ubuntu18.04
-```
-Docker Hub repository
 
 ```
 alideeprec/deeprec-release:deeprec2206-gpu-py36-cu110-ubuntu18.04

diff --git a/sparse_operation_kit/.gitignore → addons/sparse_operation_kit/.gitignore b/sparse_operation_kit/.gitignore → addons/sparse_operation_kit/.gitignore
diff --git a/sparse_operation_kit/.pypirc → addons/sparse_operation_kit/.pypirc b/sparse_operation_kit/.pypirc → addons/sparse_operation_kit/.pypirc
diff --git a/sparse_operation_kit/CMakeLists.txt → addons/sparse_operation_kit/CMakeLists.txt b/sparse_operation_kit/CMakeLists.txt → addons/sparse_operation_kit/CMakeLists.txt
diff --git a/sparse_operation_kit/MANIFEST.in → addons/sparse_operation_kit/MANIFEST.in b/sparse_operation_kit/MANIFEST.in → addons/sparse_operation_kit/MANIFEST.in
diff --git a/sparse_operation_kit/ReadMe.md → addons/sparse_operation_kit/ReadMe.md b/sparse_operation_kit/ReadMe.md → addons/sparse_operation_kit/ReadMe.md
diff --git a/sparse_operation_kit/cmakes/FindNCCL.cmake → ...parse_operation_kit/cmakes/FindNCCL.cmake b/sparse_operation_kit/cmakes/FindNCCL.cmake → ...parse_operation_kit/cmakes/FindNCCL.cmake
diff --git a/sparse_operation_kit/cmakes/FindNVTX.cmake → ...parse_operation_kit/cmakes/FindNVTX.cmake b/sparse_operation_kit/cmakes/FindNVTX.cmake → ...parse_operation_kit/cmakes/FindNVTX.cmake
diff --git a/...operation_kit/cmakes/FindTensorFlow.cmake → ...operation_kit/cmakes/FindTensorFlow.cmake b/...operation_kit/cmakes/FindTensorFlow.cmake → ...operation_kit/cmakes/FindTensorFlow.cmake
diff --git a/sparse_operation_kit/documents/Makefile → ...s/sparse_operation_kit/documents/Makefile b/sparse_operation_kit/documents/Makefile → ...s/sparse_operation_kit/documents/Makefile
diff --git a/sparse_operation_kit/documents/ReadMe.md → .../sparse_operation_kit/documents/ReadMe.md b/sparse_operation_kit/documents/ReadMe.md → .../sparse_operation_kit/documents/ReadMe.md
diff --git a/sparse_operation_kit/documents/make.bat → ...s/sparse_operation_kit/documents/make.bat b/sparse_operation_kit/documents/make.bat → ...s/sparse_operation_kit/documents/make.bat
diff --git a/...ion_kit/documents/source/_static/.gitkeep → ...ion_kit/documents/source/_static/.gitkeep b/...ion_kit/documents/source/_static/.gitkeep → ...ion_kit/documents/source/_static/.gitkeep
diff --git a/...documents/source/_templates/versions.html → ...documents/source/_templates/versions.html b/...documents/source/_templates/versions.html → ...documents/source/_templates/versions.html
diff --git a/...s/source/api/embeddings/dense/all2all.rst → ...s/source/api/embeddings/dense/all2all.rst b/...s/source/api/embeddings/dense/all2all.rst → ...s/source/api/embeddings/dense/all2all.rst
diff --git a/...nts/source/api/embeddings/dense/index.rst → ...nts/source/api/embeddings/dense/index.rst b/...nts/source/api/embeddings/dense/index.rst → ...nts/source/api/embeddings/dense/index.rst
diff --git a/...documents/source/api/embeddings/index.rst → ...documents/source/api/embeddings/index.rst b/...documents/source/api/embeddings/index.rst → ...documents/source/api/embeddings/index.rst
diff --git a/...documents/source/api/embeddings/saver.rst → ...documents/source/api/embeddings/saver.rst b/...documents/source/api/embeddings/saver.rst → ...documents/source/api/embeddings/saver.rst
diff --git a/...rce/api/embeddings/sparse/distributed.rst → ...rce/api/embeddings/sparse/distributed.rst b/...rce/api/embeddings/sparse/distributed.rst → ...rce/api/embeddings/sparse/distributed.rst
diff --git a/...ts/source/api/embeddings/sparse/index.rst → ...ts/source/api/embeddings/sparse/index.rst b/...ts/source/api/embeddings/sparse/index.rst → ...ts/source/api/embeddings/sparse/index.rst
diff --git a/...i/embeddings/tf_distributed_embedding.rst → ...i/embeddings/tf_distributed_embedding.rst b/...i/embeddings/tf_distributed_embedding.rst → ...i/embeddings/tf_distributed_embedding.rst
diff --git a/...ration_kit/documents/source/api/index.rst → ...ration_kit/documents/source/api/index.rst b/...ration_kit/documents/source/api/index.rst → ...ration_kit/documents/source/api/index.rst
diff --git a/...eration_kit/documents/source/api/init.rst → ...eration_kit/documents/source/api/init.rst b/...eration_kit/documents/source/api/init.rst → ...eration_kit/documents/source/api/init.rst
diff --git a/.../documents/source/api/optimizers/opts.rst → .../documents/source/api/optimizers/opts.rst b/.../documents/source/api/optimizers/opts.rst → .../documents/source/api/optimizers/opts.rst
diff --git a/..._kit/documents/source/api/utils/index.rst → ..._kit/documents/source/api/utils/index.rst b/..._kit/documents/source/api/utils/index.rst → ..._kit/documents/source/api/utils/index.rst
diff --git a/.../documents/source/api/utils/opt_scope.rst → .../documents/source/api/utils/opt_scope.rst b/.../documents/source/api/utils/opt_scope.rst → .../documents/source/api/utils/opt_scope.rst
diff --git a/.../documents/source/api/utils/opt_utils.rst → .../documents/source/api/utils/opt_utils.rst b/.../documents/source/api/utils/opt_utils.rst → .../documents/source/api/utils/opt_utils.rst
diff --git a/...se_operation_kit/documents/source/conf.py → ...se_operation_kit/documents/source/conf.py b/...se_operation_kit/documents/source/conf.py → ...se_operation_kit/documents/source/conf.py
diff --git a/...kit/documents/source/env_vars/env_vars.md → ...kit/documents/source/env_vars/env_vars.md b/...kit/documents/source/env_vars/env_vars.md → ...kit/documents/source/env_vars/env_vars.md
diff --git a/...t/documents/source/examples/dense_demo.md → ...t/documents/source/examples/dense_demo.md b/...t/documents/source/examples/dense_demo.md → ...t/documents/source/examples/dense_demo.md
diff --git a/...ion_kit/documents/source/examples/dlrm.md → ...ion_kit/documents/source/examples/dlrm.md b/...ion_kit/documents/source/examples/dlrm.md → ...ion_kit/documents/source/examples/dlrm.md
diff --git a/...n_kit/documents/source/examples/index.rst → ...n_kit/documents/source/examples/index.rst b/...n_kit/documents/source/examples/index.rst → ...n_kit/documents/source/examples/index.rst
diff --git a/...kit/documents/source/features/features.md → ...kit/documents/source/features/features.md b/...kit/documents/source/features/features.md → ...kit/documents/source/features/features.md
diff --git a/...cuments/source/get_started/get_started.md → ...cuments/source/get_started/get_started.md b/...cuments/source/get_started/get_started.md → ...cuments/source/get_started/get_started.md
diff --git a/...source/images/all2all_dense_embedding.png → ...source/images/all2all_dense_embedding.png b/...source/images/all2all_dense_embedding.png → ...source/images/all2all_dense_embedding.png
diff --git a/...ts/source/images/demo_model_structure.png → ...ts/source/images/demo_model_structure.png b/...ts/source/images/demo_model_structure.png → ...ts/source/images/demo_model_structure.png
diff --git a/...e/images/distributed_sparse_embedding.png → ...e/images/distributed_sparse_embedding.png b/...e/images/distributed_sparse_embedding.png → ...e/images/distributed_sparse_embedding.png
diff --git a/...rce/images/workflow_of_embeddinglayer.png → ...rce/images/workflow_of_embeddinglayer.png b/...rce/images/workflow_of_embeddinglayer.png → ...rce/images/workflow_of_embeddinglayer.png
diff --git a/..._operation_kit/documents/source/index.rst → ..._operation_kit/documents/source/index.rst b/..._operation_kit/documents/source/index.rst → ..._operation_kit/documents/source/index.rst
diff --git a/...ration_kit/documents/source/intro_link.md → ...ration_kit/documents/source/intro_link.md b/...ration_kit/documents/source/intro_link.md → ...ration_kit/documents/source/intro_link.md
diff --git a/...t/documents/source/known_issues/issues.md → ...t/documents/source/known_issues/issues.md b/...t/documents/source/known_issues/issues.md → ...t/documents/source/known_issues/issues.md
diff --git a/...ocuments/source/performance/dense_demo.md → ...ocuments/source/performance/dense_demo.md b/...ocuments/source/performance/dense_demo.md → ...ocuments/source/performance/dense_demo.md
diff --git a/...it/documents/source/performance/index.rst → ...it/documents/source/performance/index.rst b/...it/documents/source/performance/index.rst → ...it/documents/source/performance/index.rst
diff --git a/...nts/source/release_notes/release_notes.md → ...nts/source/release_notes/release_notes.md b/...nts/source/release_notes/release_notes.md → ...nts/source/release_notes/release_notes.md
diff --git a/...se_operation_kit/documents/source/util.py → ...se_operation_kit/documents/source/util.py b/...se_operation_kit/documents/source/util.py → ...se_operation_kit/documents/source/util.py
diff --git a/...on_kit/documents/tutorials/DLRM/ReadMe.md → ...on_kit/documents/tutorials/DLRM/ReadMe.md b/...on_kit/documents/tutorials/DLRM/ReadMe.md → ...on_kit/documents/tutorials/DLRM/ReadMe.md
diff --git a/modelzoo/SOK/DLRM/data/bin2csv.py → ...n_kit/documents/tutorials/DLRM/bin2csv.py b/modelzoo/SOK/DLRM/data/bin2csv.py → ...n_kit/documents/tutorials/DLRM/bin2csv.py
diff --git a/...n_kit/documents/tutorials/DLRM/dataset.py → ...n_kit/documents/tutorials/DLRM/dataset.py b/...n_kit/documents/tutorials/DLRM/dataset.py → ...n_kit/documents/tutorials/DLRM/dataset.py
diff --git a/...tion_kit/documents/tutorials/DLRM/main.py → ...tion_kit/documents/tutorials/DLRM/main.py b/...tion_kit/documents/tutorials/DLRM/main.py → ...tion_kit/documents/tutorials/DLRM/main.py
diff --git a/...on_kit/documents/tutorials/DLRM/models.py → ...on_kit/documents/tutorials/DLRM/models.py b/...on_kit/documents/tutorials/DLRM/models.py → ...on_kit/documents/tutorials/DLRM/models.py
diff --git a/...ion_kit/documents/tutorials/DLRM/utils.py → ...ion_kit/documents/tutorials/DLRM/utils.py b/...ion_kit/documents/tutorials/DLRM/utils.py → ...ion_kit/documents/tutorials/DLRM/utils.py
diff --git a/...t/documents/tutorials/DenseDemo/ReadMe.md → ...t/documents/tutorials/DenseDemo/ReadMe.md b/...t/documents/tutorials/DenseDemo/ReadMe.md → ...t/documents/tutorials/DenseDemo/ReadMe.md
diff --git a/...documents/tutorials/DenseDemo/gen_data.py → ...documents/tutorials/DenseDemo/gen_data.py b/...documents/tutorials/DenseDemo/gen_data.py → ...documents/tutorials/DenseDemo/gen_data.py
diff --git a/...t/documents/tutorials/DenseDemo/models.py → ...t/documents/tutorials/DenseDemo/models.py b/...t/documents/tutorials/DenseDemo/models.py → ...t/documents/tutorials/DenseDemo/models.py
diff --git a/...als/DenseDemo/run_sok_MirroredStrategy.py → ...als/DenseDemo/run_sok_MirroredStrategy.py b/...als/DenseDemo/run_sok_MirroredStrategy.py → ...als/DenseDemo/run_sok_MirroredStrategy.py
diff --git a/...ials/DenseDemo/run_sok_MultiWorker_mpi.py → ...ials/DenseDemo/run_sok_MultiWorker_mpi.py b/...ials/DenseDemo/run_sok_MultiWorker_mpi.py → ...ials/DenseDemo/run_sok_MultiWorker_mpi.py
diff --git a/...ts/tutorials/DenseDemo/run_sok_horovod.py → ...ts/tutorials/DenseDemo/run_sok_horovod.py b/...ts/tutorials/DenseDemo/run_sok_horovod.py → ...ts/tutorials/DenseDemo/run_sok_horovod.py
diff --git a/...t/documents/tutorials/DenseDemo/run_tf.py → ...t/documents/tutorials/DenseDemo/run_tf.py b/...t/documents/tutorials/DenseDemo/run_tf.py → ...t/documents/tutorials/DenseDemo/run_tf.py
diff --git a/...cuments/tutorials/DenseDemo/split_data.py → ...cuments/tutorials/DenseDemo/split_data.py b/...cuments/tutorials/DenseDemo/split_data.py → ...cuments/tutorials/DenseDemo/split_data.py
diff --git a/.../documents/tutorials/SparseDemo/ReadMe.md → .../documents/tutorials/SparseDemo/ReadMe.md b/.../documents/tutorials/SparseDemo/ReadMe.md → .../documents/tutorials/SparseDemo/ReadMe.md
diff --git a/...ration_kit/documents/tutorials/utility.py → ...ration_kit/documents/tutorials/utility.py b/...ration_kit/documents/tutorials/utility.py → ...ration_kit/documents/tutorials/utility.py
diff --git a/sparse_operation_kit/install.sh → addons/sparse_operation_kit/install.sh b/sparse_operation_kit/install.sh → addons/sparse_operation_kit/install.sh
diff --git a/...ork/compat/kernels/gpu_device_functions.h → ...ork/compat/kernels/gpu_device_functions.h b/...ork/compat/kernels/gpu_device_functions.h → ...ork/compat/kernels/gpu_device_functions.h
diff --git a/...mework/compat/kernels/gpu_kernel_helper.h → ...mework/compat/kernels/gpu_kernel_helper.h b/...mework/compat/kernels/gpu_kernel_helper.h → ...mework/compat/kernels/gpu_kernel_helper.h
diff --git a/...kit_cc/framework/compat/kernels/unique.cu → ...kit_cc/framework/compat/kernels/unique.cu b/...kit_cc/framework/compat/kernels/unique.cu → ...kit_cc/framework/compat/kernels/unique.cu
diff --git a/...rk/compat/kernels/unsorted_segment_sum.cu → ...rk/compat/kernels/unsorted_segment_sum.cu b/...rk/compat/kernels/unsorted_segment_sum.cu → ...rk/compat/kernels/unsorted_segment_sum.cu
diff --git a/...kit/kit_cc/framework/compat/ops/unique.cc → ...kit/kit_cc/framework/compat/ops/unique.cc b/...kit/kit_cc/framework/compat/ops/unique.cc → ...kit/kit_cc/framework/compat/ops/unique.cc
diff --git a/...mework/compat/ops/unsorted_segment_sum.cc → ...mework/compat/ops/unsorted_segment_sum.cc b/...mework/compat/ops/unsorted_segment_sum.cc → ...mework/compat/ops/unsorted_segment_sum.cc
diff --git a/...k/kernels/assign_embedding_variable_op.cc → ...k/kernels/assign_embedding_variable_op.cc b/...k/kernels/assign_embedding_variable_op.cc → ...k/kernels/assign_embedding_variable_op.cc
diff --git a/...ion_kit/kit_cc/framework/kernels/bprop.cc → ...ion_kit/kit_cc/framework/kernels/bprop.cc b/...ion_kit/kit_cc/framework/kernels/bprop.cc → ...ion_kit/kit_cc/framework/kernels/bprop.cc
diff --git a/...amework/kernels/create_embedding_dense.cc → ...amework/kernels/create_embedding_dense.cc b/...amework/kernels/create_embedding_dense.cc → ...amework/kernels/create_embedding_dense.cc
diff --git a/...mework/kernels/create_embedding_sparse.cc → ...mework/kernels/create_embedding_sparse.cc b/...mework/kernels/create_embedding_sparse.cc → ...mework/kernels/create_embedding_sparse.cc
diff --git a/...k/kernels/create_global_adam_optimizer.cc → ...k/kernels/create_global_adam_optimizer.cc b/...k/kernels/create_global_adam_optimizer.cc → ...k/kernels/create_global_adam_optimizer.cc
diff --git a/...it/kit_cc/framework/kernels/create_var.cc → ...it/kit_cc/framework/kernels/create_var.cc b/...it/kit_cc/framework/kernels/create_var.cc → ...it/kit_cc/framework/kernels/create_var.cc
diff --git a/...rnels/custom_optimizer_apply_gradients.cc → ...rnels/custom_optimizer_apply_gradients.cc b/...rnels/custom_optimizer_apply_gradients.cc → ...rnels/custom_optimizer_apply_gradients.cc
diff --git a/...t/kit_cc/framework/kernels/dense_fprop.cc → ...t/kit_cc/framework/kernels/dense_fprop.cc b/...t/kit_cc/framework/kernels/dense_fprop.cc → ...t/kit_cc/framework/kernels/dense_fprop.cc
diff --git a/.../kit_cc/framework/kernels/dump_to_file.cc → .../kit_cc/framework/kernels/dump_to_file.cc b/.../kit_cc/framework/kernels/dump_to_file.cc → .../kit_cc/framework/kernels/dump_to_file.cc
diff --git a/...c/framework/kernels/embedding_variable.cc → ...c/framework/kernels/embedding_variable.cc b/...c/framework/kernels/embedding_variable.cc → ...c/framework/kernels/embedding_variable.cc
diff --git a/...cc/framework/kernels/embedding_variable.h → ...cc/framework/kernels/embedding_variable.h b/...cc/framework/kernels/embedding_variable.h → ...cc/framework/kernels/embedding_variable.h
diff --git a/.../kernels/embedding_variable_assign_sub.cu → .../kernels/embedding_variable_assign_sub.cu b/.../kernels/embedding_variable_assign_sub.cu → .../kernels/embedding_variable_assign_sub.cu
diff --git a/...t_cc/framework/kernels/gen_random_seed.cc → ...t_cc/framework/kernels/gen_random_seed.cc b/...t_cc/framework/kernels/gen_random_seed.cc → ...t_cc/framework/kernels/gen_random_seed.cc
diff --git a/...c/framework/kernels/get_nccl_unique_id.cc → ...c/framework/kernels/get_nccl_unique_id.cc b/...c/framework/kernels/get_nccl_unique_id.cc → ...c/framework/kernels/get_nccl_unique_id.cc
diff --git a/...ramework/kernels/load_embedding_values.cc → ...ramework/kernels/load_embedding_values.cc b/...ramework/kernels/load_embedding_values.cc → ...ramework/kernels/load_embedding_values.cc
diff --git a/...it_cc/framework/kernels/optimizer_init.cc → ...it_cc/framework/kernels/optimizer_init.cc b/...it_cc/framework/kernels/optimizer_init.cc → ...it_cc/framework/kernels/optimizer_init.cc
diff --git a/...t/kit_cc/framework/kernels/plugin_init.cc → ...t/kit_cc/framework/kernels/plugin_init.cc b/...t/kit_cc/framework/kernels/plugin_init.cc → ...t/kit_cc/framework/kernels/plugin_init.cc
diff --git a/...ork/kernels/read_embedding_variable_op.cc → ...ork/kernels/read_embedding_variable_op.cc b/...ork/kernels/read_embedding_variable_op.cc → ...ork/kernels/read_embedding_variable_op.cc
diff --git a/...cc/framework/kernels/restore_from_file.cc → ...cc/framework/kernels/restore_from_file.cc b/...cc/framework/kernels/restore_from_file.cc → ...cc/framework/kernels/restore_from_file.cc
diff --git a/.../kit_cc/framework/kernels/sparse_fprop.cc → .../kit_cc/framework/kernels/sparse_fprop.cc b/.../kit_cc/framework/kernels/sparse_fprop.cc → .../kit_cc/framework/kernels/sparse_fprop.cc
diff --git a/...n_kit/kit_cc/framework/kernels/test_op.cc → ...n_kit/kit_cc/framework/kernels/test_op.cc b/...n_kit/kit_cc/framework/kernels/test_op.cc → ...n_kit/kit_cc/framework/kernels/test_op.cc
diff --git a/...n_kit/kit_cc/framework/kernels/test_op.cu → ...n_kit/kit_cc/framework/kernels/test_op.cu b/...n_kit/kit_cc/framework/kernels/test_op.cu → ...n_kit/kit_cc/framework/kernels/test_op.cu
diff --git a/...ework/ops/assign_embedding_variable_op.cc → ...ework/ops/assign_embedding_variable_op.cc b/...ework/ops/assign_embedding_variable_op.cc → ...ework/ops/assign_embedding_variable_op.cc
diff --git a/...eration_kit/kit_cc/framework/ops/bprop.cc → ...eration_kit/kit_cc/framework/ops/bprop.cc b/...eration_kit/kit_cc/framework/ops/bprop.cc → ...eration_kit/kit_cc/framework/ops/bprop.cc
diff --git a/...c/framework/ops/create_embedding_dense.cc → ...c/framework/ops/create_embedding_dense.cc b/...c/framework/ops/create_embedding_dense.cc → ...c/framework/ops/create_embedding_dense.cc
diff --git a/.../framework/ops/create_embedding_sparse.cc → .../framework/ops/create_embedding_sparse.cc b/.../framework/ops/create_embedding_sparse.cc → .../framework/ops/create_embedding_sparse.cc
diff --git a/...ework/ops/create_global_adam_optimizer.cc → ...ework/ops/create_global_adam_optimizer.cc b/...ework/ops/create_global_adam_optimizer.cc → ...ework/ops/create_global_adam_optimizer.cc
diff --git a/...on_kit/kit_cc/framework/ops/create_var.cc → ...on_kit/kit_cc/framework/ops/create_var.cc b/...on_kit/kit_cc/framework/ops/create_var.cc → ...on_kit/kit_cc/framework/ops/create_var.cc
diff --git a/...k/ops/custom_optimizer_apply_gradients.cc → ...k/ops/custom_optimizer_apply_gradients.cc b/...k/ops/custom_optimizer_apply_gradients.cc → ...k/ops/custom_optimizer_apply_gradients.cc
diff --git a/...n_kit/kit_cc/framework/ops/dense_fprop.cc → ...n_kit/kit_cc/framework/ops/dense_fprop.cc b/...n_kit/kit_cc/framework/ops/dense_fprop.cc → ...n_kit/kit_cc/framework/ops/dense_fprop.cc
diff --git a/..._kit/kit_cc/framework/ops/dump_to_file.cc → ..._kit/kit_cc/framework/ops/dump_to_file.cc b/..._kit/kit_cc/framework/ops/dump_to_file.cc → ..._kit/kit_cc/framework/ops/dump_to_file.cc
diff --git a/...work/ops/embedding_variable_assign_sub.cc → ...work/ops/embedding_variable_assign_sub.cc b/...work/ops/embedding_variable_assign_sub.cc → ...work/ops/embedding_variable_assign_sub.cc
diff --git a/...peration_kit/kit_cc/framework/ops/init.cc → ...peration_kit/kit_cc/framework/ops/init.cc b/...peration_kit/kit_cc/framework/ops/init.cc → ...peration_kit/kit_cc/framework/ops/init.cc
diff --git a/...cc/framework/ops/load_embedding_values.cc → ...cc/framework/ops/load_embedding_values.cc b/...cc/framework/ops/load_embedding_values.cc → ...cc/framework/ops/load_embedding_values.cc
diff --git a/...it/kit_cc/framework/ops/optimizer_init.cc → ...it/kit_cc/framework/ops/optimizer_init.cc b/...it/kit_cc/framework/ops/optimizer_init.cc → ...it/kit_cc/framework/ops/optimizer_init.cc
diff --git a/...amework/ops/read_embedding_variable_op.cc → ...amework/ops/read_embedding_variable_op.cc b/...amework/ops/read_embedding_variable_op.cc → ...amework/ops/read_embedding_variable_op.cc
diff --git a/...kit_cc/framework/ops/restore_from_file.cc → ...kit_cc/framework/ops/restore_from_file.cc b/...kit_cc/framework/ops/restore_from_file.cc → ...kit_cc/framework/ops/restore_from_file.cc
diff --git a/..._kit/kit_cc/framework/ops/sparse_fprop.cc → ..._kit/kit_cc/framework/ops/sparse_fprop.cc b/..._kit/kit_cc/framework/ops/sparse_fprop.cc → ..._kit/kit_cc/framework/ops/sparse_fprop.cc
diff --git a/...ation_kit/kit_cc/framework/ops/test_op.cc → ...ation_kit/kit_cc/framework/ops/test_op.cc b/...ation_kit/kit_cc/framework/ops/test_op.cc → ...ation_kit/kit_cc/framework/ops/test_op.cc
diff --git a/..._kit/kit_cc/kit_cc_infra/include/.gitkeep → ..._kit/kit_cc/kit_cc_infra/include/.gitkeep b/..._kit/kit_cc/kit_cc_infra/include/.gitkeep → ..._kit/kit_cc/kit_cc_infra/include/.gitkeep
diff --git a/...it/kit_cc/kit_cc_infra/include/common.cuh → ...it/kit_cc/kit_cc_infra/include/common.cuh b/...it/kit_cc/kit_cc_infra/include/common.cuh → ...it/kit_cc/kit_cc_infra/include/common.cuh
diff --git a/..._kit/kit_cc/kit_cc_infra/include/common.h → ..._kit/kit_cc/kit_cc_infra/include/common.h b/..._kit/kit_cc/kit_cc_infra/include/common.h → ..._kit/kit_cc/kit_cc_infra/include/common.h
diff --git a/...a/include/dispatcher/dispatcher_builder.h → ...a/include/dispatcher/dispatcher_builder.h b/...a/include/dispatcher/dispatcher_builder.h → ...a/include/dispatcher/dispatcher_builder.h
diff --git a/...nfra/include/embeddings/embedding_layer.h → ...nfra/include/embeddings/embedding_layer.h b/...nfra/include/embeddings/embedding_layer.h → ...nfra/include/embeddings/embedding_layer.h
diff --git a/...a/include/embeddings/embedding_lookuper.h → ...a/include/embeddings/embedding_lookuper.h b/...a/include/embeddings/embedding_lookuper.h → ...a/include/embeddings/embedding_lookuper.h
diff --git a/...e/embeddings/embedding_lookuper_builder.h → ...e/embeddings/embedding_lookuper_builder.h b/...e/embeddings/embedding_lookuper_builder.h → ...e/embeddings/embedding_lookuper_builder.h
diff --git a/...kit_cc_infra/include/embeddings/manager.h → ...kit_cc_infra/include/embeddings/manager.h b/...kit_cc_infra/include/embeddings/manager.h → ...kit_cc_infra/include/embeddings/manager.h
diff --git a/..._kit/kit_cc/kit_cc_infra/include/facade.h → ..._kit/kit_cc/kit_cc_infra/include/facade.h b/..._kit/kit_cc/kit_cc_infra/include/facade.h → ..._kit/kit_cc/kit_cc_infra/include/facade.h
diff --git a/...t_cc_infra/include/hashtable/cudf/LICENSE → ...t_cc_infra/include/hashtable/cudf/LICENSE b/...t_cc_infra/include/hashtable/cudf/LICENSE → ...t_cc_infra/include/hashtable/cudf/LICENSE
diff --git a/...shtable/cudf/concurrent_unordered_map.cuh → ...shtable/cudf/concurrent_unordered_map.cuh b/...shtable/cudf/concurrent_unordered_map.cuh → ...shtable/cudf/concurrent_unordered_map.cuh
diff --git a/...include/hashtable/cudf/hash_functions.cuh → ...include/hashtable/cudf/hash_functions.cuh b/...include/hashtable/cudf/hash_functions.cuh → ...include/hashtable/cudf/hash_functions.cuh
diff --git a/..._infra/include/hashtable/cudf/managed.cuh → ..._infra/include/hashtable/cudf/managed.cuh b/..._infra/include/hashtable/cudf/managed.cuh → ..._infra/include/hashtable/cudf/managed.cuh
diff --git a/...lude/hashtable/cudf/managed_allocator.cuh → ...lude/hashtable/cudf/managed_allocator.cuh b/...lude/hashtable/cudf/managed_allocator.cuh → ...lude/hashtable/cudf/managed_allocator.cuh
diff --git a/...it_cc_infra/include/hashtable/hashtable.h → ...it_cc_infra/include/hashtable/hashtable.h b/...it_cc_infra/include/hashtable/hashtable.h → ...it_cc_infra/include/hashtable/hashtable.h
diff --git a/...ra/include/hashtable/identity_hashtable.h → ...ra/include/hashtable/identity_hashtable.h b/...ra/include/hashtable/identity_hashtable.h → ...ra/include/hashtable/identity_hashtable.h
diff --git a/...cc_infra/include/hashtable/nv_hashtable.h → ...cc_infra/include/hashtable/nv_hashtable.h b/...cc_infra/include/hashtable/nv_hashtable.h → ...cc_infra/include/hashtable/nv_hashtable.h
diff --git a/..._infra/include/hashtable/nv_hashtable.hpp → ..._infra/include/hashtable/nv_hashtable.hpp b/..._infra/include/hashtable/nv_hashtable.hpp → ..._infra/include/hashtable/nv_hashtable.hpp
diff --git a/...nfra/include/hashtable/simple_hashtable.h → ...nfra/include/hashtable/simple_hashtable.h b/...nfra/include/hashtable/simple_hashtable.h → ...nfra/include/hashtable/simple_hashtable.h
diff --git a/...nclude/initializer/constant_initializer.h → ...nclude/initializer/constant_initializer.h b/...nclude/initializer/constant_initializer.h → ...nclude/initializer/constant_initializer.h
diff --git a/...clude/initializer/initializer_interface.h → ...clude/initializer/initializer_interface.h b/...clude/initializer/initializer_interface.h → ...clude/initializer/initializer_interface.h
diff --git a/...nfra/include/initializer/random_uniform.h → ...nfra/include/initializer/random_uniform.h b/...nfra/include/initializer/random_uniform.h → ...nfra/include/initializer/random_uniform.h
diff --git a/...fra/include/operation/builder_container.h → ...fra/include/operation/builder_container.h b/...fra/include/operation/builder_container.h → ...fra/include/operation/builder_container.h
diff --git a/.../include/operation/construction_context.h → .../include/operation/construction_context.h b/.../include/operation/construction_context.h → .../include/operation/construction_context.h
diff --git a/...t_cc_infra/include/operation/op_context.h → ...t_cc_infra/include/operation/op_context.h b/...t_cc_infra/include/operation/op_context.h → ...t_cc_infra/include/operation/op_context.h
diff --git a/...it_cc_infra/include/operation/operation.h → ...it_cc_infra/include/operation/operation.h b/...it_cc_infra/include/operation/operation.h → ...it_cc_infra/include/operation/operation.h
diff --git a/...fra/include/operation/operation_builder.h → ...fra/include/operation/operation_builder.h b/...fra/include/operation/operation_builder.h → ...fra/include/operation/operation_builder.h
diff --git a/...nfra/include/operation/operation_helper.h → ...nfra/include/operation/operation_helper.h b/...nfra/include/operation/operation_helper.h → ...nfra/include/operation/operation_helper.h
diff --git a/...a/include/operation/operation_interface.h → ...a/include/operation/operation_interface.h b/...a/include/operation/operation_interface.h → ...a/include/operation/operation_interface.h
diff --git a/..._infra/include/optimizer/adam_optimizer.h → ..._infra/include/optimizer/adam_optimizer.h b/..._infra/include/optimizer/adam_optimizer.h → ..._infra/include/optimizer/adam_optimizer.h
diff --git a/.../include/optimizer/grad_update_preparer.h → .../include/optimizer/grad_update_preparer.h b/.../include/optimizer/grad_update_preparer.h → .../include/optimizer/grad_update_preparer.h
diff --git a/...a/include/optimizer/optimizer_interface.h → ...a/include/optimizer/optimizer_interface.h b/...a/include/optimizer/optimizer_interface.h → ...a/include/optimizer/optimizer_interface.h
diff --git a/...fra/include/optimizer/prepare_functions.h → ...fra/include/optimizer/prepare_functions.h b/...fra/include/optimizer/prepare_functions.h → ...fra/include/optimizer/prepare_functions.h
diff --git a/...nfra/include/optimizer/update_functions.h → ...nfra/include/optimizer/update_functions.h b/...nfra/include/optimizer/update_functions.h → ...nfra/include/optimizer/update_functions.h
diff --git a/...infra/include/optimizer/update_preparer.h → ...infra/include/optimizer/update_preparer.h b/...infra/include/optimizer/update_preparer.h → ...infra/include/optimizer/update_preparer.h
diff --git a/...ra/include/parameters/manager_interface.h → ...ra/include/parameters/manager_interface.h b/...ra/include/parameters/manager_interface.h → ...ra/include/parameters/manager_interface.h
diff --git a/...nfra/include/parameters/param_interface.h → ...nfra/include/parameters/param_interface.h b/...nfra/include/parameters/param_interface.h → ...nfra/include/parameters/param_interface.h
diff --git a/...cc_infra/include/parameters/raw_manager.h → ...cc_infra/include/parameters/raw_manager.h b/...cc_infra/include/parameters/raw_manager.h → ...cc_infra/include/parameters/raw_manager.h
diff --git a/...t_cc_infra/include/parameters/raw_param.h → ...t_cc_infra/include/parameters/raw_param.h b/...t_cc_infra/include/parameters/raw_param.h → ...t_cc_infra/include/parameters/raw_param.h
diff --git a/...t_cc_infra/include/parameters/raw_state.h → ...t_cc_infra/include/parameters/raw_state.h b/...t_cc_infra/include/parameters/raw_state.h → ...t_cc_infra/include/parameters/raw_state.h
diff --git a/...nfra/include/parameters/state_interface.h → ...nfra/include/parameters/state_interface.h b/...nfra/include/parameters/state_interface.h → ...nfra/include/parameters/state_interface.h
diff --git a/...cc_infra/include/resources/cpu_resource.h → ...cc_infra/include/resources/cpu_resource.h b/...cc_infra/include/resources/cpu_resource.h → ...cc_infra/include/resources/cpu_resource.h
diff --git a/...cc/kit_cc_infra/include/resources/event.h → ...cc/kit_cc_infra/include/resources/event.h b/...cc/kit_cc_infra/include/resources/event.h → ...cc/kit_cc_infra/include/resources/event.h
diff --git a/...c_infra/include/resources/event_manager.h → ...c_infra/include/resources/event_manager.h b/...c_infra/include/resources/event_manager.h → ...c_infra/include/resources/event_manager.h
diff --git a/...cc_infra/include/resources/gpu_resource.h → ...cc_infra/include/resources/gpu_resource.h b/...cc_infra/include/resources/gpu_resource.h → ...cc_infra/include/resources/gpu_resource.h
diff --git a/.../kit_cc_infra/include/resources/manager.h → .../kit_cc_infra/include/resources/manager.h b/.../kit_cc_infra/include/resources/manager.h → .../kit_cc_infra/include/resources/manager.h
diff --git a/..._cc_infra/include/resources/mpi_context.h → ..._cc_infra/include/resources/mpi_context.h b/..._cc_infra/include/resources/mpi_context.h → ..._cc_infra/include/resources/mpi_context.h
diff --git a/.../include/tensor_buffer/embedding_buffer.h → .../include/tensor_buffer/embedding_buffer.h b/.../include/tensor_buffer/embedding_buffer.h → .../include/tensor_buffer/embedding_buffer.h
diff --git a/...include/tensor_buffer/general_buffer2.hpp → ...include/tensor_buffer/general_buffer2.hpp b/...include/tensor_buffer/general_buffer2.hpp → ...include/tensor_buffer/general_buffer2.hpp
diff --git a/...c_infra/include/tensor_buffer/tensor2.hpp → ...c_infra/include/tensor_buffer/tensor2.hpp b/...c_infra/include/tensor_buffer/tensor2.hpp → ...c_infra/include/tensor_buffer/tensor2.hpp
diff --git a/...a/include/tensor_buffer/tensor2_wrapper.h → ...a/include/tensor_buffer/tensor2_wrapper.h b/...a/include/tensor_buffer/tensor2_wrapper.h → ...a/include/tensor_buffer/tensor2_wrapper.h
diff --git a/.../include/tensor_buffer/tensor_interface.h → .../include/tensor_buffer/tensor_interface.h b/.../include/tensor_buffer/tensor_interface.h → .../include/tensor_buffer/tensor_interface.h
diff --git a/...include/tensor_buffer/tf_tensor_wrapper.h → ...include/tensor_buffer/tf_tensor_wrapper.h b/...include/tensor_buffer/tf_tensor_wrapper.h → ...include/tensor_buffer/tf_tensor_wrapper.h
diff --git a/...tion_kit/kit_cc/kit_cc_infra/src/.gitkeep → ...tion_kit/kit_cc/kit_cc_infra/src/.gitkeep b/...tion_kit/kit_cc/kit_cc_infra/src/.gitkeep → ...tion_kit/kit_cc/kit_cc_infra/src/.gitkeep
diff --git a/...ion_kit/kit_cc/kit_cc_infra/src/common.cc → ...ion_kit/kit_cc/kit_cc_infra/src/common.cc b/...ion_kit/kit_cc/kit_cc_infra/src/common.cc → ...ion_kit/kit_cc/kit_cc_infra/src/common.cc
diff --git a/...ion_kit/kit_cc/kit_cc_infra/src/common.cu → ...ion_kit/kit_cc/kit_cc_infra/src/common.cu b/...ion_kit/kit_cc/kit_cc_infra/src/common.cu → ...ion_kit/kit_cc/kit_cc_infra/src/common.cu
diff --git a/...c_infra/src/embeddings/embedding_layer.cc → ...c_infra/src/embeddings/embedding_layer.cc b/...c_infra/src/embeddings/embedding_layer.cc → ...c_infra/src/embeddings/embedding_layer.cc
diff --git a/...nfra/src/embeddings/embedding_lookuper.cc → ...nfra/src/embeddings/embedding_lookuper.cc b/...nfra/src/embeddings/embedding_lookuper.cc → ...nfra/src/embeddings/embedding_lookuper.cc
diff --git a/...cc/kit_cc_infra/src/embeddings/manager.cc → ...cc/kit_cc_infra/src/embeddings/manager.cc b/...cc/kit_cc_infra/src/embeddings/manager.cc → ...cc/kit_cc_infra/src/embeddings/manager.cc
diff --git a/...ion_kit/kit_cc/kit_cc_infra/src/facade.cc → ...ion_kit/kit_cc/kit_cc_infra/src/facade.cc b/...ion_kit/kit_cc/kit_cc_infra/src/facade.cc → ...ion_kit/kit_cc/kit_cc_infra/src/facade.cc
diff --git a/...infra/src/hashtable/identity_hashtable.cu → ...infra/src/hashtable/identity_hashtable.cu b/...infra/src/hashtable/identity_hashtable.cu → ...infra/src/hashtable/identity_hashtable.cu
diff --git a/...it_cc_infra/src/hashtable/nv_hashtable.cc → ...it_cc_infra/src/hashtable/nv_hashtable.cc b/...it_cc_infra/src/hashtable/nv_hashtable.cc → ...it_cc_infra/src/hashtable/nv_hashtable.cc
diff --git a/...it_cc_infra/src/hashtable/nv_hashtable.cu → ...it_cc_infra/src/hashtable/nv_hashtable.cu b/...it_cc_infra/src/hashtable/nv_hashtable.cu → ...it_cc_infra/src/hashtable/nv_hashtable.cu
diff --git a/...c_infra/src/hashtable/simple_hashtable.cu → ...c_infra/src/hashtable/simple_hashtable.cu b/...c_infra/src/hashtable/simple_hashtable.cu → ...c_infra/src/hashtable/simple_hashtable.cu
diff --git a/...a/src/initializer/constant_initializer.cu → ...a/src/initializer/constant_initializer.cu b/...a/src/initializer/constant_initializer.cu → ...a/src/initializer/constant_initializer.cu
diff --git a/.../src/initializer/initializer_interface.cc → .../src/initializer/initializer_interface.cc b/.../src/initializer/initializer_interface.cc → .../src/initializer/initializer_interface.cc
diff --git a/...c_infra/src/initializer/random_uniform.cu → ...c_infra/src/initializer/random_uniform.cu b/...c_infra/src/initializer/random_uniform.cu → ...c_infra/src/initializer/random_uniform.cu
diff --git a/..._infra/src/operation/builder_container.cc → ..._infra/src/operation/builder_container.cc b/..._infra/src/operation/builder_container.cc → ..._infra/src/operation/builder_container.cc
diff --git a/...fra/src/operation/construction_context.cc → ...fra/src/operation/construction_context.cc b/...fra/src/operation/construction_context.cc → ...fra/src/operation/construction_context.cc
diff --git a/.../kit_cc_infra/src/operation/op_context.cc → .../kit_cc_infra/src/operation/op_context.cc b/.../kit_cc_infra/src/operation/op_context.cc → .../kit_cc_infra/src/operation/op_context.cc
diff --git a/...c/kit_cc_infra/src/operation/operation.cc → ...c/kit_cc_infra/src/operation/operation.cc b/...c/kit_cc_infra/src/operation/operation.cc → ...c/kit_cc_infra/src/operation/operation.cc
diff --git a/...c_infra/src/operation/operation_helper.cc → ...c_infra/src/operation/operation_helper.cc b/...c_infra/src/operation/operation_helper.cc → ...c_infra/src/operation/operation_helper.cc
diff --git a/..._cc_infra/src/optimizer/adam_optimizer.cc → ..._cc_infra/src/optimizer/adam_optimizer.cc b/..._cc_infra/src/optimizer/adam_optimizer.cc → ..._cc_infra/src/optimizer/adam_optimizer.cc
diff --git a/...fra/src/optimizer/grad_update_preparer.cc → ...fra/src/optimizer/grad_update_preparer.cc b/...fra/src/optimizer/grad_update_preparer.cc → ...fra/src/optimizer/grad_update_preparer.cc
diff --git a/...nfra/src/optimizer/optimizer_interface.cc → ...nfra/src/optimizer/optimizer_interface.cc b/...nfra/src/optimizer/optimizer_interface.cc → ...nfra/src/optimizer/optimizer_interface.cc
diff --git a/..._infra/src/optimizer/prepare_functions.cu → ..._infra/src/optimizer/prepare_functions.cu b/..._infra/src/optimizer/prepare_functions.cu → ..._infra/src/optimizer/prepare_functions.cu
diff --git a/...c_infra/src/optimizer/update_functions.cu → ...c_infra/src/optimizer/update_functions.cu b/...c_infra/src/optimizer/update_functions.cu → ...c_infra/src/optimizer/update_functions.cu
diff --git a/...infra/src/parameters/manager_interface.cc → ...infra/src/parameters/manager_interface.cc b/...infra/src/parameters/manager_interface.cc → ...infra/src/parameters/manager_interface.cc
diff --git a/...c_infra/src/parameters/param_interface.cc → ...c_infra/src/parameters/param_interface.cc b/...c_infra/src/parameters/param_interface.cc → ...c_infra/src/parameters/param_interface.cc
diff --git a/...it_cc_infra/src/parameters/raw_manager.cc → ...it_cc_infra/src/parameters/raw_manager.cc b/...it_cc_infra/src/parameters/raw_manager.cc → ...it_cc_infra/src/parameters/raw_manager.cc
diff --git a/.../kit_cc_infra/src/parameters/raw_param.cc → .../kit_cc_infra/src/parameters/raw_param.cc b/.../kit_cc_infra/src/parameters/raw_param.cc → .../kit_cc_infra/src/parameters/raw_param.cc
diff --git a/.../kit_cc_infra/src/parameters/raw_state.cc → .../kit_cc_infra/src/parameters/raw_state.cc b/.../kit_cc_infra/src/parameters/raw_state.cc → .../kit_cc_infra/src/parameters/raw_state.cc
diff --git a/...it_cc_infra/src/resources/cpu_resource.cc → ...it_cc_infra/src/resources/cpu_resource.cc b/...it_cc_infra/src/resources/cpu_resource.cc → ...it_cc_infra/src/resources/cpu_resource.cc
diff --git a/...it_cc/kit_cc_infra/src/resources/event.cc → ...it_cc/kit_cc_infra/src/resources/event.cc b/...it_cc/kit_cc_infra/src/resources/event.cc → ...it_cc/kit_cc_infra/src/resources/event.cc
diff --git a/...t_cc_infra/src/resources/event_manager.cc → ...t_cc_infra/src/resources/event_manager.cc b/...t_cc_infra/src/resources/event_manager.cc → ...t_cc_infra/src/resources/event_manager.cc
diff --git a/...it_cc_infra/src/resources/gpu_resource.cc → ...it_cc_infra/src/resources/gpu_resource.cc b/...it_cc_infra/src/resources/gpu_resource.cc → ...it_cc_infra/src/resources/gpu_resource.cc
diff --git a/..._cc/kit_cc_infra/src/resources/manager.cc → ..._cc/kit_cc_infra/src/resources/manager.cc b/..._cc/kit_cc_infra/src/resources/manager.cc → ..._cc/kit_cc_infra/src/resources/manager.cc
diff --git a/...kit_cc_infra/src/resources/mpi_context.cc → ...kit_cc_infra/src/resources/mpi_context.cc b/...kit_cc_infra/src/resources/mpi_context.cc → ...kit_cc_infra/src/resources/mpi_context.cc
diff --git a/...fra/src/tensor_buffer/embedding_buffer.cc → ...fra/src/tensor_buffer/embedding_buffer.cc b/...fra/src/tensor_buffer/embedding_buffer.cc → ...fra/src/tensor_buffer/embedding_buffer.cc
diff --git a/...ra/src/tensor_buffer/tf_tensor_wrapper.cc → ...ra/src/tensor_buffer/tf_tensor_wrapper.cc b/...ra/src/tensor_buffer/tf_tensor_wrapper.cc → ...ra/src/tensor_buffer/tf_tensor_wrapper.cc
diff --git a/...n/CXX11/src/ThreadPool/SimpleThreadPool.h → ...n/CXX11/src/ThreadPool/SimpleThreadPool.h b/...n/CXX11/src/ThreadPool/SimpleThreadPool.h → ...n/CXX11/src/ThreadPool/SimpleThreadPool.h
diff --git a/.../CXX11/src/ThreadPool/ThreadEnvironment.h → .../CXX11/src/ThreadPool/ThreadEnvironment.h b/.../CXX11/src/ThreadPool/ThreadEnvironment.h → .../CXX11/src/ThreadPool/ThreadEnvironment.h
diff --git a/...XX11/src/ThreadPool/ThreadPoolInterface.h → ...XX11/src/ThreadPool/ThreadPoolInterface.h b/...XX11/src/ThreadPool/ThreadPoolInterface.h → ...XX11/src/ThreadPool/ThreadPoolInterface.h
diff --git a/...ing/common/include/backward_functions.cuh → ...ing/common/include/backward_functions.cuh b/...ing/common/include/backward_functions.cuh → ...ing/common/include/backward_functions.cuh
diff --git a/...dding/common/include/backward_functions.h → ...dding/common/include/backward_functions.h b/...dding/common/include/backward_functions.h → ...dding/common/include/backward_functions.h
diff --git a/...ing/common/include/conversion_kernels.cuh → ...ing/common/include/conversion_kernels.cuh b/...ing/common/include/conversion_kernels.cuh → ...ing/common/include/conversion_kernels.cuh
diff --git a/...edding/common/include/dumping_functions.h → ...edding/common/include/dumping_functions.h b/...edding/common/include/dumping_functions.h → ...edding/common/include/dumping_functions.h
diff --git a/...ding/common/include/forward_functions.cuh → ...ding/common/include/forward_functions.cuh b/...ding/common/include/forward_functions.cuh → ...ding/common/include/forward_functions.cuh
diff --git a/...edding/common/include/forward_functions.h → ...edding/common/include/forward_functions.h b/...edding/common/include/forward_functions.h → ...edding/common/include/forward_functions.h
diff --git a/...mbedding/common/src/backward_functions.cu → ...mbedding/common/src/backward_functions.cu b/...mbedding/common/src/backward_functions.cu → ...mbedding/common/src/backward_functions.cu
diff --git a/...embedding/common/src/dumping_functions.cc → ...embedding/common/src/dumping_functions.cc b/...embedding/common/src/dumping_functions.cc → ...embedding/common/src/dumping_functions.cc
diff --git a/...embedding/common/src/dumping_functions.cu → ...embedding/common/src/dumping_functions.cu b/...embedding/common/src/dumping_functions.cu → ...embedding/common/src/dumping_functions.cu
diff --git a/...embedding/common/src/forward_functions.cu → ...embedding/common/src/forward_functions.cu b/...embedding/common/src/forward_functions.cu → ...embedding/common/src/forward_functions.cu
diff --git a/...ng/dispatcher/all2all_input_dispatcher.cu → ...ng/dispatcher/all2all_input_dispatcher.cu b/...ng/dispatcher/all2all_input_dispatcher.cu → ...ng/dispatcher/all2all_input_dispatcher.cu
diff --git a/...g/dispatcher/all2all_output_dispatcher.cu → ...g/dispatcher/all2all_output_dispatcher.cu b/...g/dispatcher/all2all_output_dispatcher.cu → ...g/dispatcher/all2all_output_dispatcher.cu
diff --git a/...dding/dispatcher/all_gather_dispatcher.cu → ...dding/dispatcher/all_gather_dispatcher.cu b/...dding/dispatcher/all_gather_dispatcher.cu → ...dding/dispatcher/all_gather_dispatcher.cu
diff --git a/...g/dispatcher/reduce_scatter_dispatcher.cu → ...g/dispatcher/reduce_scatter_dispatcher.cu b/...g/dispatcher/reduce_scatter_dispatcher.cu → ...g/dispatcher/reduce_scatter_dispatcher.cu
diff --git a/...c_impl/embedding/lookuper/dense_gather.cu → ...c_impl/embedding/lookuper/dense_gather.cu b/...c_impl/embedding/lookuper/dense_gather.cu → ...c_impl/embedding/lookuper/dense_gather.cu
diff --git a/...cc_impl/embedding/lookuper/distributed.cc → ...cc_impl/embedding/lookuper/distributed.cc b/...cc_impl/embedding/lookuper/distributed.cc → ...cc_impl/embedding/lookuper/distributed.cc
diff --git a/...g/operation/csr_conversion_distributed.cu → ...g/operation/csr_conversion_distributed.cu b/...g/operation/csr_conversion_distributed.cu → ...g/operation/csr_conversion_distributed.cu
diff --git a/...kit/kit_cc_impl/operations/CMakeLists.txt → ...kit/kit_cc_impl/operations/CMakeLists.txt b/...kit/kit_cc_impl/operations/CMakeLists.txt → ...kit/kit_cc_impl/operations/CMakeLists.txt
diff --git a/...on_kit/kit_cc_impl/operations/cc/.gitkeep → ...on_kit/kit_cc_impl/operations/cc/.gitkeep b/...on_kit/kit_cc_impl/operations/cc/.gitkeep → ...on_kit/kit_cc_impl/operations/cc/.gitkeep
diff --git a/...t/kit_cc_impl/operations/include/.gitkeep → ...t/kit_cc_impl/operations/include/.gitkeep b/...t/kit_cc_impl/operations/include/.gitkeep → ...t/kit_cc_impl/operations/include/.gitkeep
diff --git a/sparse_operation_kit/notebooks/ReadMe.md → .../sparse_operation_kit/notebooks/ReadMe.md b/sparse_operation_kit/notebooks/ReadMe.md → .../sparse_operation_kit/notebooks/ReadMe.md
diff --git a/...e_operation_kit/notebooks/benchmark.ipynb → ...e_operation_kit/notebooks/benchmark.ipynb b/...e_operation_kit/notebooks/benchmark.ipynb → ...e_operation_kit/notebooks/benchmark.ipynb
diff --git a/...ooks/demo_with_nvtabular_datareader.ipynb → ...ooks/demo_with_nvtabular_datareader.ipynb b/...ooks/demo_with_nvtabular_datareader.ipynb → ...ooks/demo_with_nvtabular_datareader.ipynb
diff --git a/...notebooks/sparse_operation_kit_demo.ipynb → ...notebooks/sparse_operation_kit_demo.ipynb b/...notebooks/sparse_operation_kit_demo.ipynb → ...notebooks/sparse_operation_kit_demo.ipynb
diff --git a/sparse_operation_kit/pyproject.toml → addons/sparse_operation_kit/pyproject.toml b/sparse_operation_kit/pyproject.toml → addons/sparse_operation_kit/pyproject.toml
diff --git a/sparse_operation_kit/request.sh → addons/sparse_operation_kit/request.sh b/sparse_operation_kit/request.sh → addons/sparse_operation_kit/request.sh
diff --git a/sparse_operation_kit/setup.py → addons/sparse_operation_kit/setup.py b/sparse_operation_kit/setup.py → addons/sparse_operation_kit/setup.py
diff --git a/...tion_kit/sparse_operation_kit/__init__.py → ...tion_kit/sparse_operation_kit/__init__.py b/...tion_kit/sparse_operation_kit/__init__.py → ...tion_kit/sparse_operation_kit/__init__.py
diff --git a/...kit/sparse_operation_kit/core/__init__.py → ...kit/sparse_operation_kit/core/__init__.py b/...kit/sparse_operation_kit/core/__init__.py → ...kit/sparse_operation_kit/core/__init__.py
diff --git a/...kit/sparse_operation_kit/core/_version.py → ...kit/sparse_operation_kit/core/_version.py b/...kit/sparse_operation_kit/core/_version.py → ...kit/sparse_operation_kit/core/_version.py
diff --git a/...parse_operation_kit/core/context_scope.py → ...parse_operation_kit/core/context_scope.py b/...parse_operation_kit/core/context_scope.py → ...parse_operation_kit/core/context_scope.py
diff --git a/...ration_kit/core/embedding_layer_handle.py → ...ration_kit/core/embedding_layer_handle.py b/...ration_kit/core/embedding_layer_handle.py → ...ration_kit/core/embedding_layer_handle.py
diff --git a/...eration_kit/core/embedding_variable_v1.py → ...eration_kit/core/embedding_variable_v1.py b/...eration_kit/core/embedding_variable_v1.py → ...eration_kit/core/embedding_variable_v1.py
diff --git a/...eration_kit/core/embedding_variable_v2.py → ...eration_kit/core/embedding_variable_v2.py b/...eration_kit/core/embedding_variable_v2.py → ...eration_kit/core/embedding_variable_v2.py
diff --git a/...t/sparse_operation_kit/core/graph_keys.py → ...t/sparse_operation_kit/core/graph_keys.py b/...t/sparse_operation_kit/core/graph_keys.py → ...t/sparse_operation_kit/core/graph_keys.py
diff --git a/...t/sparse_operation_kit/core/initialize.py → ...t/sparse_operation_kit/core/initialize.py b/...t/sparse_operation_kit/core/initialize.py → ...t/sparse_operation_kit/core/initialize.py
diff --git a/...arse_operation_kit/embeddings/__init__.py → ...arse_operation_kit/embeddings/__init__.py b/...arse_operation_kit/embeddings/__init__.py → ...arse_operation_kit/embeddings/__init__.py
diff --git a/...kit/embeddings/all2all_dense_embedding.py → ...kit/embeddings/all2all_dense_embedding.py b/...kit/embeddings/all2all_dense_embedding.py → ...kit/embeddings/all2all_dense_embedding.py
diff --git a/...n_kit/embeddings/distributed_embedding.py → ...n_kit/embeddings/distributed_embedding.py b/...n_kit/embeddings/distributed_embedding.py → ...n_kit/embeddings/distributed_embedding.py
diff --git a/...operation_kit/embeddings/embedding_ops.py → ...operation_kit/embeddings/embedding_ops.py b/...operation_kit/embeddings/embedding_ops.py → ...operation_kit/embeddings/embedding_ops.py
diff --git a/...ration_kit/embeddings/get_embedding_op.py → ...ration_kit/embeddings/get_embedding_op.py b/...ration_kit/embeddings/get_embedding_op.py → ...ration_kit/embeddings/get_embedding_op.py
diff --git a/...it/embeddings/tf_distributed_embedding.py → ...it/embeddings/tf_distributed_embedding.py b/...it/embeddings/tf_distributed_embedding.py → ...it/embeddings/tf_distributed_embedding.py
diff --git a/...ation_kit/sparse_operation_kit/kit_lib.py → ...ation_kit/sparse_operation_kit/kit_lib.py b/...ation_kit/sparse_operation_kit/kit_lib.py → ...ation_kit/sparse_operation_kit/kit_lib.py
diff --git a/...arse_operation_kit/operations/__init__.py → ...arse_operation_kit/operations/__init__.py b/...arse_operation_kit/operations/__init__.py → ...arse_operation_kit/operations/__init__.py
diff --git a/...peration_kit/operations/compat_ops_lib.py → ...peration_kit/operations/compat_ops_lib.py b/...peration_kit/operations/compat_ops_lib.py → ...peration_kit/operations/compat_ops_lib.py
diff --git a/...arse_operation_kit/optimizers/__init__.py → ...arse_operation_kit/optimizers/__init__.py b/...arse_operation_kit/optimizers/__init__.py → ...arse_operation_kit/optimizers/__init__.py
diff --git a/...t/sparse_operation_kit/optimizers/adam.py → ...t/sparse_operation_kit/optimizers/adam.py b/...t/sparse_operation_kit/optimizers/adam.py → ...t/sparse_operation_kit/optimizers/adam.py
diff --git a/...rse_operation_kit/optimizers/optimizer.py → ...rse_operation_kit/optimizers/optimizer.py b/...rse_operation_kit/optimizers/optimizer.py → ...rse_operation_kit/optimizers/optimizer.py
diff --git a/.../sparse_operation_kit/optimizers/utils.py → .../sparse_operation_kit/optimizers/utils.py b/.../sparse_operation_kit/optimizers/utils.py → .../sparse_operation_kit/optimizers/utils.py
diff --git a/...n_kit/sparse_operation_kit/saver/Saver.py → ...n_kit/sparse_operation_kit/saver/Saver.py b/...n_kit/sparse_operation_kit/saver/Saver.py → ...n_kit/sparse_operation_kit/saver/Saver.py
diff --git a/...it/sparse_operation_kit/saver/__init__.py → ...it/sparse_operation_kit/saver/__init__.py b/...it/sparse_operation_kit/saver/__init__.py → ...it/sparse_operation_kit/saver/__init__.py
diff --git a/...n_kit/sparse_operation_kit/tf/__init__.py → ...n_kit/sparse_operation_kit/tf/__init__.py b/...n_kit/sparse_operation_kit/tf/__init__.py → ...n_kit/sparse_operation_kit/tf/__init__.py
diff --git a/...sparse_operation_kit/tf/keras/__init__.py → ...sparse_operation_kit/tf/keras/__init__.py b/...sparse_operation_kit/tf/keras/__init__.py → ...sparse_operation_kit/tf/keras/__init__.py
diff --git a/...ation_kit/tf/keras/optimizers/__init__.py → ...ation_kit/tf/keras/optimizers/__init__.py b/...ation_kit/tf/keras/optimizers/__init__.py → ...ation_kit/tf/keras/optimizers/__init__.py
diff --git a/...operation_kit/tf/keras/optimizers/adam.py → ...operation_kit/tf/keras/optimizers/adam.py b/...operation_kit/tf/keras/optimizers/adam.py → ...operation_kit/tf/keras/optimizers/adam.py
diff --git a/...eration_kit/tf/keras/optimizers/common.py → ...eration_kit/tf/keras/optimizers/common.py b/...eration_kit/tf/keras/optimizers/common.py → ...eration_kit/tf/keras/optimizers/common.py
diff --git a/...tion_kit/tf/keras/optimizers/lazy_adam.py → ...tion_kit/tf/keras/optimizers/lazy_adam.py b/...tion_kit/tf/keras/optimizers/lazy_adam.py → ...tion_kit/tf/keras/optimizers/lazy_adam.py
diff --git a/sparse_operation_kit/unit_test/.gitkeep → ...s/sparse_operation_kit/unit_test/.gitkeep b/sparse_operation_kit/unit_test/.gitkeep → ...s/sparse_operation_kit/unit_test/.gitkeep
diff --git a/...se_operation_kit/unit_test/CMakeLists.txt → ...se_operation_kit/unit_test/CMakeLists.txt b/...se_operation_kit/unit_test/CMakeLists.txt → ...se_operation_kit/unit_test/CMakeLists.txt
diff --git a/..._test/cc/kernels/all_gather_dispatcher.cc → ..._test/cc/kernels/all_gather_dispatcher.cc b/..._test/cc/kernels/all_gather_dispatcher.cc → ..._test/cc/kernels/all_gather_dispatcher.cc
diff --git a/.../cc/kernels/csr_conversion_distributed.cc → .../cc/kernels/csr_conversion_distributed.cc b/.../cc/kernels/csr_conversion_distributed.cc → .../cc/kernels/csr_conversion_distributed.cc
diff --git a/...t/cc/kernels/reduce_scatter_dispatcher.cc → ...t/cc/kernels/reduce_scatter_dispatcher.cc b/...t/cc/kernels/reduce_scatter_dispatcher.cc → ...t/cc/kernels/reduce_scatter_dispatcher.cc
diff --git a/...ion_kit/unit_test/cc/ops/unit_test_ops.cc → ...ion_kit/unit_test/cc/ops/unit_test_ops.cc b/...ion_kit/unit_test/cc/ops/unit_test_ops.cc → ...ion_kit/unit_test/cc/ops/unit_test_ops.cc
diff --git a/...operation_kit/unit_test/cc/unit_tester.cc → ...operation_kit/unit_test/cc/unit_tester.cc b/...operation_kit/unit_test/cc/unit_tester.cc → ...operation_kit/unit_test/cc/unit_tester.cc
diff --git a/..._operation_kit/unit_test/cc/unit_tester.h → ..._operation_kit/unit_test/cc/unit_tester.h b/..._operation_kit/unit_test/cc/unit_tester.h → ..._operation_kit/unit_test/cc/unit_tester.h
diff --git a/...kit/unit_test/python/sok_unit_test_lib.py → ...kit/unit_test/python/sok_unit_test_lib.py b/...kit/unit_test/python/sok_unit_test_lib.py → ...kit/unit_test/python/sok_unit_test_lib.py
diff --git a/...ation_kit/unit_test/test_scripts/tf1/core → ...ation_kit/unit_test/test_scripts/tf1/core b/...ation_kit/unit_test/test_scripts/tf1/core → ...ation_kit/unit_test/test_scripts/tf1/core
diff --git a/...nit_test/test_scripts/tf1/dense_models.py → ...nit_test/test_scripts/tf1/dense_models.py b/...nit_test/test_scripts/tf1/dense_models.py → ...nit_test/test_scripts/tf1/dense_models.py
diff --git a/..._kit/unit_test/test_scripts/tf1/script.sh → ..._kit/unit_test/test_scripts/tf1/script.sh b/..._kit/unit_test/test_scripts/tf1/script.sh → ..._kit/unit_test/test_scripts/tf1/script.sh
diff --git a/...it_test/test_scripts/tf1/sparse_models.py → ...it_test/test_scripts/tf1/sparse_models.py b/...it_test/test_scripts/tf1/sparse_models.py → ...it_test/test_scripts/tf1/sparse_models.py
diff --git a/...test/test_scripts/tf1/strategy_wrapper.py → ...test/test_scripts/tf1/strategy_wrapper.py b/...test/test_scripts/tf1/strategy_wrapper.py → ...test/test_scripts/tf1/strategy_wrapper.py
diff --git a/...t/test_scripts/tf1/test_dense_emb_demo.py → ...t/test_scripts/tf1/test_dense_emb_demo.py b/...t/test_scripts/tf1/test_dense_emb_demo.py → ...t/test_scripts/tf1/test_dense_emb_demo.py
diff --git a/.../test_scripts/tf1/test_sparse_emb_demo.py → .../test_scripts/tf1/test_sparse_emb_demo.py b/.../test_scripts/tf1/test_sparse_emb_demo.py → .../test_scripts/tf1/test_sparse_emb_demo.py
diff --git a/...n_kit/unit_test/test_scripts/tf1/utils.py → ...n_kit/unit_test/test_scripts/tf1/utils.py b/...n_kit/unit_test/test_scripts/tf1/utils.py → ...n_kit/unit_test/test_scripts/tf1/utils.py
diff --git a/triton/BUILD → addons/triton/BUILD b/triton/BUILD → addons/triton/BUILD
@@ -1,8 +1,6 @@
 # Description:
 # TRITON API.
 
-exports_files(["tf_triton_version_script.lds"])
-
 cc_library(
     name = "triton_tf",
     visibility = ["//visibility:public"],

diff --git a/triton/tensorflow_backend_tf.cc → addons/triton/tensorflow_backend_tf.cc b/triton/tensorflow_backend_tf.cc → addons/triton/tensorflow_backend_tf.cc
diff --git a/triton/tensorflow_backend_tf.h → addons/triton/tensorflow_backend_tf.h b/triton/tensorflow_backend_tf.h → addons/triton/tensorflow_backend_tf.h
diff --git a/cibuild/Dockerfile/Dockerfile.py3.6-cu112-ubuntu18.04 b/cibuild/Dockerfile/Dockerfile.py3.6-cu112-ubuntu18.04
@@ -0,0 +1,23 @@
+FROM nvidia/cuda:11.2.2-cudnn8-devel-ubuntu18.04
+
+RUN apt-get update && \
+    apt-get install -y --allow-unauthenticated \
+    wget \
+    cmake \
+    git \
+    unzip \
+    curl \
+    libssl-dev \
+    libcurl4-openssl-dev \
+    zlib1g-dev \
+    python3 \
+    python3-dev \
+    python3-pip \
+    && apt-get clean && \
+    ln -sf python3 /usr/bin/python && \
+    ln -sf pip3 /usr/bin/pip
+
+RUN pip install astor==0.8.1
+RUN pip install numpy==1.16.6
+RUN pip install protobuf==3.17.3
+RUN pip --no-deps keras-preprocessing==1.0.5
diff --git a/cibuild/Dockerfile/Dockerfile.py3.6-cu117-ubuntu18.04 b/cibuild/Dockerfile/Dockerfile.py3.6-cu117-ubuntu18.04
@@ -0,0 +1,23 @@
+FROM nvidia/cuda:11.7.1-cudnn8-devel-ubuntu18.04
+
+RUN apt-get update && \
+    apt-get install -y --allow-unauthenticated \
+    wget \
+    cmake \
+    git \
+    unzip \
+    curl \
+    libssl-dev \
+    libcurl4-openssl-dev \
+    zlib1g-dev \
+    python3 \
+    python3-dev \
+    python3-pip \
+    && apt-get clean && \
+    ln -sf python3 /usr/bin/python && \
+    ln -sf pip3 /usr/bin/pip
+
+RUN pip install astor==0.8.1
+RUN pip install numpy==1.16.6
+RUN pip install protobuf==3.17.3
+RUN pip install --no-deps keras-preprocessing==1.0.5
diff --git a/scripts/install_pmem_dependencies.sh → ...file/scripts/install_pmem_dependencies.sh b/scripts/install_pmem_dependencies.sh → ...file/scripts/install_pmem_dependencies.sh
diff --git a/cibuild/gpu-ut/gpu-python-ut.sh b/cibuild/gpu-ut/gpu-python-ut.sh
@@ -114,7 +114,7 @@ for i in $(seq 1 3); do
     [ $i -gt 1 ] && echo "WARNING: cmd execution failed, will retry in $((i-1)) times later" && sleep 2
     ret=0
     bazel test -c opt --config=cuda --verbose_failures --test_env='NVIDIA_TF32_OVERRIDE=0' \
-    --run_under=//tensorflow/tools/ci_build/gpu_build:parallel_gpu_execute  \
+    --run_under=//tensorflow/tools/ci_build/gpu_build:parallel_gpu_execute --config=opt \
     --test_timeout="300,450,1200,3600" --local_test_jobs=20 --test_output=errors \
     -- $TF_BUILD_BAZEL_TARGET && break || ret=$?
 done

diff --git a/configure.py b/configure.py
@@ -1586,7 +1586,7 @@ def main():
   print('Preconfigured Bazel build configs. You can use any of the below by '
         'adding "--config=<>" to your build command. See .bazelrc for more '
         'details.')
-  config_info_line('mkl', 'Build with MKL support.')
+  config_info_line('mkl_threadpool', 'Build with oneDNN support.')
   config_info_line('monolithic', 'Config for mostly static monolithic build.')
   config_info_line('gdr', 'Build with GDR support.')
   config_info_line('verbs', 'Build with libverbs support.')

diff --git a/docs/AdamW-Optimizer.md b/docs/AdamW-Optimizer.md
@@ -0,0 +1,55 @@
+# AdamW Optimizer
+## 介绍
+AdamW优化器支持Embedding Variable，相对于Adam优化器增加了了权重衰减的功能。
+
+这是Loshch ilov & Hutter (https://arxiv.org/abs/1711.05101)的 "Decoupled Weight Decay Regularization"中描述的AdamW优化器的一个实现。
+
+
+## 用户接口
+训练时只需要定义`tf.train.AdamWOptimizer`即可，和其他TF原生Optimizer使用方式相同。具体定义如下：
+```python
+class AdamWOptimizer(DecoupledWeightDecayExtension, adam.AdamOptimizer):
+  def __init__(self,
+               weight_decay,
+               learning_rate=0.001,
+               beta1=0.9,
+               beta2=0.999,
+               epsilon=1e-8,
+               use_locking=False,
+               name="AdamW"):
+
+# 调用方法：
+optimizer = tf.train.AdamWOptimizer(
+               weight_decay=weight_decay_new
+               learning_rate=learning_rate_new,
+               beta1=0.9,
+               beta2=0.999,
+               epsilon=1e-8)
+```
+## 使用示例
+```python
+import tensorflow as tf
+
+var = tf.get_variable("var_0", shape=[10,16],
+                       initializer=tf.ones_initializer(tf.float32))
+
+emb = tf.nn.embedding_lookup(var, tf.cast([0,1,2,5,6,7], tf.int64))
+fun = tf.multiply(emb, 2.0, name='multiply')
+loss = tf.reduce_sum(fun, name='reduce_sum')
+
+gs= tf.train.get_or_create_global_step()
+opt = tf.train.AdamWOptimizer(weight_decay=0.01, learning_rate=0.1)
+
+g_v = opt.compute_gradients(loss)
+train_op = opt.apply_gradients(g_v)
+
+init = tf.global_variables_initializer()
+
+sess_config = tf.ConfigProto(allow_soft_placement=True, log_device_placement=False)
+with tf.Session(config=sess_config) as sess:
+  sess.run([init])
+  print(sess.run([emb, train_op, loss]))
+  print(sess.run([emb, train_op, loss]))
+  print(sess.run([emb, train_op, loss]))
+```
+
diff --git a/docs/DeepRec-Compile-And-Install.md b/docs/DeepRec-Compile-And-Install.md
@@ -4,44 +4,28 @@
 
 **CPU Base Docker Image**
 
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-base-cpu-py36-ubuntu18.04
-```
-
-Docker Hub repository
 ```
 alideeprec/deeprec-base:deeprec-base-cpu-py36-ubuntu18.04
 ```
 
-**GPU(cuda11.0) Base Docker Image**
-
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-base-gpu-py36-cu110-ubuntu18.04
-```
+**GPU Base Docker Image**
 
-Docker Hub repository
-```
-alideeprec/deeprec-base:deeprec-base-gpu-py36-cu110-ubuntu18.04
-```
+| CUDA VERSION |                           IMAGE                                 |
+| ------------ | --------------------------------------------------------------- |
+| CUDA 11.0.3  | alideeprec/deeprec-base:deeprec-base-gpu-py36-cu110-ubuntu18.04 |
+| CUDA 11.2.2  | alideeprec/deeprec-base:deeprec-base-gpu-py36-cu112-ubuntu18.04 |
+| CUDA 11.4.2  | alideeprec/deeprec-base:deeprec-base-gpu-py36-cu114-ubuntu18.04 |
+| CUDA 11.6.1  | alideeprec/deeprec-base:deeprec-base-gpu-py36-cu116-ubuntu18.04 |
+| CUDA 11.7.1  | alideeprec/deeprec-base:deeprec-base-gpu-py36-cu117-ubuntu18.04 |
 
 **CPU Dev Docker (with bazel cache)**
 
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-dev-cpu-py36-ubuntu18.04
-```
-
-Docker Hub repository
 ```
 alideeprec/deeprec-build:deeprec-dev-cpu-py36-ubuntu18.04
 ```
 
 **GPU(cuda11.0) Dev Docker (with bazel cache)**
 
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-dev-gpu-py36-cu110-ubuntu18.04
-```
-
-Docker Hub repository
 ```
 alideeprec/deeprec-build:deeprec-dev-gpu-py36-cu110-ubuntu18.04
 ```
@@ -110,22 +94,12 @@ pip3 install /tmp/tensorflow_pkg/tensorflow-1.15.5+${version}-cp36-cp36m-linux_x
 
 **GPU CUDA11.0镜像**
 
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec2206-gpu-py36-cu110-ubuntu18.04
-```
-
-Docker Hub repository
 ```
 alideeprec/deeprec-release:deeprec2206-gpu-py36-cu110-ubuntu18.04
 ```
 
 **CPU镜像**
 
-```
-registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec2206-cpu-py36-ubuntu18.04
-```
-
-Docker Hub repository
 ```
 alideeprec/deeprec-release:deeprec2206-cpu-py36-ubuntu18.04
 ```

diff --git a/docs/Embedding-Variable-GPU.md b/docs/Embedding-Variable-GPU.md
@@ -37,4 +37,4 @@ with tf.device('/gpu:0'):
                 initializer=tf.ones_initializer(tf.dtypes.float32))
 ```
 
-注意：GPU版本的EmbeddingVariable暂时无法和TensorFlow自带Saver一起使用，我们后面会修复这个问题。
+注意：目前GPU EV不支持incremental checkpoint，如果使用的话EV相关的OP会被放置到CPU上，这个问题我们后续会修复。
diff --git a/docs/Embedding-Variable.md b/docs/Embedding-Variable.md
@@ -106,7 +106,7 @@ W = tf.feature_column.embedding_column(categorical_column=columns,
             initializer=tf.ones_initializer(tf.dtypes.float32))
 
 ids={}
-ids["col_emb"] = tf.SparseTensor(indices=[[0,0],[1,1],[2,2],[3,3],[4,4]], values=tf.cast([1,2,3,4,5], tf.dtypes.int64), dense_shape=[5, 4])
+ids["col_emb"] = tf.SparseTensor(indices=[[0,0],[1,1],[2,2],[3,3],[4,4]], values=tf.cast([1,2,3,4,5], tf.dtypes.int64), dense_shape=[5, 5])
 
 emb = tf.feature_column.input_layer(ids, [W])
 fun = tf.multiply(emb, 2.0, name='multiply')
@@ -137,7 +137,7 @@ W = feature_column.embedding_column(sparse_id_column=columns,
             initializer=tf.ones_initializer(tf.dtypes.float32))
 
 ids={}
-ids["col_emb"] = tf.SparseTensor(indices=[[0,0],[1,1],[2,2],[3,3],[4,4]], values=tf.cast([1,2,3,4,5], tf.dtypes.int64), dense_shape=[5, 4])
+ids["col_emb"] = tf.SparseTensor(indices=[[0,0],[1,1],[2,2],[3,3],[4,4]], values=tf.cast([1,2,3,4,5], tf.dtypes.int64), dense_shape=[5, 5])
 
 emb = feature_column_ops.input_from_feature_columns(columns_to_tensors=ids, feature_columns=[W])
 fun = tf.multiply(emb, 2.0, name='multiply')
@@ -147,6 +147,69 @@ g_v = opt.compute_gradients(loss)
 train_op = opt.apply_gradients(g_v)
 init = tf.global_variables_initializer()
 
+with tf.Session() as sess:
+    sess.run(init)
+    print("init global done")
+    print(sess.run([emb, train_op,loss]))
+    print(sess.run([emb, train_op,loss]))
+    print(sess.run([emb, train_op,loss]))
+```
+使用`sequence_categorical_column_with_embedding`接口：
+```python
+import tensorflow as tf
+from tensorflow.python.feature_column import sequence_feature_column
+
+
+columns = sequence_feature_column.sequence_categorical_column_with_embedding(key="col_emb", dtype=tf.dtypes.int32)
+W = tf.feature_column.embedding_column(categorical_column=columns,
+            dimension=3,
+            initializer=tf.ones_initializer(tf.dtypes.float32))
+
+ids={}
+ids["col_emb"] = tf.SparseTensor(indices=[[0,0],[0,1],[1,1],[2,2],[3,3],[4,4]], \
+                                 values=tf.cast([1,3,2,3,4,5], tf.dtypes.int64), 
+                                 dense_shape=[5, 5])
+
+emb, length = tf.contrib.feature_column.sequence_input_layer(ids, [W])
+fun = tf.multiply(emb, 2.0, name='multiply')
+loss = tf.reduce_sum(fun, name='reduce_sum')
+opt = tf.train.FtrlOptimizer(0.1, l1_regularization_strength=2.0, l2_regularization_strength=0.00001)
+g_v = opt.compute_gradients(loss)
+train_op = opt.apply_gradients(g_v)
+init = tf.global_variables_initializer()
+
+with tf.Session() as sess:
+    sess.run(init)
+    print("init global done")
+    print(sess.run([emb, train_op,loss]))
+    print(sess.run([emb, train_op,loss]))
+    print(sess.run([emb, train_op,loss]))
+```
+使用`weighted_categorical_column`接口：
+```python
+import tensorflow as tf
+
+
+categorical_column = tf.feature_column.categorical_column_with_embedding("col_emb", dtype=tf.dtypes.int64)
+
+ids={}
+ids["col_emb"] = tf.SparseTensor(indices=[[0,0],[0,1],[1,1],[2,2],[3,3],[4,3],[4,4]], \
+                        values=tf.cast([1,3,2,3,4,5,3], tf.dtypes.int64), dense_shape=[5, 5])    
+ids['weight'] = [[2.0],[5.0],[4.0],[8.0],[3.0],[1.0],[2.5]]
+
+columns = tf.feature_column.weighted_categorical_column(categorical_column, 'weight')
+
+W = tf.feature_column.embedding_column(categorical_column=columns,
+            dimension=3,
+            initializer=tf.ones_initializer(tf.dtypes.float32))
+emb = tf.feature_column.input_layer(ids, [W])
+fun = tf.multiply(emb, 2.0, name='multiply')
+loss = tf.reduce_sum(fun, name='reduce_sum')
+opt = tf.train.FtrlOptimizer(0.1, l1_regularization_strength=2.0, l2_regularization_strength=0.00001)
+g_v = opt.compute_gradients(loss)
+train_op = opt.apply_gradients(g_v)
+init = tf.global_variables_initializer()
+
 with tf.Session() as sess:
     sess.run(init)
     print("init global done")
@@ -186,9 +249,11 @@ emb_var = tf.feature_column.categorical_column_with_embedding("var", ev_option=e
 class InitializerOption(object):
   def __init__(self,
                initializer = None,
-               default_value_dim = 4096):
+               default_value_dim = 4096,
+               default_value_no_permission = .0):
     self.initializer = initializer
     self.default_value_dim  = default_value_dim
+    self.default_value_no_permission = default_value_no_permission
     if default_value_dim <=0:
       print("default value dim must larger than 1, the default value dim is set to default 4096.")
       default_value_dim = 4096
@@ -197,6 +262,7 @@ class InitializerOption(object):
 
 - `initializer`：Embedding Variable使用的Initializer，如果不配置的话则会被设置EV默认设置为truncated normal initializer。
 - `default value dim`：生成的default value的数量，设置可以参考hash bucket size或是特征的数量，默认是4096。
+- `default value no permission`：当使用准入功能时，如果特征未准入，返回的Embedding默认值。
 
 
 
diff --git a/docs/Embedding-Variable/img_2.jpg b/docs/Embedding-Variable/img_2.jpg
diff --git a/docs/Estimator-Compile-And-Install.md b/docs/Estimator-Compile-And-Install.md
@@ -51,7 +51,8 @@ alideeprec/deeprec-build:deeprec-dev-gpu-py36-cu110-ubuntu18.04
 由于DeepRec新增了分布式grpc++、star_server等protocol，在使用DeepRec配合原生Estimator会存在像grpc++, star_server功能使用时无法通过Estimator检查的问题，因为我们提供了针对DeepRec版本的Estimator.
 
 代码库：[https://github.com/AlibabaPAI/estimator](https://github.com/AlibabaPAI/estimator)
-分支：deeprec
+
+开发分支：master，最新Release分支：deeprec2206
 
 ## Estimator编译