Merge pull request #1361 from JinHai-CN/crud

Merge from master

CHANGELOG.md

@@ -25,6 +25,7 @@ Please mark all change in change log and use the issue from GitHub
 - \#1115 - http server support load table into memory
 - \#1211 - Server down caused by searching with index_type: HNSW
 - \#1240 - Update license declaration
+- \#1298 - Unittest failed when on CPU2GPU case
 
 ## Feature
 - \#216 - Add CLI to get server info
@@ -56,6 +57,8 @@ Please mark all change in change log and use the issue from GitHub
 - \#1297 - Hide partition_name parameter, avid user directly access partition table
 - \#1310 - Add default partition tag for a table
 - \#1105 - Error message is not clear when creating IVFSQ8H index without gpu resources
+- \#740, #849, #878, #972, #1033, #1161, #1173, #1199, #1190, #1223, #1222, #1257, #1264, #1269, #1164, #1304, #1324 - Various fixes and improvements for Milvus documentation.
+- \#1320 - Remove debug logging from faiss
 
 ## Task
 

DESIGN.md

@@ -0,0 +1,9 @@
+# Milvus Design Documents
+
+If you would like to propose a new feature for Milvus, it is recommended that you submit a design document following the [design document template](https://docs.google.com/document/d/1KhWs9b53K6WQUZ_FGWhIaeTraaSqsLQF7v2v68sAh_c/edit?usp=sharing).
+
+The following list contains existing design documents for Milvus.
+
+- [Support DSL interface](https://docs.google.com/document/d/1U83LY36TyaG3WD67Q9HWg9saD3qQcz9BfMcEScgwQPM/edit?usp=sharing)
+- [Support delete/search by ID, attribute filtering, ID de-duplication](https://docs.google.com/document/d/1CDKdTj_DnE90YaZrPgsMaphqOTkMdbKETNrsFKj_Bco/edit?usp=sharing)
+- [Support write-ahead logging](https://docs.google.com/document/d/12N8RC_wJb2dvEKY9jrlh8hU_eH8jxQVBewoPuHNqcXE/edit?usp=sharing)

INSTALL.md

@@ -3,30 +3,31 @@
 <!-- TOC -->
 
 - [Build from source](#build-from-source)
-    - [Requirements](#requirements)
-    - [Compilation](#compilation)
-    - [Launch Milvus server](#launch-milvus-server)
+  - [Requirements](#requirements)
+  - [Compilation](#compilation)
+  - [Launch Milvus server](#launch-milvus-server)
 - [Compile Milvus on Docker](#compile-milvus-on-docker)
-    - [Step 1 Pull Milvus Docker images](#step-1-pull-milvus-docker-images)
-    - [Step 2 Start the Docker container](#step-2-start-the-docker-container)
-    - [Step 3 Download Milvus source code](#step-3-download-milvus-source-code)
-    - [Step 4 Compile Milvus in the container](#step-4-compile-milvus-in-the-container)
+  - [Step 1 Pull Milvus Docker images](#step-1-pull-milvus-docker-images)
+  - [Step 2 Start the Docker container](#step-2-start-the-docker-container)
+  - [Step 3 Download Milvus source code](#step-3-download-milvus-source-code)
+  - [Step 4 Compile Milvus in the container](#step-4-compile-milvus-in-the-container)
 - [Troubleshooting](#troubleshooting)
-    - [Error message: `protocol https not supported or disabled in libcurl`](#error-message-protocol-https-not-supported-or-disabled-in-libcurl)
-    - [Error message: `internal compiler error`](#error-message-internal-compiler-error)
-    - [Error message: `error while loading shared libraries: libmysqlpp.so.3`](#error-message-error-while-loading-shared-libraries-libmysqlppso3)
-    - [CMake version is not supported](#cmake-version-is-not-supported)
+  - [Error message: `protocol https not supported or disabled in libcurl`](#error-message-protocol-https-not-supported-or-disabled-in-libcurl)
+  - [Error message: `internal compiler error`](#error-message-internal-compiler-error)
+  - [Error message: `error while loading shared libraries: libmysqlpp.so.3`](#error-message-error-while-loading-shared-libraries-libmysqlppso3)
+  - [CMake version is not supported](#cmake-version-is-not-supported)
 
 <!-- /TOC -->
 
 ## Build from source
 
 ### Requirements
 
-- Ubuntu 18.04 or higher
-- CentOS 7
+- Operating system
+  - Ubuntu 18.04 or higher
+  - CentOS 7
 
-If your operating system does not meet the requirements, we recommend that you pull a Docker image of [Ubuntu 18.04](https://docs.docker.com/install/linux/docker-ce/ubuntu/) or [CentOS 7](https://docs.docker.com/install/linux/docker-ce/centos/) as your compilation environment.
+  If your operating system does not meet the requirements, we recommend that you pull a Docker image of [Ubuntu 18.04](https://docs.docker.com/install/linux/docker-ce/ubuntu/) or [CentOS 7](https://docs.docker.com/install/linux/docker-ce/centos/) as your compilation environment.
 
 - GCC 7.0 or higher to support C++ 17
 - CMake 3.12 or higher
@@ -40,31 +41,31 @@ For GPU-enabled version, you will also need:
 
 #### Step 1 Install dependencies
 
+##### Install in Ubuntu
+
 ```shell
 $ cd [Milvus root path]/core
 $ ./ubuntu_build_deps.sh
 ```
 
-#### Step 2 Build
-
-##### Build in Ubuntu
+##### Install in CentOS
 
 ```shell
 $ cd [Milvus root path]/core
-$ ./build.sh -t Debug
+$ ./centos7_build_deps.sh
 ```
 
-or 
+#### Step 2 Build
 
 ```shell
-$ ./build.sh -t Release
+$ cd [Milvus root path]/core
+$ ./build.sh -t Debug
 ```
 
-##### Build in CentOS
+or
 
 ```shell
-$ cd [Milvus root path]/core
-$ ./centos7_build_deps.sh
+$ ./build.sh -t Release
 ```
 
 By default, it will build CPU-only version. To build GPU version, add `-g` option.
@@ -79,7 +80,7 @@ If you want to know the complete build options, run the following command.
 $./build.sh -h
 ```
 
-When the build is completed, all the stuff that you need in order to run Milvus will be installed under `[Milvus root path]/core/milvus`.
+When the build is completed, everything that you need in order to run Milvus will be installed under `[Milvus root path]/core/milvus`.
 
 ### Launch Milvus server
 
@@ -141,6 +142,7 @@ To enter the container:
 ```shell
 $ docker exec -it [container_id] bash
 ```
+
 ### Step 3 Download Milvus source code
 
 Download latest Milvus source code:
@@ -195,12 +197,18 @@ Follow the steps below to solve this problem:
 
 ### Error message: `internal compiler error`
 
-Try increasing the memory allocated to docker.
+Try increasing the memory allocated to Docker. If this doesn't work, you can reduce the number of threads in CMake build in `[Milvus root path]/core/build.sh`.
+
+```shell
+make -j 8 install || exit 1 # The default number of threads is 8.
+```
+
+Note: You might also need to configure CMake build for faiss in `[Milvus root path]/core/src/index/thirdparty/faiss`.
 
 ### Error message: `error while loading shared libraries: libmysqlpp.so.3`
 
 Follow the steps below to solve this problem:
-    
+
 1. Check whether `libmysqlpp.so.3` is correctly installed.
 2. If `libmysqlpp.so.3` is installed, check whether it is added to `LD_LIBRARY_PATH`.
 

OSSMETADATA

@@ -0,0 +1,2 @@
+osslifecycle=active
+

README.md

@@ -1,21 +1,33 @@
 ![Milvuslogo](https://github.com/milvus-io/docs/blob/master/assets/milvus_logo.png)
 
 [![Slack](https://img.shields.io/badge/Join-Slack-orange)](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk)
-![LICENSE](https://img.shields.io/badge/license-Apache--2.0-brightgreen)
-![Language](https://img.shields.io/badge/language-C%2B%2B-blue)
-[![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-jinhai-cn-milvus-master)
-![Release](https://img.shields.io/badge/release-v0.6.0-yellowgreen)
-![Release_date](https://img.shields.io/badge/release%20date-December-yellowgreen)
-[![codecov](https://codecov.io/gh/milvus-io/milvus/branch/master/graph/badge.svg)](https://codecov.io/gh/milvus-io/milvus)
-[![All Contributors](https://img.shields.io/badge/all_contributors-7-orange.svg?style=flat-square)](#contributors-)
-[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3563/badge)](https://bestpractices.coreinfrastructure.org/projects/3563)
+
+![GitHub](https://img.shields.io/github/license/milvus-io/milvus)
+![Language](https://img.shields.io/github/languages/count/milvus-io/milvus)
+![GitHub top language](https://img.shields.io/github/languages/top/milvus-io/milvus)
+![GitHub release (latest by date)](https://img.shields.io/github/v/release/milvus-io/milvus)
+![GitHub Release Date](https://img.shields.io/github/release-date/milvus-io/milvus)
+
+[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://github.com/milvus-io/milvus/pulse/monthly)
+![OSS Lifecycle](https://img.shields.io/osslifecycle/milvus-io/milvus)
+[![HitCount](http://hits.dwyl.com/milvus-io/milvus.svg)](http://hits.dwyl.com/milvus-io/milvus)
+![Docker pulls](https://img.shields.io/docker/pulls/milvusdb/milvus)
+
 [![Build Status](http://internal.zilliz.com:18080/jenkins/job/milvus-ci/job/master/badge/icon)](http://internal.zilliz.com:18080/jenkins/job/milvus-ci/job/master/)
+[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3563/badge)](https://bestpractices.coreinfrastructure.org/projects/3563)
+[![codecov](https://codecov.io/gh/milvus-io/milvus/branch/master/graph/badge.svg)](https://codecov.io/gh/milvus-io/milvus)
+[![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-milvus-io-milvus-master)
+[![CodeFactor Grade](https://www.codefactor.io/repository/github/milvus-io/milvus/badge)](https://www.codefactor.io/repository/github/milvus-io/milvus)
+[![Codacy Badge](https://api.codacy.com/project/badge/Grade/c4bb2ccfb51b47f99e43bfd1705edd95)](https://app.codacy.com/gh/milvus-io/milvus?utm_source=github.com&utm_medium=referral&utm_content=milvus-io/milvus&utm_campaign=Badge_Grade_Dashboard)
+[![Scrutinizer Quality Score](https://scrutinizer-ci.com/g/milvus-io/milvus/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/milvus-io/milvus/)
+
+[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
 
 [中文版](README_CN.md) | [日本語版](README_JP.md)
 
 ## What is Milvus
 
-Milvus is the world's fastest similarity search engine for massive-scale feature vectors. Built with heterogeneous computing architecture for the best cost efficiency. Searches over billion-scale vectors take only milliseconds with minimum computing resources.
+Milvus is an open source vector similarity search engine. Built with heterogeneous computing architecture for the best cost efficiency. Searches over billion-scale vectors take only milliseconds with minimum computing resources.
 
 For more detailed introduction of Milvus and its architecture, see [Milvus overview](https://www.milvus.io/docs/about_milvus/overview.md).
 
@@ -56,7 +68,7 @@ To edit Milvus settings, read [Milvus configuration](https://www.milvus.io/docs/
 
 ### Try your first Milvus program
 
-Try running a program with Milvus using [Python](https://www.milvus.io/docs/guides/get_started/example_code.md), [Java](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples), or [C++ example code](https://github.com/milvus-io/milvus/tree/master/sdk/examples). 
+Try running a program with Milvus using [Python](https://www.milvus.io/docs/guides/get_started/example_code.md), [Java](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples), or [C++ example code](https://github.com/milvus-io/milvus/tree/master/sdk/examples).
 
 
 ## Contribution guidelines
@@ -67,7 +79,7 @@ We use [GitHub issues](https://github.com/milvus-io/milvus/issues) to track issu
 
 ## Join our community
 
-:heart:To connect with other users and contributors, welcome to join our [Slack channel](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk). 
+:heart:To connect with other users and contributors, welcome to join our [Slack channel](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk).
 
 See our [community](https://github.com/milvus-io/community) repository to learn about our governance and access more community resources.
 

README_CN.md

@@ -1,26 +1,65 @@
-![Milvuslogo](https://raw.githubusercontent.com/milvus-io/docs/master/assets/milvus_logo.png)
+![Milvuslogo](https://github.com/milvus-io/docs/blob/master/assets/milvus_logo.png)
 
 [![Slack](https://img.shields.io/badge/Join-Slack-orange)](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk)
-![LICENSE](https://img.shields.io/badge/license-Apache--2.0-brightgreen)
-![Language](https://img.shields.io/badge/language-C%2B%2B-blue)
-[![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-jinhai-cn-milvus-master)
-![Release](https://img.shields.io/badge/release-v0.6.0-yellowgreen)
-![Release_date](https://img.shields.io/badge/release_date-December-yellowgreen)
+
+![GitHub](https://img.shields.io/github/license/milvus-io/milvus)
+![Language](https://img.shields.io/github/languages/count/milvus-io/milvus)
+![GitHub top language](https://img.shields.io/github/languages/top/milvus-io/milvus)
+![GitHub release (latest by date)](https://img.shields.io/github/v/release/milvus-io/milvus)
+![GitHub Release Date](https://img.shields.io/github/release-date/milvus-io/milvus)
+
+[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://github.com/milvus-io/milvus/pulse/monthly)
+![OSS Lifecycle](https://img.shields.io/osslifecycle/milvus-io/milvus)
+[![HitCount](http://hits.dwyl.com/milvus-io/milvus.svg)](http://hits.dwyl.com/milvus-io/milvus)
+![Docker pulls](https://img.shields.io/docker/pulls/milvusdb/milvus)
+
+[![Build Status](http://internal.zilliz.com:18080/jenkins/job/milvus-ci/job/master/badge/icon)](http://internal.zilliz.com:18080/jenkins/job/milvus-ci/job/master/)
+[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3563/badge)](https://bestpractices.coreinfrastructure.org/projects/3563)
 [![codecov](https://codecov.io/gh/milvus-io/milvus/branch/master/graph/badge.svg)](https://codecov.io/gh/milvus-io/milvus)
+[![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-milvus-io-milvus-master)
+[![Codacy Badge](https://api.codacy.com/project/badge/Grade/c4bb2ccfb51b47f99e43bfd1705edd95)](https://app.codacy.com/gh/milvus-io/milvus?utm_source=github.com&utm_medium=referral&utm_content=milvus-io/milvus&utm_campaign=Badge_Grade_Dashboard)
+[![Scrutinizer Quality Score](https://scrutinizer-ci.com/g/milvus-io/milvus/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/milvus-io/milvus/)
 
+[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
 
 # 欢迎来到 Milvus
 
 ## Milvus 是什么
 
-Milvus 是一款开源的、针对海量特征向量的相似性搜索引擎。基于异构众核计算框架设计，成本更低，性能更好。在有限的计算资源下，十亿向量搜索仅毫秒响应。
 
-若要了解 Milvus 详细介绍和整体架构，请访问 [Milvus 简介](https://www.milvus.io/cn/docs/about_milvus/overview.md)。
+Milvus 是一款开源的特征向量相似性搜索引擎。基于异构众核计算框架设计，成本更低，性能更好。在有限的计算资源下，十亿向量搜索仅毫秒响应。
 
-Milvus 提供稳定的 [Python](https://github.com/milvus-io/pymilvus)、[Java](https://github.com/milvus-io/milvus-sdk-java) 以及[C++](https://github.com/milvus-io/milvus/tree/master/sdk) 的 SDK。
+若要了解 Milvus 详细介绍和整体架构，请访问 [Milvus 简介](https://www.milvus.io/cn/docs/about_milvus/overview.md)。
 
 通过 [版本发布说明](https://www.milvus.io/cn/docs/v0.6.0/releases/v0.6.0.md) 获取最新版本的功能和更新。
 
+## 路线图
+
+您可以参考我们的[路线图](https://github.com/milvus-io/milvus/milestones)，了解 Milvus 即将实现的新特性。
+
+路线图尚未完成，并且可能会存在合理改动。我们欢迎各种针对路线图的意见、需求和建议。
+
+## 应用场景
+
+Milvus 可以广泛应用于各类场景。下图展示了基于 Milvus 和 VGG 构建的以图搜图系统。
+
+[![image retrieval demo](https://raw.githubusercontent.com/milvus-io/docs/v0.7.0/assets/image_retrieval.png)](https://raw.githubusercontent.com/milvus-io/docs/v0.7.0/assets/image_retrieval.png)
+
+您可以访问我们的[训练营](https://github.com/milvus-io/bootcamp)探索更多解决方案和应用场景。
+
+## 测试报告
+
+关于 Milvus 性能基准的更多信息，请参考[测试报告](https://github.com/milvus-io/milvus/tree/master/docs)。
+
+## 支持的客户端
+
+- [Go](https://github.com/milvus-io/milvus-sdk-go)
+- [Python](https://github.com/milvus-io/pymilvus)
+- [Java](https://github.com/milvus-io/milvus-sdk-java)
+- [C++](https://github.com/milvus-io/milvus/tree/master/sdk)
+- [RESTful API](https://github.com/milvus-io/milvus/tree/master/core/src/server/web_impl)
+- [Node.js](https://www.npmjs.com/package/@arkie-ai/milvus-client) (由 [arkie](https://www.arkie.cn/) 提供)
+
 ## 开始使用 Milvus
 
 请参阅 [Milvus 安装指南](https://www.milvus.io/cn/docs/guides/get_started/install_milvus/install_milvus.md) 使用 Docker 容器安装 Milvus。若要基于源码编译，请访问 [源码安装](INSTALL.md)。
@@ -29,7 +68,7 @@ Milvus 提供稳定的 [Python](https://github.com/milvus-io/pymilvus)、[Java](
 
 ### 开始您的第一个 Milvus 程序
 
-您可以尝试用 [Python](https://www.milvus.io/cn/docs/guides/get_started/example_code.md)，[Java](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples)，或者 [C++](https://github.com/milvus-io/milvus/tree/master/sdk/examples) 运行 Milvus 示例代码。
+您可以尝试用 [Python](https://www.milvus.io/cn/docs/guides/get_started/example_code.md)，[Java](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples)，[Go](https://github.com/milvus-io/milvus-sdk-go/tree/master/examples)，或者 [C++](https://github.com/milvus-io/milvus/tree/master/sdk/examples) 运行 Milvus 示例代码。
 
 
 ## 贡献者指南
@@ -61,6 +100,8 @@ Milvus 提供稳定的 [Python](https://github.com/milvus-io/pymilvus)、[Java](
 
 - [Milvus Facebook](https://www.facebook.com/io.milvus.5)
 
+- [Milvus 设计文档](design.md)
+
 ## 许可协议
 
 [Apache 许可协议2.0版](https://github.com/milvus-io/milvus/blob/master/LICENSE)