merge master

.github/workflows/core.yml

@@ -1,16 +1,23 @@
 name: Core
 
+# This workflow is triggered on pushes or pull request to the repository.
 on:
   push:
+    # file paths to consider in the event. Optional; defaults to all.
     paths:
       - 'ci/**'
       - 'core/**'
       - 'docker/**'
+      - '!**.md'
+      - '!ci/jenkins/**'
   pull_request:
+    # file paths to consider in the event. Optional; defaults to all.
     paths:
       - 'ci/**'
       - 'core/**'
       - 'docker/**'
+      - '!**.md'
+      - '!ci/jenkins/**'
 
 jobs:
   ubuntu:
@@ -23,6 +30,7 @@ jobs:
     env:
       UBUNTU: ${{ matrix.ubuntu }}
     steps:
+      # This step checks out a copy of your repository.
       - name: Checkout Milvus
         uses: actions/checkout@v1
       - name: Check Dockerfile

CHANGELOG.md

@@ -11,6 +11,7 @@ Please mark all change in change log and use the issue from GitHub
 - \#805 - IVFTest.gpu_seal_test unittest failed
 - \#831 - Judge branch error in CommonUtil.cpp
 - \#977 - Server crash when create tables concurrently
+- \#990 - check gpu resources setting when assign repeated value
 - \#995 - table count set to 0 if no tables found
 - \#1010 - improve error message when offset or page_size is equal 0
 - \#1022 - check if partition name is legal
@@ -19,6 +20,11 @@ Please mark all change in change log and use the issue from GitHub
 - \#1066 - optimize http insert and search speed
 - \#1067 - Add binary vectors support in http server
 - \#1152 - Error log output continuously after server start
+- \#1075 - improve error message when page size or offset is illegal
+- \#1082 - check page_size or offset value to avoid float
+- \#1115 - http server support load table into memory
+- \#1211 - Server down caused by searching with index_type: HNSW
+- \#1240 - Update license declaration
 
 ## Feature
 - \#216 - Add CLI to get server info
@@ -31,6 +37,7 @@ Please mark all change in change log and use the issue from GitHub
 - \#813 - Add push mode for prometheus monitor
 - \#815 - Support MinIO storage
 - \#823 - Support binary vector tanimoto/jaccard/hamming metric
+- \#853 - Support HNSW
 - \#910 - Change Milvus c++ standard to c++17
 - \#1204 - Add api to get table data information
 - \#1302 - Get all record IDs in a segment by given a segment id
@@ -49,6 +56,7 @@ Please mark all change in change log and use the issue from GitHub
 - \#1078 - Move 'insert_buffer_size' to Cache Config section
 - \#1297 - Hide partition_name parameter, avid user directly access partition table
 - \#1310 - Add default partition tag for a table
+- \#1105 - Error message is not clear when creating IVFSQ8H index without gpu resources
 
 ## Task
 

CODE_OF_CONDUCT.md

@@ -36,7 +36,7 @@ The Code of Conduct also applies within all project spaces and in public spaces
 
 ## Enforcement
 
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at support@zilliz.com. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at conduct@lfai.foundation. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
 
 Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
 

COMMUNITY.md

@@ -0,0 +1,55 @@
+# Community Roles
+
+<!-- TOC -->
+
+- [TSC Members](#tsc-members)
+- [Committers](#committers)
+- [Reviewers](#reviewers)
+- [Contributors](#contributors)
+
+<!-- /TOC -->
+
+## TSC Members
+
+The Technical Steering Committee (TSC) functions as the core management team that oversees the Milvus community. The TSC decides the roadmap of the project and makes major decisions related to the community.
+
+TSC members have the following responsibilities:
+
+- Coordinate the technical direction of the project.
+- Approve project or system proposals.
+- Decide on formal releases of the project's software.
+- Organize and remove sub-projects.
+- Coordinate any marketing, events, or communications regarding the project.
+
+To become a TSC member, a committer must be sponsored by a TSC member and the sponsorship must be approved by 2/3 of all TSC members.
+
+## Committers
+
+Committers lead project feature discussions and oversee the overall project quality.
+
+Committers have the following responsibilities:
+
+- Lead feature design discussions and implementation.
+- Ensure the overall project quality and approve PRs.
+- Participate in product release, feature planning, and roadmap design.
+- Have a constructive and friendly attitude in all community interactions.
+- Mentor reviewers and contributors.
+
+To become a committer, a reviewer must have contributed broadly throughout the Milvus project. A reviewer must also be sponsored by a committer and the sponsorship must be approved by the TSC.
+
+## Reviewers
+
+Reviewers review new code contributions and ensure the quality of existing code.
+
+Reviewers have the following responsibilities:
+
+- Participate in feature design discussion and implementation.
+- Ensure the quality of owned code modules.
+- Ensure the technical accuracy of documentation.
+- Quickly respond to issues and PRs and conduct code reviews.
+
+To become a reviewer, a contributor must have provided continued and quality contribution to the Milvus project for at least 6 months and have contributed at least one major component where the contributor has taken an ownership role.
+
+## Contributors
+
+Contributors can be anyone who has successfully submitted at least one PR to the Milvus project.

INSTALL.md

@@ -1,22 +1,37 @@
 # Install Milvus from Source Code
 
+<!-- TOC -->
+
 - [Build from source](#build-from-source)
+    - [Requirements](#requirements)
+    - [Compilation](#compilation)
+    - [Launch Milvus server](#launch-milvus-server)
 - [Compile Milvus on Docker](#compile-milvus-on-docker)
-
-If you encounter any problems/issues compiling Milvus from source, please refer to [Troubleshooting](#troubleshooting).
+    - [Step 1 Pull Milvus Docker images](#step-1-pull-milvus-docker-images)
+    - [Step 2 Start the Docker container](#step-2-start-the-docker-container)
+    - [Step 3 Download Milvus source code](#step-3-download-milvus-source-code)
+    - [Step 4 Compile Milvus in the container](#step-4-compile-milvus-in-the-container)
+- [Troubleshooting](#troubleshooting)
+    - [Error message: `protocol https not supported or disabled in libcurl`](#error-message-protocol-https-not-supported-or-disabled-in-libcurl)
+    - [Error message: `internal compiler error`](#error-message-internal-compiler-error)
+    - [Error message: `error while loading shared libraries: libmysqlpp.so.3`](#error-message-error-while-loading-shared-libraries-libmysqlppso3)
+    - [CMake version is not supported](#cmake-version-is-not-supported)
+
+<!-- /TOC -->
 
 ## Build from source
 
 ### Requirements
 
 - Ubuntu 18.04 or higher
+- CentOS 7
 
-  If your operating system is not Ubuntu 18.04 or higher, we recommend you to pull a [docker image of Ubuntu 18.04](https://docs.docker.com/install/linux/docker-ce/ubuntu/) as your compilation environment.
+If your operating system does not meet the requirements, we recommend that you pull a Docker image of [Ubuntu 18.04](https://docs.docker.com/install/linux/docker-ce/ubuntu/) or [CentOS 7](https://docs.docker.com/install/linux/docker-ce/centos/) as your compilation environment.
 
-- GCC 7.0 or higher to support C++17
+- GCC 7.0 or higher to support C++ 17
 - CMake 3.12 or higher
 
-##### For GPU-enabled version, you will also need:
+For GPU-enabled version, you will also need:
 
 - CUDA 10.0 or higher
 - NVIDIA driver 418 or higher
@@ -32,19 +47,34 @@ $ ./ubuntu_build_deps.sh
 
 #### Step 2 Build
 
+##### Build in Ubuntu
+
 ```shell
 $ cd [Milvus root path]/core
 $ ./build.sh -t Debug
+```
+
 or 
+
+```shell
 $ ./build.sh -t Release
 ```
 
-By default, it will build CPU-only version. To build GPU version, add `-g` option
+##### Build in CentOS
+
+```shell
+$ cd [Milvus root path]/core
+$ ./centos7_build_deps.sh
+```
+
+By default, it will build CPU-only version. To build GPU version, add `-g` option.
+
 ```shell
 $ ./build.sh -g
 ```
 
-If you want to know the complete build options, run
+If you want to know the complete build options, run the following command.
+
 ```shell
 $./build.sh -h
 ```
@@ -78,33 +108,33 @@ $ ./stop_server.sh
 
 ## Compile Milvus on Docker
 
-With the following Docker images, you should be able to compile Milvus on any Linux platform that run Docker. To build a GPU supported Milvus, you neeed to install [NVIDIA Docker](https://github.com/NVIDIA/nvidia-docker/) first.
+With the following Docker images, you should be able to compile Milvus on any Linux platform that run Docker. To build a GPU supported Milvus, you need to install [NVIDIA Docker](https://github.com/NVIDIA/nvidia-docker/) first.
 
 ### Step 1 Pull Milvus Docker images
 
 Pull CPU-only image:
 
 ```shell
-$ docker pull milvusdb/milvus-cpu-build-env:v0.6.0-ubuntu18.04
+$ docker pull milvusdb/milvus-cpu-build-env:latest
 ```
 
 Pull GPU-enabled image:
 
 ```shell
-$ docker pull milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04
+$ docker pull milvusdb/milvus-gpu-build-env:latest
 ```
 ### Step 2 Start the Docker container
 
 Start a CPU-only container:
 
 ```shell
-$ docker run -it -p 19530:19530 -d milvusdb/milvus-cpu-build-env:v0.6.0-ubuntu18.04
+$ docker run -it -p 19530:19530 -d milvusdb/milvus-cpu-build-env:latest
 ```
 
 Start a GPU container:
 
 ```shell
-$ docker run --runtime=nvidia -it -p 19530:19530 -d milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04
+$ docker run --runtime=nvidia -it -p 19530:19530 -d milvusdb/milvus-gpu-build-env:latest
 ```
 To enter the container:
 
@@ -113,50 +143,82 @@ $ docker exec -it [container_id] bash
 ```
 ### Step 3 Download Milvus source code
 
-Download Milvus source code:
+Download latest Milvus source code:
 
 ```shell
 $ cd /home
-$ wget https://github.com/milvus-io/milvus/archive/0.6.0.tar.gz
+$ git clone https://github.com/milvus-io/milvus
 ```
 
-Extract the source package:
+To enter its core directory:
 
 ```shell
-$ tar xvf ./v0.6.0.tar.gz
+$ cd ./milvus/core
 ```
 
-The source code is extracted into a folder called `milvus-0.6.0`. To enter its core directory:
-
-```shell
-$ cd ./milvus-0.6.0/core
-```
 ### Step 4 Compile Milvus in the container
 
 If you are using a CPU-only image, compile it like this:
+
 ```shell
 $ ./build.sh -t Release
 ```
 
 If you are using a GPU-enabled image, you need to add a `-g` parameter:
+
 ```shell
 $ ./build.sh -g -t Release
 ```
 
 Then start Milvus server：
+
 ```shell
 $ ./start_server.sh
 ```
 
 ## Troubleshooting
-1. If you encounter the following error when compiling: 
-`protocol https not supported or disabled in libcurl`.
-First, make sure you have `libcurl4-openssl-dev` installed in your system.
-Then try reinstalling the latest CMake from source with `--system-curl` option:
+
+### Error message: `protocol https not supported or disabled in libcurl`
+
+Follow the steps below to solve this problem:
+
+1. Make sure you have `libcurl4-openssl-dev` installed in your system.
+2. Try reinstalling the latest CMake from source with `--system-curl` option:
 
    ```shell
-   $ ./bootstrap --system-curl 
-   $ make 
+   $ ./bootstrap --system-curl
+   $ make
    $ sudo make install
    ```
-If the `--system-curl` command doesn't work, you can also reinstall CMake in **Ubuntu Software** on your local computer.
+
+   If the `--system-curl` command doesn't work, you can also reinstall CMake in **Ubuntu Software** on your local computer.
+
+### Error message: `internal compiler error`
+
+Try increasing the memory allocated to docker.
+
+### Error message: `error while loading shared libraries: libmysqlpp.so.3`
+
+Follow the steps below to solve this problem:
+
+1. Check whether `libmysqlpp.so.3` is correctly installed.
+2. If `libmysqlpp.so.3` is installed, check whether it is added to `LD_LIBRARY_PATH`.
+
+### CMake version is not supported
+
+Follow the steps below to install a supported version of CMake:
+
+1. Remove the unsupported version of CMake.
+2. Get CMake 3.12 or higher. Here we get CMake 3.12.
+
+    ```shell
+    $ wget https://cmake.org/files/v3.12/cmake-3.12.2-Linux-x86_64.tar.gz
+    ```
+
+3. Extract the file and install CMake.
+
+    ```shell
+    $ tar zxvf cmake-3.12.2-Linux-x86_64.tar.gz
+    $ mv cmake-3.12.2-Linux-x86_64 /opt/cmake-3.12.2
+    $ ln -sf /opt/cmake-3.12.2/bin/* /usr/bin/
+    ```