fix: run addindex failed

docs/en/reference/sql/ddl/CREATE_INDEX_STATEMENT.md

@@ -1,7 +1,6 @@
 # CREATE INDEX
 
-The `CREATE INDEX` statement is used to create a new index on existing table. If there is data in the table, data will be loaded asynchronously. 
-The job status can be checked through the `showopstatus` command of `ns_client`, see [Operations in CLI](../../../maintain/cli.md#showopstatus).
+The `CREATE INDEX` statement is used to create a new index on existing table. Running `CREATE INDEX` will initiates an asynchronous job, and you can check the status of the job by executing `SHOW JOBS FROM NAMESERVER`.
 
 ## Syntax
 
@@ -48,7 +47,8 @@ CREATE INDEX index2 ON t5 (col2);
 -- SUCCEED
 ```
 ```{note}
-If `OPTIONS` is not provided, the SQL with the created index cannot be deployed online, since the index doesn't have TS (timestamp).
+1. If `OPTIONS` is not provided, the SQL with the created index cannot be deployed online, since the index doesn't have TS (timestamp).
+2. The data type of `TS` column should be BigInt or Timestamp.
 ```
 We can also set `TS` column as below:
 ```SQL
@@ -59,4 +59,4 @@ Please refer [here](./CREATE_TABLE_STATEMENT.md) for more details about `TTL` an
 
 ## Related SQL
 
-[DROP INDEX](./DROP_INDEX_STATEMENT.md)
+[DROP INDEX](./DROP_INDEX_STATEMENT.md)

docs/en/reference/sql/ddl/CREATE_TABLE_STATEMENT.md

@@ -226,7 +226,7 @@ The index key must be configured, and other configuration items are optional. Th
 | Configuration Item        | Note                                                                                                                                                                                                                                                                                                                        | Expression                                                                                      | Example                                                                                                                    |
 |------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------|
 | `KEY`      | It defines the index column (required). OpenMLDB supports single-column indexes as well as joint indexes. When `KEY`=one column, a single-column index is configured. When `KEY`=multiple columns, the joint index of these columns is configured: several columns are concatenated into a new string as an index in order. | Single-column index: `ColumnName`<br/>Joint index: <br/>`(ColumnName (, ColumnName)* ) `        | Single-column index: `INDEX(KEY=col1)`<br />Joint index: `INDEX(KEY=(col1, col2))`                                         |
-| `TS`       | It defines the index time column (optional). Data on the same index will be sorted by the index time column. When `TS` is not explicitly configured, the timestamp of data insertion is used as the index time.                                                                                                             | `ColumnName`                                                                                    | `INDEX(KEY=col1, TS=std_time)`。 The index column is col1, and the data rows with the same col1 value are sorted by std_time. |
+| `TS`       | It defines the index time column (optional). Data on the same index will be sorted by the index time column. When `TS` is not explicitly configured, the timestamp of data insertion is used as the index time. The data type of time column should be BigInt or Timestamp                                                                                                             | `ColumnName`                                                                                    | `INDEX(KEY=col1, TS=std_time)`。 The index column is col1, and the data rows with the same col1 value are sorted by std_time. |
 | `TTL_TYPE` | It defines the elimination rules (optional). Including four types. When `TTL_TYPE` is not explicitly configured, the `ABSOLUTE` expiration configuration is used by default.                                                                                                                                                | Supported expr: `ABSOLUTE` <br/> `LATEST`<br/>`ABSORLAT`<br/> `ABSANDLAT`。                      | For specific usage, please refer to **Configuration Rules for TTL and TTL_TYP** below.                                     |
 | `TTL`      | It defines the maximum survival time/number. Different TTL_TYPEs determines different `TTL` configuration methods. When `TTL` is not explicitly configured, `TTL=0` which means OpenMLDB will not evict records.                                                                                                            | Supported expr: `int_literal`<br/>  `interval_literal`<br/>`( interval_literal , int_literal )` | For specific usage, please refer to "Configuration Rules for TTL and TTL_TYPE" below.                                      |
 

docs/zh/openmldb_sql/ddl/CREATE_INDEX_STATEMENT.md

@@ -1,7 +1,6 @@
 # CREATE INDEX
 
-`CREATE INDEX` 语句用来创建索引。 如果表里有数据，添加索引会发起异步任务来加载数据。
-通过`ns_client`中的`showopstatus`命令可以查看任务状态，详见[运维 CLI](../../maintain/cli.md#showopstatus)。
+`CREATE INDEX` 语句用来创建索引。添加索引会发起异步任务来加载数据, 可以通过执行`SHOW JOBS FROM NAMESERVER`来查看任务状态
 
 ## 语法
 
@@ -40,15 +39,14 @@ OptionEntry ::=
 
 ```
 
-
-
 ## **示例**
 ```SQL
 CREATE INDEX index2 ON t5 (col2);
 -- SUCCEED
 ```
 ```{note}
-如果不指定Options, 创建的索引就没有指定`TS`列，因此不能用在需要上线的SQL中。
+1. 如果不指定Options, 创建的索引就没有指定`TS`列，因此不能用在需要上线的SQL中。
+2. 指定`TS`列的类型只能是BitInt或者Timestamp
 ```
 我们可以通过类似如下命令在创建索引时指定`TS`列:
 ```SQL
@@ -59,4 +57,4 @@ CREATE INDEX index3 ON t5 (col3) OPTIONS (ts=ts1, ttl_type=absolute, ttl=30d);
 
 ## 相关SQL
 
-[DROP INDEX](./DROP_INDEX_STATEMENT.md)
+[DROP INDEX](./DROP_INDEX_STATEMENT.md)

docs/zh/openmldb_sql/ddl/CREATE_TABLE_STATEMENT.md

@@ -223,7 +223,7 @@ IndexOption ::=
 | 配置项        | 描述                                                                                                      | expr                                                                                                           | 用法示例                                                                                   |
 |------------|---------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|
 | `KEY`      | 索引列（必选）。OpenMLDB支持单列索引，也支持联合索引。当`KEY`后只有一列时，仅在该列上建立索引。当`KEY`后有多列时，建立这几列的联合索引：将多列按顺序拼接成一个字符串作为索引。        | 支持单列索引：`ColumnName`<br/>或联合索引：<br/>`(ColumnName (, ColumnName)* ) `                                            | 单列索引：`INDEX(KEY=col1)`<br />联合索引：`INDEX(KEY=(col1, col2))`                             |
-| `TS`       | 索引时间列（可选）。同一个索引上的数据将按照时间索引列排序。当不显式配置`TS`时，使用数据插入的时间戳作为索引时间。                                             | `ColumnName`                                                                                                   | `INDEX(KEY=col1, TS=std_time)`。索引列为col1,col1相同的数据行按std_time排序。                         |
+| `TS`       | 索引时间列（可选）。同一个索引上的数据将按照时间索引列排序。当不显式配置`TS`时，使用数据插入的时间戳作为索引时间。时间列的类型只能为BigInt或者Timestamp                                             | `ColumnName`                                                                                                   | `INDEX(KEY=col1, TS=std_time)`。索引列为col1,col1相同的数据行按std_time排序。                         |
 | `TTL_TYPE` | 淘汰规则（可选）。包括四种类型，当不显式配置`TTL_TYPE`时，默认使用`ABSOLUTE`过期配置。                                                   | 支持的expr如下：`ABSOLUTE` <br/> `LATEST`<br/>`ABSORLAT`<br/> `ABSANDLAT`。                                           | 具体用法可以参考下文“TTL和TTL_TYPE的配置细则”                                                          |
 | `TTL`      | 最大存活时间/条数（可选）。依赖于`TTL_TYPE`，不同的`TTL_TYPE`有不同的`TTL` 配置方式。当不显式配置`TTL`时，`TTL=0`，表示不设置淘汰规则，OpenMLDB将不会淘汰记录。 | 支持数值：`int_literal`<br/>  或数值带时间单位(`S,M,H,D`)：`interval_literal`<br/>或元组形式：`( interval_literal , int_literal )` |具体用法可以参考下文“TTL和TTL_TYPE的配置细则” |
 

src/base/status.h

@@ -88,6 +88,7 @@ enum ReturnCode {
     kCreateFunctionFailed = 159,
     kExceedMaxMemory = 160,
     kInvalidArgs = 161,
+    kCheckIndexFailed = 162,
     kNameserverIsNotLeader = 300,
     kAutoFailoverIsEnabled = 301,
     kEndpointIsNotExist = 302,
@@ -122,6 +123,7 @@ enum ReturnCode {
     kCheckParameterFailed = 331,
     kCreateProcedureFailedOnTablet = 332,
     kCreateFunctionFailedOnTablet = 333,
+    kOPAlreadyExists = 317,
     kReplicaClusterAliasDuplicate = 400,
     kConnectRelicaClusterZkFailed = 401,
     kNotSameReplicaName = 402,

src/nameserver/name_server_impl.cc

@@ -8270,14 +8270,13 @@ void NameServerImpl::DeleteIndex(RpcController* controller, const DeleteIndexReq
             tablet_client_map.insert(std::make_pair(kv.second->client_->GetEndpoint(), kv.second->client_));
         }
     }
-    for (int idx = 0; idx < table_info->table_partition_size(); idx++) {
-        for (int meta_idx = 0; meta_idx < table_info->table_partition(idx).partition_meta_size(); meta_idx++) {
-            std::string endpoint = table_info->table_partition(idx).partition_meta(meta_idx).endpoint();
-            if (!table_info->table_partition(idx).partition_meta(meta_idx).is_alive()) {
+    for (const auto& table_partition : table_info->table_partition()) {
+        for (const auto& partition_meta : table_partition.partition_meta()) {
+            const std::string& endpoint = partition_meta.endpoint();
+            if (!partition_meta.is_alive()) {
                 response->set_code(::openmldb::base::ReturnCode::kTableHasNoAliveLeaderPartition);
                 response->set_msg("partition is not alive!");
-                PDLOG(WARNING, "partition[%s][%d] is not alive!", endpoint.c_str(),
-                      table_info->table_partition(idx).pid());
+                PDLOG(WARNING, "partition[%s][%d] is not alive!", endpoint.c_str(), table_partition.pid());
                 return;
             }
             if (tablet_client_map.find(endpoint) == tablet_client_map.end()) {
@@ -8289,14 +8288,14 @@ void NameServerImpl::DeleteIndex(RpcController* controller, const DeleteIndexReq
         }
     }
     bool delete_failed = false;
-    for (int idx = 0; idx < table_info->table_partition_size(); idx++) {
-        for (int meta_idx = 0; meta_idx < table_info->table_partition(idx).partition_meta_size(); meta_idx++) {
-            std::string endpoint = table_info->table_partition(idx).partition_meta(meta_idx).endpoint();
+    for (const auto& table_partition : table_info->table_partition()) {
+        for (const auto& partition_meta : table_partition.partition_meta()) {
+            const std::string& endpoint = partition_meta.endpoint();
             std::string msg;
-            if (!tablet_client_map[endpoint]->DeleteIndex(table_info->tid(), table_info->table_partition(idx).pid(),
+            if (!tablet_client_map[endpoint]->DeleteIndex(table_info->tid(), table_partition.pid(),
                                                           request->idx_name(), &msg)) {
-                PDLOG(WARNING, "delete index failed. name %s pid %u endpoint %s msg %s", request->table_name().c_str(),
-                      table_info->table_partition(idx).pid(), endpoint.c_str(), msg.c_str());
+                PDLOG(WARNING, "delete index failed. name %s pid %u endpoint %s msg %s",
+                        request->table_name().c_str(), table_partition.pid(), endpoint.c_str(), msg.c_str());
                 delete_failed = true;
             }
         }
@@ -8523,8 +8522,21 @@ void NameServerImpl::AddIndex(RpcController* controller, const AddIndexRequest*
         openmldb::common::VersionPair* pair = table_info->add_schema_versions();
         pair->CopyFrom(new_pair);
     }
+    if (auto status = schema::IndexUtil::CheckIndex(col_map,
+                schema::IndexUtil::Convert2PB(column_key_vec)); !status.OK()) {
+        base::SetResponseStatus(ReturnCode::kCheckIndexFailed, status.msg, response);
+        LOG(WARNING) << status.msg;
+        return;
+    }
     if (IsClusterMode() && !request->skip_load_data()) {
         std::lock_guard<std::mutex> lock(mu_);
+        if (IsExistActiveOp(db, name, api::kAddIndexOP)) {
+            LOG(WARNING) << "create AddIndexOP failed. there is already a task running. db "
+                << db << " table " << name;
+            base::SetResponseStatus(ReturnCode::kOPAlreadyExists,
+                    "there is already a task running", response);
+            return;
+        }
         auto status = CreateAddIndexOP(name, db, column_key_vec);
         if (!status.OK()) {
             LOG(WARNING) << "create AddIndexOP failed, table " << name << " msg " << status.GetMsg();
@@ -10480,8 +10492,14 @@ void NameServerImpl::DeploySQL(RpcController* controller, const DeploySQLRequest
             }
         }
     }
-    uint64_t op_id = 0;
     std::lock_guard<std::mutex> lock(mu_);
+    if (IsExistActiveOp(db, "", api::OPType::kDeployOP)) {
+        LOG(WARNING) << "create DeployOP failed. there is already a task running in db " << db;
+        base::SetResponseStatus(ReturnCode::kOPAlreadyExists,
+                "there is already a task running", response);
+        return;
+    }
+    uint64_t op_id = 0;
     auto status = CreateDeployOP(*request, &op_id);
     if (!status.OK()) {
         PDLOG(WARNING, "%s", status.GetMsg().c_str());
@@ -10497,7 +10515,7 @@ base::Status NameServerImpl::CreateDeployOP(const DeploySQLRequest& request, uin
     std::string value;
     auto op_type = api::OPType::kDeployOP;
     if (CreateOPData(op_type, value, op_data, sp_info.main_table(), sp_info.db_name(), 0) < 0) {
-        return {-1, absl::StrCat("create AddIndexOP data error. deploy name ", deploy_name)};
+        return {-1, absl::StrCat("create DeployOP data error. deploy name ", deploy_name)};
     }
     auto task = CreateTask<AddMultiTableIndexTaskMeta>(op_data->GetOpId(), op_type, request.index());
     if (!task) {
@@ -10517,5 +10535,29 @@ base::Status NameServerImpl::CreateDeployOP(const DeploySQLRequest& request, uin
     return {};
 }
 
+bool NameServerImpl::IsExistActiveOp(const std::string& db, const std::string& name, api::OPType op_type) {
+    for (const auto& op_list : task_vec_) {
+        if (op_list.empty()) {
+            continue;
+        }
+        for (const auto& op_data : op_list) {
+            if (op_data->op_info_.op_type() != op_type) {
+                continue;
+            }
+            if (!db.empty() && op_data->op_info_.db() != db) {
+                continue;
+            }
+            if (!name.empty() && op_data->op_info_.name() != name) {
+                continue;
+            }
+            if (op_data->op_info_.task_status() == api::TaskStatus::kInited ||
+                    op_data->op_info_.task_status() == api::TaskStatus::kDoing) {
+                return true;
+            }
+        }
+    }
+    return false;
+}
+
 }  // namespace nameserver
 }  // namespace openmldb

src/nameserver/name_server_impl.h

@@ -682,6 +682,8 @@ class NameServerImpl : public NameServer {
 
     bool IsExistDataBase(const std::string& db);
 
+    bool IsExistActiveOp(const std::string& db, const std::string& name, api::OPType op_type);
+
  private:
     std::mutex mu_;
     Tablets tablets_;

src/schema/index_util.cc

@@ -50,11 +50,14 @@ base::Status IndexUtil::CheckIndex(const std::map<std::string, ::openmldb::commo
             col_set.insert(column_name);
             has_iter = true;
             auto iter = column_map.find(column_name);
-            if ((iter != column_map.end() &&
-                 ((iter->second.data_type() == ::openmldb::type::kFloat)
-                  || (iter->second.data_type() == ::openmldb::type::kDouble)))) {
-                return {base::ReturnCode::kError,
-                    "float or double type column can not be index, column is: " + column_key.index_name()};
+            if (iter != column_map.end()) {
+                if (iter->second.data_type() == ::openmldb::type::kFloat
+                        || iter->second.data_type() == ::openmldb::type::kDouble) {
+                    return {base::ReturnCode::kError,
+                        "float or double type column can not be index, column is: " + column_key.index_name()};
+                }
+            } else {
+                return {base::ReturnCode::kError, "can not find col in schema. col: " + column_name};
             }
         }
         if (!has_iter) {
@@ -67,6 +70,15 @@ base::Status IndexUtil::CheckIndex(const std::map<std::string, ::openmldb::commo
                 return {base::ReturnCode::kError, "float or double column can not be index"};
             }
         }
+        if (column_key.has_ts_name()) {
+            if (auto iter = column_map.find(column_key.ts_name()); iter == column_map.end()) {
+                return {base::ReturnCode::kError, "can not find col in schema. col: " + column_key.ts_name()};
+            } else if (iter->second.data_type() != ::openmldb::type::kBigInt
+                  && iter->second.data_type() != ::openmldb::type::kTimestamp) {
+                return {base::ReturnCode::kError,
+                    "ts column type should be bigint or timestamp, column is: " + column_key.ts_name()};
+            }
+        }
         if (column_key.has_ttl()) {
             if (!CheckTTL(column_key.ttl())) {
                 return {base::ReturnCode::kError, "ttl check failed"};
@@ -177,6 +189,14 @@ std::vector<::openmldb::common::ColumnKey> IndexUtil::Convert2Vector(const PBInd
     return vec;
 }
 
+PBIndex IndexUtil::Convert2PB(const std::vector<::openmldb::common::ColumnKey>& index) {
+    PBIndex pb_index;
+    for (const auto& column_key : index) {
+        pb_index.Add()->CopyFrom(column_key);
+    }
+    return pb_index;
+}
+
 std::string IndexUtil::GetIDStr(const ::openmldb::common::ColumnKey& column_key) {
     std::string id_str;
     for (const auto& cur_col : column_key.col_name()) {

src/schema/index_util.h

@@ -42,6 +42,8 @@ class IndexUtil {
 
     static std::vector<::openmldb::common::ColumnKey> Convert2Vector(const PBIndex& index);
 
+    static PBIndex Convert2PB(const std::vector<::openmldb::common::ColumnKey>& index);
+
     static base::Status CheckUnique(const PBIndex& index);
 
     static bool CheckTTL(const ::openmldb::common::TTLSt& ttl);