Support materialized view schema change #3739
Conversation
…oris into schema_change_mv
…ubator-doris into schema_change_mv
HangyuanLiu
changed the title
…ubator-doris into schema_change_mv
HangyuanLiu
force-pushed
the
schema_change_mv
branch
from
67c1d76 to
4f72f88
Compare
HangyuanLiu
added
area/materialized-view
…ubator-doris into schema_change_mv
gensrc/thrift/AgentService.thrift
Outdated
| struct TAlterMaterializedViewParam { | ||
| 1: required string column_name | ||
| 2: required string origin_column_name | ||
| 3: optional string mv_expr |
There was a problem hiding this comment.
The materialized view is a calculation of an expression.But we construct materialized view in olap/schema_change.cpp. we can't use expr and exec in olap layer now.So I used some hard coding to deal with this problem.May be we can refactor this part of the code until the uniform expression evaluates the logic.
| switch (ref_column.type()) { | ||
| case OLAP_FIELD_TYPE_TINYINT: | ||
| if (*(int8_t *) src < 0) { | ||
| LOG(WARNING) << "The input: " << *(int8_t *) src |
There was a problem hiding this comment.
How about return Status with this error message?
And do not print log here, it may lead to lots of log, and this error info should return to user.
There was a problem hiding this comment.
This is a good suggestion，but it involves a lot of modification points, I will add this feature optimization in the next PR
There was a problem hiding this comment.
How about return Status with this error message?
And do not print log here, it may lead to lots of log, and this error info should return to user.
+1
…/HangyuanLiu/incubator-doris into mv_t1 � Conflicts: � fe/src/main/java/org/apache/doris/task/AlterReplicaTask.java
| switch (ref_column.type()) { | ||
| case OLAP_FIELD_TYPE_TINYINT: | ||
| if (*(int8_t *) src < 0) { | ||
| LOG(WARNING) << "The input: " << *(int8_t *) src |
There was a problem hiding this comment.
How about return Status with this error message?
And do not print log here, it may lead to lots of log, and this error info should return to user.
+1
…oris into schema_change_mv
…#3739] (#3873) + Building the materialized view function for schema_change here based on defineExpr. + This is a trick because the current storage layer does not support expression evaluation. + count distinct materialized view will set mv_expr with to_bitmap or hll_hash. + count materialized view will set mv_expr with count. + Support to regenerate historical data when a new materialized view is created in BE。 + Support to_bitmap function + Support hll_hash function + Support count(field) function For #3344
Support materialized view schema change history data
1、bitmap
2、hll
3、count