Add SchemaRegion Interfaces to SchemaEngine#5349
Merged
qiaojialin merged 17 commits intoapache:masterfrom Mar 28, 2022
Merged
Conversation
MarcosZyk
force-pushed
the
schema_region
branch
from
7d5e547 to
c378359
Compare
…region � Conflicts: � node-commons/src/main/java/org/apache/iotdb/commons/partition/SchemaRegionId.java
qiaojialin
approved these changes
Mar 28, 2022
| new File(logFilePath).getParentFile().mkdirs(); | ||
| Whitebox.setInternalState(schemaEngine, "logFilePath", logFilePath); | ||
| schemaEngine.initForMultiSchemaEngineTest(); | ||
| // schemaEngine.initForMultiSchemaEngineTest(); |
Comment on lines
+40
to
+41
| if (this == o) return true; | ||
| if (o == null || getClass() != o.getClass()) return false; |
| * This class simulates the behaviour of configNode to manage the schema configs locally. The schema | ||
| * configs include storage group, schema region and template. | ||
| */ | ||
| public class LocalSchemaConfigManager { |
Member
There was a problem hiding this comment.
Suggested change
| public class LocalSchemaConfigManager { | |
| public class LocalConfigManager { |
| import java.util.concurrent.ConcurrentHashMap; | ||
|
|
||
| // manage all the schemaRegion in this dataNode | ||
| public class SchemaRegionManager { |
Member
There was a problem hiding this comment.
Suggested change
| public class SchemaRegionManager { | |
| public class SchemaEngine { |
xinzhongtianxia
pushed a commit
to xinzhongtianxia/iotdb
that referenced
this pull request
Apr 5, 2022
…er_meta * remotes/upstream/master: Rename SchemaEngine to LocalSchemaProcessor and Rename SchemaRegionManager to SchemaEngine (apache#5359) Add SchemaRegion Interfaces to SchemaEngine (apache#5349) [IOTDB-2803] adapt metadata write operation to plannode (apache#5344) [IOTDB-2679] Support logical operators in select clauses (apache#5273) [IOTDB-2671][metrics] Improve the implementation of iotdbReporter (apache#5338) # Conflicts: # server/src/main/java/org/apache/iotdb/db/mpp/sql/planner/LogicalPlanner.java # server/src/test/java/org/apache/iotdb/db/mpp/sql/plan/LogicalPlannerTest.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Implement LocalConfigManager
Add SchemaRegion Management to LocalConfigManager
Refactor Storage Group Management
The persistent log records of storage group operation has been separated and stored in storage_group_log.bin. The folder of storage group will be created only one there's schemaRegion needed to be create.
Future work
LocalConfigManager
Rename
Rename current SchemaEngine to LocalSchemaProcessor to act only as the proxy of schema query.
Rename SchemaRegionManager to SchemaEngine to manage schemaRegion.
The development of new cluster will based on LocalConfigManager, SchemaRegion and new SchemaEngine.
Template
I'tried to implement multi schemaRegions for one storage group, which is quite similar with the schemaRegion management and metadata operation implementation in cluster mode. Everything worked fine except one feature, Template, which is too complex to be handled, and the corresponding scheme hasn't been design in new cluster.
The template can be set to any Internal node under sg in mtree, and this is the main difficulty. In current schema partition design, an Internal node may exists in several schemaRegions of one storage group, since the schema is partitioned by device hash.
I've designed and implement a version but it isn't worth enough to be mocked or applied on standalone mode. Hope there will be a nice solution.