Skip to content

Add a plan version field to QueryRequest Protobuf Message#13267

Merged
Jackie-Jiang merged 6 commits intoapache:masterfrom
vrajat:rv_plan_version
May 31, 2024
Merged

Add a plan version field to QueryRequest Protobuf Message#13267
Jackie-Jiang merged 6 commits intoapache:masterfrom
vrajat:rv_plan_version

Conversation

@vrajat
Copy link
Contributor

@vrajat vrajat commented May 30, 2024

#13221 will provide backward and forward compatibility of the serialization format as long as best practices are followed. However, this is not sufficient to support rolling upgrades for V2 workloads.

V2 Plan versioning is required because the semantics of a plan may change. Some examples are:

  • New plan nodes may be generated for the same query.
  • Behavior of a plan node may change.
  • Class definition (member variables) may change.

This implies that newer brokers & servers have to support multiple plan versions and they have to choose to use older plan versions while the upgrade is in progress.

As a first step, a plan version field is added to QueryRequest message as a string field. The field can be used to make sure that two components are communicating using the same version.

In the future, the best practices on how to manage plan versions as well as features to choose the right plan versions in mixed mode clusters will be implemented.

Tags:
multi-stage

@vrajat vrajat changed the title Add a plan version field to QueryRequest Protobug Message Add a plan version field to QueryRequest Protobuf Message May 30, 2024
@codecov-commenter
Copy link

codecov-commenter commented May 30, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 62.14%. Comparing base (59551e4) to head (dd49f3f).
Report is 522 commits behind head on master.

Files Patch % Lines
...va/org/apache/pinot/spi/utils/CommonConstants.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #13267      +/-   ##
============================================
+ Coverage     61.75%   62.14%   +0.39%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2536     +100     
  Lines        133233   139403    +6170     
  Branches      20636    21548     +912     
============================================
+ Hits          82274    86634    +4360     
- Misses        44911    46280    +1369     
- Partials       6048     6489     +441     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration <0.01% <0.00%> (-0.01%) ⬇️
integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration2 0.00% <0.00%> (ø)
java-11 62.09% <50.00%> (+0.39%) ⬆️
java-21 62.03% <50.00%> (+0.41%) ⬆️
skip-bytebuffers-false 62.13% <50.00%> (+0.39%) ⬆️
skip-bytebuffers-true 62.00% <50.00%> (+34.27%) ⬆️
temurin 62.14% <50.00%> (+0.39%) ⬆️
unittests 62.14% <50.00%> (+0.39%) ⬆️
unittests1 46.71% <50.00%> (-0.18%) ⬇️
unittests2 27.74% <0.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jackie-Jiang Jackie-Jiang added the multi-stage Related to the multi-stage query engine label May 31, 2024
@Jackie-Jiang Jackie-Jiang merged commit f54c77d into apache:master May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

multi-stage Related to the multi-stage query engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants