Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

Enable changing the compiled_query_execution setting at runtime #1609

Merged
merged 6 commits into from
Jun 7, 2021

Conversation

linmagit
Copy link
Member

@linmagit linmagit commented Jun 6, 2021

This PR adds a CompiledQueryExecution SettingsCallback to change the execution_mode_ setting in TrafficCop. So that at run time, we can use SET command to change the compiled_query_execution setting.

Note that with the existing system architecture, the system only compiles the query when it executes the query as Compiled mode for the first time, instead of when constructing the ExecutableQuery. Once the query has been executed as Compiled once, it can switch between the Interpert and Compiled modes without more compilation.

@linmagit linmagit added ready-for-review This PR passes all checks and is ready to be reviewed. Mark PRs with this. ready-for-ci Indicate that this build should be run through CI. labels Jun 6, 2021
@linmagit linmagit self-assigned this Jun 6, 2021
@linmagit linmagit requested a review from lmwnshn June 6, 2021 18:52
@noisepage-checks
Copy link

Minor Decrease in Performance

Be warned: this PR may have decreased the throughput of the system slightly.

tps (%change) benchmark_type wal_device details
-2.61% tpcc RAM disk
Detailsmaster tps=22707.64, commit tps=22114.81, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=RAM disk, max_connection_threads=32
-0.25% tpcc None
Detailsmaster tps=29494.04, commit tps=29419.98, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=None, max_connection_threads=32
0.25% tpcc HDD
Detailsmaster tps=21984.94, commit tps=22040.35, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=HDD, max_connection_threads=32
11.61% tatp RAM disk
Detailsmaster tps=6400.58, commit tps=7143.91, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=RAM disk, max_connection_threads=32
5.63% tatp None
Detailsmaster tps=7240.29, commit tps=7647.8, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=None, max_connection_threads=32
9.26% tatp HDD
Detailsmaster tps=6284.11, commit tps=6865.81, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=HDD, max_connection_threads=32

@noisepage-checks
Copy link

Minor Decrease in Performance

Be warned: this PR may have decreased the throughput of the system slightly.

tps (%change) benchmark_type wal_device details
0.43% tpcc RAM disk
Detailsmaster tps=22707.64, commit tps=22806.12, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=RAM disk, max_connection_threads=32
0.8% tpcc None
Detailsmaster tps=29494.04, commit tps=29730.66, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=None, max_connection_threads=32
-2.05% tpcc HDD
Detailsmaster tps=21984.94, commit tps=21535.27, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=HDD, max_connection_threads=32
9.65% tatp RAM disk
Detailsmaster tps=6400.58, commit tps=7018.23, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=RAM disk, max_connection_threads=32
3.3% tatp None
Detailsmaster tps=7240.29, commit tps=7479.32, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=None, max_connection_threads=32
8.37% tatp HDD
Detailsmaster tps=6284.11, commit tps=6810.28, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=HDD, max_connection_threads=32

@codecov
Copy link

codecov bot commented Jun 6, 2021

Codecov Report

Merging #1609 (c5f1b67) into master (ac8fa84) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1609      +/-   ##
==========================================
- Coverage   80.74%   80.72%   -0.02%     
==========================================
  Files         751      751              
  Lines       52791    52798       +7     
==========================================
- Hits        42626    42623       -3     
- Misses      10165    10175      +10     
Impacted Files Coverage Δ
src/include/settings/settings_defs.h 100.00% <ø> (ø)
src/include/traffic_cop/traffic_cop.h 71.42% <0.00%> (-28.58%) ⬇️
src/settings/settings_callbacks.cpp 34.80% <0.00%> (-0.88%) ⬇️
src/network/network_io_wrapper.cpp 82.25% <0.00%> (-3.23%) ⬇️
src/optimizer/memo.cpp 82.35% <0.00%> (-2.95%) ⬇️
src/storage/arrow_serializer.cpp 86.25% <0.00%> (-0.63%) ⬇️
src/execution/sema/sema_expr.cpp 76.13% <0.00%> (-0.57%) ⬇️
src/storage/data_table.cpp 97.72% <0.00%> (-0.57%) ⬇️
src/include/storage/index/bplustree.h 91.83% <0.00%> (-0.54%) ⬇️
src/execution/sql/vector.cpp 86.09% <0.00%> (+0.33%) ⬆️
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ac8fa84...c5f1b67. Read the comment docs.

@lmwnshn lmwnshn added ready-to-merge This PR is ready to be merged. Mark PRs with this. and removed ready-for-review This PR passes all checks and is ready to be reviewed. Mark PRs with this. labels Jun 7, 2021
Copy link
Contributor

@lmwnshn lmwnshn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lmwnshn lmwnshn merged commit 39d598b into cmu-db:master Jun 7, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
ready-for-ci Indicate that this build should be run through CI. ready-to-merge This PR is ready to be merged. Mark PRs with this.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants