-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
abstract common base of SQL micro-benchmarks to reduce boilerplate and standardize parameters #17383
abstract common base of SQL micro-benchmarks to reduce boilerplate and standardize parameters #17383
Conversation
…d standardize parameters changes: * adds `SqlBenchmarkDatasets` which contains commonly used benchmark data generator schemas * adds `SqlBaseBenchmark` which contains common benchmark segment generation methods for any benchmark using `SqlBenchmarkDatasets` * adds `SqlBaseQueryBenchmark` and `SqlBasePlanBenchmark` for benchmarks measuring queries and planning respectively * migrate all existing SQL jmh benchmarks to extend `SqlBaseQueryBenchmark`, quite dramatically reducing the boilerplate needed to create benchmarks, and allowing the use of multiple datasources within a benchmark file * adjustments to data generator stuff to allow passing in an ObjectMapper so that the same mapper can be used for both benchmark queries and segment generation, avoiding the need to register stuff with both mappers for benchmarks * adds `SqlProjectionsBenchmark` and `SqlComplexMetricsColumnsBenchmark` for measuring projections and measuring complex metric compression respectively
benchmarks/src/test/java/org/apache/druid/benchmark/query/SqlBenchmark.java
Dismissed
Show dismissed
Hide dismissed
...hmarks/src/test/java/org/apache/druid/benchmark/query/SqlComplexMetricsColumnsBenchmark.java
Dismissed
Show dismissed
Hide dismissed
benchmarks/src/test/java/org/apache/druid/benchmark/query/SqlExpressionBenchmark.java
Dismissed
Show dismissed
Hide dismissed
benchmarks/src/test/java/org/apache/druid/benchmark/query/SqlNestedDataBenchmark.java
Dismissed
Show dismissed
Hide dismissed
benchmarks/src/test/java/org/apache/druid/benchmark/query/SqlPlanBenchmark.java
Dismissed
Show dismissed
Hide dismissed
benchmarks/src/test/java/org/apache/druid/benchmark/query/SqlProjectionsBenchmark.java
Dismissed
Show dismissed
Hide dismissed
benchmarks/src/test/java/org/apache/druid/benchmark/query/SqlWindowFunctionsBenchmark.java
Dismissed
Show dismissed
Hide dismissed
failures are related to code coverage for changes that primarily are only used in tests or benchmarks. I should move this generator stuff into its own extension someday, but for now i think we can ignore it |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, ok to ignore the coverage failures since this is benchmarking code.
changes:
SqlBenchmarkDatasets
which contains commonly used benchmark data generator schemasSqlBaseBenchmark
which contains common benchmark segment generation methods for any benchmark usingSqlBenchmarkDatasets
SqlBaseQueryBenchmark
andSqlBasePlanBenchmark
for benchmarks measuring queries and planning respectivelySqlBaseQueryBenchmark
, quite dramatically reducing the boilerplate needed to create benchmarks, and allowing the use of multiple datasources within a benchmark fileSqlProjectionsBenchmark
andSqlComplexMetricsColumnsBenchmark
for measuring projections and measuring complex metric compression respectivelyCommon options are:
schemaType
- "explicit" or "auto", to test differences between columns created with explicit dimension schemas vsAutoTypeColumnSchema
that is used by schema discovery (and numbers have indexes and such)storageType
- "MMAP", "INCREMENTAL", "FRAME_COLUMNAR", "FRAME_ROW" for testing various backing "segment" typesstringEncoding
- "UTF8", "FRONT_CODED_DEFAULT_V1", "FRONT_CODED_16_V1", for testing different string encoding strategies (only applies to "MMAP" storageType)complexMetricCompression
- "none", "lz4" for testing different complex metric compression inIndexSpec
(only applies to "MMAP" storageType)Most query benchmarks also have a numbered
query
parameter, the exception beingSqlGroupByBenchmark
which instead has agroupingDimension
parameter.Example: