[improvement](serde) Optimizing the performance of mysql result writer #20928
Conversation
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
| int buf_ret = 0; | ||
| for (ssize_t i = start; i < end; ++i) { | ||
| if (0 != buf_ret) { | ||
| if constexpr (is_binary_format) { |
There was a problem hiding this comment.
is_binary_format can not present return_object_data_as_binary , just because we have a switch return_object_data_as_binary, so there is a situation with return_object_data_as_binary=true , but is_binary_format is false.
There was a problem hiding this comment.
Yes, thanks for your comment.
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
|
LGTM |
Proposed changes
When converting query results into MySQL format, it involves transforming columnar data storage into row-based storage. This process raises the question of choosing between sequential reading and sequential writing. In reality, sequential writing is the better choice for performance optimization.
Test with 9M rows contains more than 20 columns, this patch can reduce the conversion time from 20s to 11s.
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...