perf: optimize RecordBatch to HttpOutput conversion #4178
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Ruihang Xia <[email protected]>
Signed-off-by: Ruihang Xia <[email protected]>
Signed-off-by: Ruihang Xia <[email protected]>
Signed-off-by: Ruihang Xia <[email protected]>
Signed-off-by: Ruihang Xia <[email protected]>
waynexia
added
C-performance
Signed-off-by: Ruihang Xia <[email protected]>
Signed-off-by: Ruihang Xia <[email protected]>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4178 +/- ##
==========================================
- Coverage 85.12% 84.79% -0.33%
==========================================
Files 1020 1022 +2
Lines 179635 179887 +252
==========================================
- Hits 152920 152543 -377
- Misses 26715 27344 +629 |
Signed-off-by: Ruihang Xia <[email protected]>
waynexia
force-pushed
the
optimize-http-output
branch
from
e19faad to
4923089
Compare
evenyag
approved these changes
Jun 20, 2024
zyy17
pushed a commit
to zyy17/greptimedb
that referenced
this pull request
Jun 22, 2024
* add benchmark Signed-off-by: Ruihang Xia <[email protected]> * save 70ms Signed-off-by: Ruihang Xia <[email protected]> * add profiler Signed-off-by: Ruihang Xia <[email protected]> * save 50ms Signed-off-by: Ruihang Xia <[email protected]> * save 160ms Signed-off-by: Ruihang Xia <[email protected]> * format toml file Signed-off-by: Ruihang Xia <[email protected]> * fix license header Signed-off-by: Ruihang Xia <[email protected]> * fix windows build Signed-off-by: Ruihang Xia <[email protected]> --------- Signed-off-by: Ruihang Xia <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
Optimize
HttpRecordsOutput::try_new()method. Reduce the consumption of converting 409600 rows from ~400ms to ~120ms. Saves ~70% CPU.This patch also adds a microbenchmark for that method.
Key optimizations are:
Value::StringfromBytestoStringClonesNow the entire conversion method only has one
clonethat takes data from the data vector, and onewritethat writes the serialized content to the result vector.Checklist