chore: add more metrics about parquet and cache #4410
Conversation
MichaelScofield
requested review from
evenyag,
v0y4g3r and
waynexia
as code owners
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThe recent changes improve the caching system's monitoring capabilities by adding metrics for cache evictions and performance tracking during Parquet metadata loading. The Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- src/mito2/src/cache.rs (2 hunks)
- src/mito2/src/metrics.rs (2 hunks)
- src/mito2/src/sst/parquet/metadata.rs (2 hunks)
Additional comments not posted (8)
src/mito2/src/sst/parquet/metadata.rs (2)
71-72: LGTM!The timer for the
PARQUET_METADATA_LOAD_ELAPSEDmetric is correctly started to measure the duration of the metadata loading process.
22-22: LGTM! Verify the metric initialization.The
PARQUET_METADATA_LOAD_ELAPSEDmetric is correctly introduced.Ensure that the metric is properly initialized in the
metricsmodule.Verification successful
Metric initialization verified.
The
PARQUET_METADATA_LOAD_ELAPSEDmetric is properly initialized in themetricsmodule and used correctly in the code.
- Initialization found in
src/mito2/src/metrics.rs:pub static ref PARQUET_METADATA_LOAD_ELAPSED: Histogram = register_histogram!( "greptime_parquet_metadata_load_elapsed", "parquet metadata load elapsed", ).unwrap();Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that the `PARQUET_METADATA_LOAD_ELAPSED` metric is properly initialized in the metrics module. # Test: Search for the metric initialization. Expect: Initialization code for the metric. rg --type rust 'PARQUET_METADATA_LOAD_ELAPSED' -A 3Length of output: 987
src/mito2/src/cache.rs (4)
212-219: LGTM!The
eviction_listenerforsst_meta_cachecorrectly logs eviction reasons using theCACHE_EVICTIONmetric.
227-234: LGTM!The
eviction_listenerforvector_cachecorrectly logs eviction reasons using theCACHE_EVICTIONmetric.
242-247: LGTM!The
eviction_listenerforpage_cachecorrectly logs eviction reasons using theCACHE_EVICTIONmetric.
37-37: LGTM! Verify the metric initialization.The
CACHE_EVICTIONmetric is correctly introduced.Ensure that the metric is properly initialized in the
metricsmodule.Verification successful
The
CACHE_EVICTIONmetric is properly initialized and used.
Initialization found in
src/mito2/src/metrics.rs:pub static ref CACHE_EVICTION: IntGaugeVec = register_int_gauge_vec!( "greptime_mito_cache_eviction", "mito cache eviction", &[TYPE_LABEL, "cause"] );Usage found in
src/mito2/src/cache.rs.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that the `CACHE_EVICTION` metric is properly initialized in the metrics module. # Test: Search for the metric initialization. Expect: Initialization code for the metric. rg --type rust 'CACHE_EVICTION' -A 3Length of output: 1341
src/mito2/src/metrics.rs (2)
193-198: LGTM!The
CACHE_EVICTIONmetric is correctly introduced and initialized to track cache evictions.
324-328: LGTM!The
PARQUET_METADATA_LOAD_ELAPSEDmetric is correctly introduced and initialized to measure the elapsed time for loading Parquet metadata.
MichaelScofield
force-pushed
the
chore/more-metrics
branch
from
6b8e099 to
c260174
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4410 +/- ##
=======================================
Coverage 85.02% 85.02%
=======================================
Files 1075 1075
Lines 192542 192567 +25
=======================================
+ Hits 163709 163739 +30
+ Misses 28833 28828 -5 |
|
@v0y4g3r PTAL |
|
@evenyag PTAL |
|
@evenyag PTAL |
MichaelScofield
force-pushed
the
chore/more-metrics
branch
from
d4d086d to
05b2664
Compare
MichaelScofield
force-pushed
the
chore/more-metrics
branch
from
70522a7 to
85d6617
Compare
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
as title
useful for diagnosing slow queries
Checklist
Summary by CodeRabbit
New Features
Improvements