Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the performance of the shielded sync ledger client (backport #4016) #4017

Merged
merged 7 commits into from
Nov 12, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Nov 12, 2024

Describe your changes

The shielded sync's ledger client continues to perform poorly, even after #3578. This PR introduces some optimizations to the ledger client, and improves its UX a bit.

  • Fetches are now logged more frequently on the progress bar. Previously, they weren't being logged because concurrent fetches were starving each other on the first semaphore. To fix this, fetch_shielded_transfers now has a single semaphore point, at the top of the method, rather than two.
  • Errors returned from RPC servers are no longer printed. They weren't that helpful anyway, and were spamming the screen, obfuscating the output of the progress bars.
  • A linear backoff sleep has been added to the ledger client, on errors, to allow the client to recover from potential rate limiting/throttling imposed by the CometBFT RPC server. On successful fetches, the backoff is reduced, returning to the normal/optimal fetch speed.
  • The block batch size parameter has been exposed to the CLI and SDK, allowing users to specify the number of blocks that are fetched per concurrent fetch job. This may help performance, in some cases.

In general, the indexer masp client still performs about 10x better than the ledger masp client, while fetching shielded txs.

Checklist before merging

  • If this PR has some consensus breaking changes, I added the corresponding breaking:: labels
    • This will require 2 reviewers to approve the changes
  • If this PR requires changes to the docs or specs, a corresponding PR is opened in the namada-docs repo
    • Relevant PR if applies:
  • If this PR affects services such as namada-indexer or namada-masp-indexer, a corresponding PR is opened in that repo
    • Relevant PR if applies:

This is an automatic backport of pull request #4016 done by [Mergify](https://mergify.com).

sug0 added 7 commits November 12, 2024 17:10
(cherry picked from commit 18848f8)
(cherry picked from commit 2f8c615)
(cherry picked from commit 371db79)
(cherry picked from commit b2332bf)
@tzemanovic tzemanovic added the merge Ready to merge - mergifyio bot will add the PR to merge queue when all checks pass label Nov 12, 2024
@mergify mergify bot merged commit 430a97c into maint-0.45 Nov 12, 2024
14 of 15 checks passed
@mergify mergify bot deleted the mergify/bp/maint-0.45/pr-4016 branch November 12, 2024 17:18
Copy link

codecov bot commented Nov 12, 2024

Codecov Report

Attention: Patch coverage is 66.12903% with 21 lines in your changes missing coverage. Please review.

Project coverage is 73.89%. Comparing base (8b486b4) to head (aec0664).
Report is 8 commits behind head on maint-0.45.

Files with missing lines Patch % Lines
crates/core/src/control_flow/time.rs 38.09% 13 Missing ⚠️
crates/sdk/src/masp/utilities.rs 80.48% 8 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff               @@
##           maint-0.45    #4017      +/-   ##
==============================================
- Coverage       73.90%   73.89%   -0.01%     
==============================================
  Files             341      341              
  Lines          106451   106494      +43     
==============================================
+ Hits            78673    78698      +25     
- Misses          27778    27796      +18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge Ready to merge - mergifyio bot will add the PR to merge queue when all checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants