Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial commit: Jupyter notebook to help estimate network settings #4452

Merged
merged 1 commit into from
Sep 10, 2024

Conversation

MonsieurNicolas
Copy link
Contributor

@MonsieurNicolas MonsieurNicolas commented Sep 4, 2024

This PR adds a notebook with rudimentary support for estimating disk iops/bandwidth and network utilization.

This is just a starting point, and is probably quite off (impact of bucketlist merges, eviction and archival is not included), but should still allow to reason at the right order of magnitude when making changes.

Over time, this can be extended as to better help estimate limits (both tx level and ledger level) used in Soroban and elsewhere, especially limits that should "never be reduced" in the future.

This version outputs the following:

ledger_LE_reads:  50000
ledger_bytes_read:  28775000
ledger_LE_writes:  25000
ledger_bytes_write:  3175000
ledger_classic_LE_reads:  500000
ledger_classic_bytes_read:  291000000
ledger_classic_LE_writes:  250000
ledger_classic_bytes_write:  35000000
ledger_soroban_LE_reads:  800000
ledger_soroban_bytes_read:  1474560000.0
ledger_soroban_LE_writes:  250000
ledger_soroban_bytes_write:  665600000.0

...

flooding bytes (per connection):  147500000.0
TxSet bytes:  59000000
Network bps (GBit/s):  3.1663477420806885

...

max iops:  1963575.4285714286
max read Mbps:  2463.800726209368
max write Mbps:  2684.6885681152344
max disk bandwidth Mbps:  2703.9018592834473

which is in line with our expectation that NVMe drives will probably be needed as tps increases

@MonsieurNicolas MonsieurNicolas force-pushed the jupyterCalc branch 3 times, most recently from addc616 to 574a6ae Compare September 6, 2024 22:12
Copy link
Contributor

@anupsdf anupsdf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Tested this running in Jupyter lab online tool.

@anupsdf anupsdf added this pull request to the merge queue Sep 9, 2024
Merged via the queue into stellar:master with commit dacd02c Sep 10, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants