-
Notifications
You must be signed in to change notification settings - Fork 285
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(core): use jemalloc as the memory allocator #3541
Conversation
Thanks for submitting this pull request! A maintainer will review this in the next few days and explicitly select labels so you know what's going on. If no reviewer appears after a week, a reminder will be sent out. |
A collaborator has approved this PR. A maintainer will merge this PR shortly. If it shouldn't be merged yet, please leave a comment saying so and we'll wait. Thank you for your contribution! |
A collaborator has approved this PR. A maintainer will merge this PR shortly. If it shouldn't be merged yet, please leave a comment saying so and we'll wait. Thank you for your contribution! |
Your pull request has been merged and marked as tier 1. It will earn you $200 USD. |
Summary
This PR replaces the native memory allocator with jemalloc in Core to reduce memory fragmentation and results in a smaller memory footprint as confirmed by multiple Core nodes using the native memory allocator (glibc) vs jemalloc, where in all cases the jemalloc nodes consistently used less RAM. Furthermore, results from a sync from 0 test (using the same hardware with 2GB physical RAM to eliminate variability due to different specifications) demonstrate that jemalloc is more performant and uses far less memory by the end:
native memory allocator (glibc): Completed in 14h 54m 14s using 1.3GB RAM 30 minutes after completion.
jemalloc: Completed in 13h 37m 56s using 314.5MB RAM 30 minutes after completion.
Therefore a sync from 0 using jemalloc completed 1h 16m 18s faster than glibc on the same hardware and, upon finishing, was using ~1017MB less RAM.
On a different (much higher powered) machine with 32GB RAM, I rolled back 300,000 blocks on devnet and re-synced using glibc and jemalloc:
native memory allocator (glibc): Completed in 7 minutes and 20 seconds using 3.5GB RAM (!) 10 minutes after completion.
jemalloc: Completed in 6 minutes and 50 seconds using 410.9MB RAM 10 minutes after completion.
In that instance, jemalloc completed 30 seconds faster than glibc and used a staggering ~3173MB less RAM.
This PR does not change the default system memory allocator to jemalloc as this may not be desirable and we should not make such system-wide changes. It just makes the Core process use jemalloc and leaves everything else untouched.
Special update instructions:
Users should manually run
sudo apt-get install libjemalloc-dev
(Debian derivatives) orsudo yum install libjemalloc-devel
(RedHat derivatives) before updating since they don't re-run the Core installer (which will automatically install it for new installations).Update withark update --no-restart
thenpm2 delete all
and restart therelay
/forger
/core
process(es) viaark {relay|forger|core}:start
. Merely executingark {relay|forger|core}:restart
or letting the updater restart the processes is insufficient as it will not inject theLD_PRELOAD
variable, presumably due to pm2 caching.Now refactored so the normal
ark update
procedure is sufficient to restart Core as long as libjemalloc has been installed as above.To confirm Core is now using jemalloc, run
cat /proc/`pgrep -f "@arkecosystem/core/bin/run"`/smaps | grep jemalloc
. The output should show references tolibjemalloc.so
to confirm the process is running with jemalloc.Checklist