Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RandomX VM cannot be created with less than 2GB memory #3104

Closed
delta1 opened this issue Jul 18, 2021 · 1 comment · Fixed by #3301
Closed

RandomX VM cannot be created with less than 2GB memory #3104

delta1 opened this issue Jul 18, 2021 · 1 comment · Fixed by #3301

Comments

@delta1
Copy link
Contributor

delta1 commented Jul 18, 2021

Describe the bug
Running the base node with less than 2080MB memory results in this error, a failure to create the randomx dataset. Even with randomx default flags, which should be able to create the VM in only 256MB memory.

#3103

Investigation into tari-project/randomx-rs and tevador/randomx are required to establish why the default flags still attempt to create the dataset and VM with full memory requirements of randomx

2021-07-18 08:42:12.426544710 [c::pow::randomx_factory] WARN  Error initializing randomx dataset with flags FLAG_HARD_AES | FLAG_JIT | FLAG_ARGON2_SSSE3 | FLAG_ARGON2_AVX2 | FLAG_ARGON2. Problem creating the RandomX object:Could not allocate dataset. Fallback to default flags
2021-07-18 08:42:12.436775657 [c::bn::header_sync] WARN  Block header validation failed: Error in merge mine data:RandomX error: Problem creating the RandomX object:Could not allocate dataset

To Reproduce

  1. Run weatherwax base node in a VM with less than 2GB memory
  2. Wait for headers to start syncing
  3. Observe error to create randomx dataset

Expected behavior

RandomX VM should be started in light mode and be able to verify headers with only 256MB memory

Desktop (please complete the following information):

  • OS & Version: [e.g. iOS 10.2.1]
    ubuntu 20.04

Additional context

@StriderDM
Copy link
Contributor

Fixed here:
#3301

aviator-app bot pushed a commit to sdbondi/tari that referenced this issue Sep 6, 2021
Description
Removed dataset from RandomXVMInstance, light mode only requires a cache, this fixes a bug where light mode will consume more than 256MB of memory per RandomXVMInstance.

Made RandomXDataset optional for RandomXVMInstance.

resolves tari-project#3104 
closes tari-project#3103 

Motivation and Context
Overall memory overhead reduction

How Has This Been Tested?
Manually in a constrained vagrant instance

<img width="1048" alt="Screen Shot 2021-09-04 at 12 45 40 AM" src="https://user-images.githubusercontent.com/51991544/132176934-6a5a3753-4960-472c-8544-edebd6eec40d.png">
Cifko pushed a commit to Cifko/tari that referenced this issue Sep 10, 2021
Description
Removed dataset from RandomXVMInstance, light mode only requires a cache, this fixes a bug where light mode will consume more than 256MB of memory per RandomXVMInstance.

Made RandomXDataset optional for RandomXVMInstance.

resolves tari-project#3104 
closes tari-project#3103 

Motivation and Context
Overall memory overhead reduction

How Has This Been Tested?
Manually in a constrained vagrant instance

<img width="1048" alt="Screen Shot 2021-09-04 at 12 45 40 AM" src="https://user-images.githubusercontent.com/51991544/132176934-6a5a3753-4960-472c-8544-edebd6eec40d.png">
@delta1 delta1 closed this as completed Sep 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants