Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dynamic available memory calculation for all locales #2533

Merged
merged 9 commits into from
Jun 29, 2023
Merged

dynamic available memory calculation for all locales #2533

merged 9 commits into from
Jun 29, 2023

Conversation

hokiegeek2
Copy link
Contributor

Closes #2477

This PR delivers the following dynamic Arkouda memory tracking, reporting, and mgmt features:

  1. on each locale MemoryMgmt checks to see if the projected memory required for an incoming cmd can be accommodated by the memory allocated to the Arkouda chapel process on every node. If so, the command is allowed to proceed.
  2. if the answer to (1) is false, on each locale MemoryMgmt checks to see if additional memory can be allocated to accommodate the new, projected total Arkouda memory on every node. If so, the command is allowed to proceed. If not, a server-side error is thrown and the locales that cannot support the new malloc request are logged.
  3. message is sent back to the client that OOM occurred on 1..n locales
  4. new CLI arguments to enable dynamic memory mgmt. The default is the existing, static memory tracking, reporting, and mgmt Arkouda feature

@hokiegeek2
changed to iomode enum for Chapel 1.29.0 Bears-R-Us#2477
618b22c
@hokiegeek2
Merge branch '2477_dynamic_available_memory_calculation_for_all_local…
5148689 
…es' of github.com:hokiegeek2/arkouda into 2477_dynamic_available_memory_calculation_for_all_locales
Ethan-DeBandi99
Ethan-DeBandi99 approved these changes Jun 29, 2023
View reviewed changes
Copy link
Contributor

@Ethan-DeBandi99 Ethan-DeBandi99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. 1 thing to look at, but I don't think it will impact functionality.

src/MemoryMgmt.chpl Outdated Show resolved Hide resolved
stress-tess
stress-tess reviewed Jun 29, 2023
View reviewed changes
Copy link
Member

@stress-tess stress-tess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hokiegeek2 got some quick questions but i think this looks good

src/MemoryMgmt.chpl Outdated Show resolved Hide resolved
src/MemoryMgmt.chpl Show resolved Hide resolved
src/MemoryMgmt.chpl Show resolved Hide resolved
src/MemoryMgmt.chpl Outdated Show resolved Hide resolved
src/ServerConfig.chpl Show resolved Hide resolved
stress-tess
stress-tess approved these changes Jun 29, 2023
View reviewed changes
Copy link
Member

@stress-tess stress-tess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@stress-tess stress-tess added this pull request to the merge queue Jun 29, 2023
Merged via the queue into Bears-R-Us:master with commit 38a23a3 Jun 29, 2023
@hokiegeek2 hokiegeek2 deleted the 2477_dynamic_available_memory_calculation_for_all_locales branch March 27, 2024 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Ethan-DeBandi99 Ethan-DeBandi99 Ethan-DeBandi99 approved these changes

@stress-tess stress-tess stress-tess approved these changes

@jaketrookman jaketrookman jaketrookman approved these changes

@TheOddCzar789 TheOddCzar789 Awaiting requested review from TheOddCzar789

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Dynamic available memory calculation for all locales
4 participants
@hokiegeek2 @Ethan-DeBandi99 @stress-tess @jaketrookman