Skip to content
This repository has been archived by the owner on May 20, 2024. It is now read-only.

enable loop parallelism in NL_Function_Eval #13

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ian-bertolacci
Copy link

@ian-bertolacci ian-bertolacci commented Jun 19, 2019

In progress to #11

Current progress:

  • Split the scatter section of the loop out into a new loop
  • Store temporary variables into a vector
  • Reverse scatter pattern into a gather pattern
  • Verify output
  • Check performance changes (in serial)
  • Attempt parallelization

Loop calculating contributions from 2nd order
derivatives and gravity had a scatter pattern that
prevented parallelism.
This is changed by:
1. storing the temporary values (u_right, u_front, u_upper)
2. Splitting scatter portion into a second loop below
3. reversing the accesses to perform a gather instead of a scatter
@ian-bertolacci
Copy link
Author

ian-bertolacci commented Jun 21, 2019

I can assert that in the regular LW test case (72hr sim with outputs every 3 simulated hrs) that the difference between the baseline case (from master at commit 1de045b) compared to this branch (commit f2f7d68) in the output pressures is less than 6.5e-11 for all output time-steps, and in the output saturation is less than 1.6e-14 for all output time-steps.

@ian-bertolacci
Copy link
Author

ian-bertolacci commented Jun 21, 2019

Whole program run times (in seconds ± 1 standard deviation) for baseline and 511_split_loop in the regular LW test case (72hr sim with outputs every 3 simulated hrs)

Time Class baseline (master 1de045b) 511_split_loop (f2f7d68)
real 332.3808 ± 1.7961 334.5446 ± 1.3524
user 331.4917 ± 1.7961 333.6424 ± 1.3506
sys 0.7596 ± 0.0166 0.7657 ± 0.0225

Easy to see the 2.1638 second slowdown in the split-loop case.
This is most likely from the additional memory usage and the loss of caching from performing the reduction using the created data in the second loop instead of immediately when that data is created.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant