Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintain sanity (or madness) #165

Merged
merged 28 commits into from
May 4, 2023
Merged

Maintain sanity (or madness) #165

merged 28 commits into from
May 4, 2023

Conversation

brryan
Copy link
Collaborator

@brryan brryan commented Apr 19, 2023

PR Summary

Long-duration simulations of Fishbone torii tend to accumulate large amounts of magnetic flux on the black hole (MAD state). While this may suggest that MADs are ubiquitous in nature, it would be nice to be able to simulate accretion onto weakly magnetized black holes (SANEs) for long times. This PR adds a runtime option for setting a specific phi = Phi / sqrt(Mdot) amd some timescale on which the code should maintain that value. Phi is maintained by adding a (hopefully small) amount of divergence-free nearly vertical field lines to the black hole every timestep.

This PR also fixes a bug in the history flux evaluators, where the block index wasn't being passed in to the geometry calls. I think this should have been caught in the geometry infrastructure (it is caught with analytic but not cached geometry) but that is a separate issue (#171). The geometry worked correctly for single-meshblock-per-MPI-rank calculations.

Here is an example 256^2 2D torus run with no sanity maintenance (torus.hst) and the sanitized version with a target phi = 1 (indicated by the dashed line in the phi plot) and a characteristic time for changing the net field tau = 500M:

image

The early hiccup in phi with the sanitized run is probably due to mdot being all over the place there. I also included a timer so that the phi changing code can be disabled for some initial coordinate time.

Restarts also look fine (here, a bit after 500M):
image

PR Checklist

  • Adds a test for any bugs fixed. Adds tests for new features.
  • Format your changes by calling scripts/bash/format.sh.
  • Explain what you did.
  • Use the Parthenon task-based reduction machinery.
  • Do initialization only once.
  • Ensure restarts work with flux enforcement logic.

auto &sc = pmb->meshblock_data.Get(stage_name[stage]);

auto p2c = tl.AddTask(none, fluid::PrimitiveToConserved, sc.get());
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new set of regions is good motivation for getting the rest of the hydro to be per-meshdata I think

@brryan brryan requested review from AstroBarker and bprather April 19, 2023 20:39
Copy link
Collaborator

@Yurlungur Yurlungur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Some comments below. Nothing really blocking, but I think we shouldn't demand that num_partitions=1 across the whole code if we can help it, as that limits what we can do with load balancing (i.e., with monte carlo) and with AMR (when we get there)

src/fixup/fixup.cpp Outdated Show resolved Hide resolved
src/fixup/fixup.cpp Outdated Show resolved Hide resolved
src/fixup/fixup.cpp Outdated Show resolved Hide resolved
src/fixup/fixup.cpp Outdated Show resolved Hide resolved
src/fixup/fixup.cpp Outdated Show resolved Hide resolved
src/phoebus_driver.cpp Show resolved Hide resolved
src/phoebus_driver.cpp Outdated Show resolved Hide resolved
Copy link
Collaborator

@AstroBarker AstroBarker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, agree with @Yurlungur's comments. It would also be good to start documenting runtime options, but that's for another day..

@brryan brryan changed the title Maintain sanity (or madness) Draft: Maintain sanity (or madness) Apr 20, 2023
@brryan brryan changed the title Draft: Maintain sanity (or madness) Maintain sanity (or madness) Apr 28, 2023
@@ -32,10 +32,17 @@ Real inline Min(const Real &x) {
return xmin;
}

Real inline Sum(const Real &x) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not actually used now but I think we can leave this in in case the need arises in a problem generator file at some point

@brryan
Copy link
Collaborator Author

brryan commented May 2, 2023

OK this is ready for re-review @Yurlungur. The function used to paint on B field lines is still unnecessarily re-evaluated every so often but this cost should always be small so I didn't bother to rewrite everything to work with restarts for this pretty specific feature. If you think this is too ugly or problematic let me know and I can revisit. Otherwise this feature seems to do what it's supposed to.

@brryan brryan mentioned this pull request May 2, 2023
5 tasks
scripts/python/phoedf.py Show resolved Hide resolved
src/phoebus_driver.cpp Outdated Show resolved Hide resolved
@Yurlungur Yurlungur merged commit ca15cb2 into main May 4, 2023
@Yurlungur Yurlungur deleted the brryan/maintain_sanity branch May 4, 2023 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants