From 606cee8bcbc65dab3bf7d8eebe2d49cecf40b811 Mon Sep 17 00:00:00 2001 From: Aaron Black Date: Tue, 6 Feb 2024 11:18:21 -0800 Subject: [PATCH 1/3] add more docs on memory use --- doc/sphinx/06_umt/umt.rst | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/doc/sphinx/06_umt/umt.rst b/doc/sphinx/06_umt/umt.rst index 6a23cde8..45aee479 100644 --- a/doc/sphinx/06_umt/umt.rst +++ b/doc/sphinx/06_umt/umt.rst @@ -94,8 +94,6 @@ for UMT. Use '-B global' to specify that the size is for the global mesh, which is suitable for strong scaling studies. If performing a weak scaling study, you can specify '-B local' to specify the size of the mesh per rank instead. -Benchmark problems should target roughly half the node memory (for CPUs) or half the device memory (for GPUs). - For example, to create a global mesh of size 20,20,20 tiles: .. code-block:: @@ -104,6 +102,34 @@ For example, to create a global mesh of size 20,20,20 tiles: where num = 1 for SPP 1 or num = 2 for SPP 2. +Benchmark problems should target roughly half the node memory (for CPUs) or half the device memory (for GPUs). The problem size +(and therefore memory used) can be adjusted by increasing or decreasing the number of mesh tiles the problem runs on. + +When tuning the problem size, you can check the UMT memory usage in the output. For example, here is an example output from +benchmark #1 with a 10x10x10 tile mesh: + +.. code-block:: + + ================================================================= + Solving for 221184000 global unknowns. + (24000 spatial elements * 72 directions (angles) * 128 energy groups) + CPU memory needed (rank 0) for PSI: 1687.5MB + Current CPU memory use (rank 0): 2667.74MB + Iteration control: relative tolerance set to 1e-10. + ================================================================= + +When predicting memory usage, a rough ballpark estimate is: + +.. code-block:: + + global memory estimate = # global unknowns to solve * 8 bytes ( size of a double data type, typically 8 bytes ) * 175% + + # unknowns to solve = # spatial elements * # directions * # energy bins + +Each mesh tile has 192 3d corner spatial elements. Benchmark #1 has 72 directions and 128 energy bins. Benchmark #2 has 32 +directions and 16 energy bins. + + Example FOM Results =================== From a9466c4ec1e728e0457a1dc109475bb853a9a683 Mon Sep 17 00:00:00 2001 From: Jonah Maxwell Miller Date: Tue, 6 Feb 2024 14:18:22 -0700 Subject: [PATCH 2/3] update vibe --- doc/sphinx/03_vibe/vibe.rst | 24 ++++++++++++++++++++++-- parthenon | 2 +- 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/doc/sphinx/03_vibe/vibe.rst b/doc/sphinx/03_vibe/vibe.rst index 6bfcd96b..7cff2d55 100644 --- a/doc/sphinx/03_vibe/vibe.rst +++ b/doc/sphinx/03_vibe/vibe.rst @@ -204,9 +204,29 @@ Throughput performance of Parthenon-VIBE on a 40GB A100 is provided within the f Validation ========== - +Parthenon-VIBE prints to a history file (default name ``burgers.hst``) a +time series of the sum of squares of evolved variables integrated over +volume for each octant of the domain, as well as the total number of +meshblocks in the simulation at that time. To compare these quantities +between runs, we provide the ``burgers_diff.py`` program in the +benchmark folder. This will diff two history files and report when the +relative difference is greater than some tolerance. + +.. note:: + + ``burgers.hst`` is **appended** to when the executable is re-run. So + if you want to compare two different history files, rename the + history file by changing either ``problem_id`` in the ``parthenon/job`` + block in the input deck (this can be done on the command line. When + you start the program, add ``parthenon/job/problem_id=mynewname`` to + the command line argument), or copy the old file to back it up. + +To check that a modified calculation is still correct, run +``burgers_diff.py`` to compare a new run to the fiducial one at the +default tolerance. If no diffs are reported, the modified calculation +is correct. References ========== -.. [Parthenon-VIBE] Jonah Miller, 'Parthenon', 2023. [Online]. Available: https://github.com/parthenon-hpc-lab/parthenon. [Accessed: 20- Mar- 2023] +.. [Parthenon-VIBE] Jonah Miller, 'Parthenon', 2024. [Online]. Available: https://github.com/parthenon-hpc-lab/parthenon. [Accessed: 06- Feb- 2024] diff --git a/parthenon b/parthenon index 11c53d1c..f7ea1d01 160000 --- a/parthenon +++ b/parthenon @@ -1 +1 @@ -Subproject commit 11c53d1cd4ada0629e06d069b70b410234ed0bde +Subproject commit f7ea1d01fe7282a1e08f3978a73c9acf6654209e From ea88d5f4538fbe133980e96d5c88754bc6cce68d Mon Sep 17 00:00:00 2001 From: Aaron Black Date: Tue, 6 Feb 2024 13:45:48 -0800 Subject: [PATCH 3/3] update umt docs to requires conduit 0.9.0, as it was just released. --- doc/sphinx/06_umt/umt.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/sphinx/06_umt/umt.rst b/doc/sphinx/06_umt/umt.rst index 45aee479..ae48b82d 100644 --- a/doc/sphinx/06_umt/umt.rst +++ b/doc/sphinx/06_umt/umt.rst @@ -63,9 +63,9 @@ UMT can be found on github and cloned via: Build Requirements ------------------ -* C/C++ compiler(s) with support for C++11 and Fortran compiler(s) with support for F2003. -* `CMake 3.18X `_ -* `Conduit v0.8.9 (pending), or the develop branch as of 1/1/2024. `_ +* C/C++ compiler(s) with support for C++14 and Fortran compiler(s) with support for F2003. +* `CMake 3.21X `_ +* `Conduit v0.9.0 `_ * `Spack `_ (optional) * MPI 3.0+