Skip to content

Dynamic_Topology

Jump to bottom
Greg Sjaardema edited this page Nov 8, 2024 · 16 revisions

Motivation

Proof of Concept

Current "proof of concept" is to use the NetCDF hierarchical groups functionality. Instead of storing each topology change in a separate file, we instead create a new "group" each time the topology changes. In essence, each previous independent file is now a group within a single file.

Exodus API

New ex_inquire() flags:

  EX_INQ_NUM_CHILD_GROUPS    = 52, /**< inquire number of groups contained in this (exoid) group */
  EX_INQ_GROUP_PARENT        = 53, /**< inquire id of parent of this (exoid) group; returns exoid if at root */
  EX_INQ_GROUP_ROOT          = 54, /**< inquire id of root group "/" of this (exoid) group; returns exoid if at root */
  EX_INQ_GROUP_NAME_LEN      = 55, /**< inquire length of name of group exoid */
  EX_INQ_GROUP_NAME          = 56, /**< inquire name of group exoid. "/" returned for root group (char* GROUP_NAME_LEN+1 size) */
  EX_INQ_FULL_GROUP_NAME_LEN = 57, /**< inquire length of full path name of this (exoid) group */
  EX_INQ_FULL_GROUP_NAME     = 58, /**< inquire full "/"-separated path name of this (exoid) group */

New API functions:

EXODUS_EXPORT int ex_create_group(int parent_id, const char *group_name);
EXODUS_EXPORT int ex_get_group_id(int parent_id, const char *group_name, int *group_id);
EXODUS_EXPORT int ex_get_group_ids(int parent_id, int *num_groups, int *group_ids);

New error codes:

#define EX_NOTROOTID -1002    /**< file id is not the root id; it is a subgroup id */

New defines:

/* Used to map between root (file id) and group ids when using groups */
#define EX_FILE_ID_MASK (0xffff0000) /**< Must match FILE_ID_MASK in NetCDF nc4internal.h */
#define EX_GRP_ID_MASK  (0x0000ffff) /**< Must match GRP_ID_MASK in NetCDF nc4internal.h */

Given an exodus file id exoid which may refer to a group within the file, the rootid file id for the file is:

int rootid = exoid & EX_FILE_ID_MASK;
  • NOTE: This is the same mask that is used in netCDF and will give a maximum of 32,378 possible groups within a single netCDF file (and 32,378 simultaneously open files). This should not be an issue, but if it is, we could probably propose an enhancement to make the mask a dynamic property of a file instead of static in the library.

Exodus Python API:

  • No support for groups yet.

IOSS Functionality:

If a database exists and is created with the property APPEND_OUTPUT set to Ioss::DB_APPEND_GROUP, then it will not be overwritten and the client can add groups to the database. You should also add the following property:

propertyManager.add(Ioss::Property("ENABLE_FILE_GROUPS", 1));

prior to the output database creation call which will tell Exodus/netCDF to create a netCDF-4 database in "non-classic" mode which is needed for the group support.

Two api functions have been added at the Ioss::DatabaseIO level.

The first function will open a group if the database type supports groups and the database contains groups:

  • If the group_name begins with /, it specifies the absolute path name from the root with / separating groups.
  • Otherwise, the group_name specifies a child group of the currently active group.
  • If group_name is / then the root group is opened.
  • The function returns True if the group was successfully opened.
  • All subsequent operations on that file (reads/writes) will apply to that group within the file.
    bool open_group(const std::string &group_name)

The second function will create a group if the database type supports groups.

  • The group named group_name will be created as a child of the current group.
  • The name of the group must not contain a '/' character.
  • If the command is successful, then the group will be the active group for all subsequent writes/reads to/from the database.
  • Returns True if successful.
    bool create_subgroup(const std::string &group_name)
  • NOTE: Additional functions are needed (some implemented in io_info and io_shell already -- move to general API)
    • Should probably add a function to query whether a database supports groups
    • Get number of groups in file.
    • Get names of groups which are children of current group -- can then iterate and build "group tree"
    • print_groups() is implemented in io_info

Example:

This is an example of how io_shell creates a group in a file and puts data into that group:

        // If there are multiple input files, io_shell puts each into its own group
        if (interFace.inputFile.size() > 1) {
           properties.add(Ioss::Property("APPEND_OUTPUT", Ioss::DB_APPEND_GROUP));
          if (!first) {
            // Putting each file into its own output group...
            // The name of the group will be the basename portion of the filename...
            Ioss::FileInfo file(inpfile);
            dbo->create_subgroup(file.tailname());
          }
          else {
            first = false;
          }
        }

        // Do normal copy...  IOSS doesn't know whether outputting to a group or a "normal" file at this point:
        Ioss::copy_database(region, output_region, options);

Here is a minimal example of opening a database with groups and selecting the user-specified group group_name in the file:

      Ioss::DatabaseIO *dbi1 =
          Ioss::IOFactory::create(interFace.inFiletype, inpfile, Ioss::READ_MODEL,
                                  Ioss::ParallelUtils::comm_world(), properties);
      if (dbi1 == nullptr || !dbi1->ok(true)) {
        std::exit(EXIT_FAILURE);
      }

      if (!group_name.empty()) {
        bool success = dbi1->open_group(group_name);
        // Assume success
      }
      Ioss::Region input_region1(dbi1, "region_1");

The only difference between a "normal" read and reading a group is the dbi1->open_group(group_name) call.

Experimenting with groups using io_shell and io_info:

  • io_shell can split a database into multiple databases each with a specified number # of timesteps using the option -split_times <#> (Not really part of the group modifications, but useful for generating a set of related files)
  • io_shell can take each of these individual files and join them into a single file with each file in its own group using: io_shell <list_of_input_files> <output_file>
    • NOTE: the output database is currently closed and reopened between the addition of each group. This is not optimal or required and io_shell should be refactored to fix this.
    • Note that the files to be joined don't need to be related in any way, but for use in the dynamic topology examples, it is better for them to be similar models.
  • io_info can give the name of all groups in the file: io_info --list_groups <group_file>
  • io_info can give information about a specific group in the file: io_info --group_name <name_of_group> <group_file>
  • io_shell can extract a specific group to a file: io_shell --extract_group <name_of_group> <group_file> <output_file>
    • NOTE: Would be good to enhance this with globbing of names or an ALL option to extract all or multiple groups

See Also:

Efficient Storage of Adaptive Topological Meshes

Adding support for change_sets (groups) to SEACAS applications:

  • io_info
    • change groups to change_sets
    • use new API in IOSS instead of going down to exodus level
    • show which change_set the info is being show for and number of change_sets in file
    • show info for multiple change sets
  • io_shell
    • change groups to change_sets
    • If copying a file, output should have same change_sets as input
    • add option to extract all change_sets or list of change_sets to output file (Use %G in filename to substitute change_set name)
    • Use IOSS API instead of going down to exodus level
    • --compare option should compare all change_sets in the file by default
    • --compare option should have option to compare specified change_sets in the file
  • exodiff
    • if input and output have change_sets, do diff on each change_set
    • allow diffing a specific change_set in a file with a specific change_set in the other file
    • allow diffing a specific change_set in a file with a file that has no change_sets.
  • conjoin
    • "union" all change_sets in a file to a single file with status variables (treat the change_sets similar to how individual files are currently handled)
  • epu
    • a set of file-per-rank files with change_sets should produce a single file with the same set of change_sets.
Clone this wiki locally