Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hydra: Add parsing of MPIEXEC_PREFIX_DEFAULT #5135

Closed
wants to merge 2 commits into from

Conversation

tkoehring
Copy link
Contributor

@tkoehring tkoehring commented Mar 9, 2021

Pull Request Description

There was no handling of the environment variable MPIEXEC_PREFIX_DEFAULT in the hydra code base. The variable adds the rank as a prefix to stdout in the form [%r] by settings its value to anything greated than zero. A new function, HYD_check_envs, was created to handle this and can be used for the handling of other environment variables in the future. The already existing function, prepend_rank_fn, is used to add the pattern to stdout. This function checks for duplicate settings so using both MPIEXEC_PREFIX_DEFAULT and -prepend-rank will correctly result in an error. These new settings were tested manually and can
confirm that it works as intended.

Fixes #4388

TODO

  • Confirm user reported issue
  • Implement Fix
  • Test Fix

Expected Impact

Users will now be able to user environment variable MPIEXEC_PREFIX_DEFAULT

Author Checklist

  • Provide Description
    Particularly focus on why, not what. Reference background, issues, test failures, xfail entries, etc.
  • Commits Follow Good Practice
    Commits are self-contained and do not do two things at once.
    Commit message is of the form: module: short description
    Commit message explains what's in the commit.
  • Passes All Tests
    Whitespace checker. Warnings test. Additional tests via comments.
  • Contribution Agreement
    For non-Argonne authors, check contribution agreement.
    If necessary, request an explicit comment from your companies PR approval manager.

}
}
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The internal function should be declared as static and you don't need a namespace for static functions.

@tkoehring tkoehring force-pushed the issue_4388 branch 4 times, most recently from c17b5f9 to 66672c2 Compare March 10, 2021 23:12
{
HYD_status status = HYD_SUCCESS;
int val;
char *arg = "MPIEXEC_PREFIX_DEFAULT";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the extra variable.

if (MPL_env2int("MPIEXEC_PREFIX_DEFAULT", &val)) {
if (val > 0) {
//prepend_rank_fn("prepend-rank", NULL);
status = HYDU_set_str(arg, &HYD_ui_info.prepend_pattern, "[%r] ");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do HYDU_ERR_POP(status, ...), and add fn_fail, fn_exit labels. This way in the future when we add addition environment variable, one wouldn't easily overwritten the status return.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When later during command-line parsing, and if additional -l is given and we set the string twice, does it result in memory leak? Please double check.

src/pm/hydra/ui/mpich/utils.c Show resolved Hide resolved
@tkoehring tkoehring force-pushed the issue_4388 branch 3 times, most recently from 502671a to 8370aeb Compare March 11, 2021 21:08
@tkoehring tkoehring requested a review from hzhou March 11, 2021 21:57
@hzhou
Copy link
Contributor

hzhou commented Mar 12, 2021

@travisk1154 Looks good. Could you swap the order of the commits? That way we keep each commits to be able to compile clean.

Also, did you manually test and confirm the user issue is fixed?

@hzhou
Copy link
Contributor

hzhou commented Mar 12, 2021

test: mpich/ch4/ofi

@tkoehring
Copy link
Contributor Author

Yea I have manually tested things. Setting MPIEXEC_PREFIX_DEFAULT to equal 1 or higher adds the rank successfully. Anything equal to zero or lower does not do anything. Having MPIEXEC_PREFIX_DEFAULT=1 and -prepend-rank sends and error and stops the program as expected. Having MPIEXEC_PREFIX_DEFAULT=0 and -prepend-rank adds the rank as expected.

@hzhou
Copy link
Contributor

hzhou commented Mar 12, 2021

Having MPIEXEC_PREFIX_DEFAULT=1 and -prepend-rank sends and error and stops the program as expected.

This is not ideal

Copy link
Contributor

@hzhou hzhou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the non-ideal behavior when we have conflicting options via environment and command options -- currently it results in error, ideally we'd like the command option take priority -- this PR addresses the issue and IMO it's ok to merge.

HYDU_set_str was updated to take a const char* instead of a char*. No
where in the function is the char* value modified and this now allows
string literals to be passed. char* will be cast in to const char* and
there should be no issue with this change.
There was no handling of the environment variable MPIEXEC_PREFIX_DEFAULT in the
hydra code base. The variable adds the rank as a prefix to stdout in the form
[%r] by settings its value to anything greated than zero. A new function,
HYD_check_envs, was created to handle this and can be used for the handling of
other environment variables in the future. The already existing function,
prepend_rank_fn, checks for duplicate settings so using both MPIEXEC_PREFIX_DEFAULT
and -prepend-rank will correctly result in an error. These new settings were tested
manually and confirm that the changes work as intended.
if (val >= 1) {
goto fn_exit;
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point of processing environment in check_envs is so we don't check environment again in another place.

@hzhou hzhou mentioned this pull request Feb 15, 2022
4 tasks
@hzhou
Copy link
Contributor

hzhou commented Feb 21, 2022

We decided to remove these environment variables. See #5848.

@hzhou hzhou closed this Feb 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

doc: MPIEXEC_PREFIX_DEFAULT doesn't work
2 participants