Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review MODE implementation to look for more computational efficiencies #2727

Open
8 of 22 tasks
JohnHalleyGotway opened this issue Nov 6, 2023 · 1 comment
Open
8 of 22 tasks
Assignees
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle component: code optimization Code optimization issue MET: Object Verification Object-based feature Verification priority: medium Medium Priority type: enhancement Improve something that it is currently doing

Comments

@JohnHalleyGotway
Copy link
Collaborator

JohnHalleyGotway commented Nov 6, 2023

Describe the Enhancement

Issue #2724 and corresponding PR #2726 enhance MODE in MET-12.0.0-beta2 by improving its convolution algorithm, adding OpenMP, and minimizing avoidable memory allocation. This issue to conduct a careful review of the rest of the MODE application and associated library code to look for additional potential efficiencies.

Some ideas to consider are copied below from this issue comment:

  • The ShapeData::n_objects() function calls split which loops over the entire grid and can be slow. Could eliminate calls to n_objects() or remove that function entirely to prevent its use.
  • Most of the time MODE loops over a grid that is very sparse, often only containing a single object. Consider adding bounding box information to the ShapeData class to keep track of the extent of non-zero object data. Update member functions to loop over that bounding box rather than the entire domain.
  • Review each call to DataPlane::get(x,y) which calls two_to_one() which performs a bounds check. Can they replaced with direct operation on the vector of Data? Or can they be replaced with a new accessor function that skips the bounds checking? Running @DanielAdriaansen test case with 75 tiny objects through a profiler, 50% of the ~10 min runtime is spent accessing data.
  • Data is read in twice in multivar_frontend.cc, with calls to ModeFrontEnd::create_multivar_simple_objects() and ModeFrontend::create_multivar_merge_objects(). Instead the data should be kept in memory when creating the simple objects and accessed when creating the merge objects.

Time Estimate

2 days?

Sub-Issues

Consider breaking the enhancement down into sub-issues.

  • Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Review default alert labels
  • Select component(s)
  • Select priority
  • Select requestor(s)

Milestone and Projects

  • Select Milestone as the next official version or Backlog of Development Ideas
  • For the next official version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the next official version
    Select: MET-X.Y.Z Development project for development toward the next official release
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: enhancement Improve something that it is currently doing alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: code optimization Code optimization issue reporting: DTC NOAA R2O NOAA Research to Operations DTC Project MET: Object Verification Object-based feature Verification labels Nov 6, 2023
@JohnHalleyGotway JohnHalleyGotway added this to the MET 12.0.0 milestone Nov 6, 2023
@JohnHalleyGotway JohnHalleyGotway self-assigned this Nov 6, 2023
@davidalbo
Copy link
Contributor

The subtask of reading data in twice was resolved in this issue: https://github.com/dtcenter/MET/issues/2745

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle component: code optimization Code optimization issue MET: Object Verification Object-based feature Verification priority: medium Medium Priority type: enhancement Improve something that it is currently doing
Projects
None yet
Development

No branches or pull requests

2 participants