Enhance TCPairs to loop by valid time and allow looping when LOOP_ORDER = processes

[georgemccabe]

Discussed in #985

^{Originally posted by mollybsmith-noaa July 6, 2021}
Is it possible to loop by valid time when verifying with METplus's TCPairs? I have a workflow that succeeds when looping by init time, with these variables:

PROCESS_LIST = TCPairs
LOOP_BY = INIT

INIT_TIME_FMT = %Y%m%d%H
INIT_BEG = 2021070100
INIT_END = 2021070100

INIT_INCREMENT = 21600

But when I replace every instance of INIT with VALID, I get errors related to missing INIT fields, such as configparser.NoOptionError: No option 'INIT_TIME_FMT' in section: 'config'. This is despite the fact that the LOOP_BY variable is definitely set to VALID.

Is there a correct way to do this? With realtime verification, verification by valid time really has to be used instead of init time, or there's no way to verify the forecast lead times that haven't happened yet.

Thanks!

Describe the Enhancement

Modify the TCPairs wrapper logic so that the VALID time info variables will be used if LOOP_BY = VALID and LOOP_ORDER = processes. See discussion for more details.

Time Estimate

<1 day

Sub-Issues

Consider breaking the enhancement down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

4.1.0-beta1

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required: George McCabe
• Select scientist(s) or no scientist required: Molly Smith (NOAA) for review

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.