Enhance TC-Pairs to disable the output of consensus track members

[JiayiPeng-NOAA]

Hi John,
I downloaded the tar file from your email and saved on Hera:
/scratch2/NCEPDEV/ensemble/save/Jiayi.Peng/bak/tc_pairs_consensus_example

From the configure file: /scratch2/NCEPDEV/ensemble/save/Jiayi.Peng/bak/tc_pairs_consensus_example/TCPairsConfig_CONSENSUS,
it shows the UK ensemble consensus setting:
name = "UEMN_CONS";
members = [ "UE00", "UE01", "UE02", "UE03", "UE04",
"UE05", "UE06", "UE07", "UE08", "UE09",
"UE10", "UE11", "UE12", "UE13", "UE14",
"UE15", "UE16", "UE17", "UE18", "UE19",
"UE20", "UE21", "UE22", "UE23", "UE24",
"UE25", "UE26", "UE27", "UE28", "UE29",
"UE30", "UE31", "UE32", "UE33", "UE34",
"UE35" ];
required = [];
min_req = 36;

The tc-pair output file (al132020_CONSENSUS.tcst) includes UEMN_CONS and the 36 ensemble members (UE00, UE01,... UE35).
How can I change the configure file (the setting) in order to do the tc-pair for the ensemble mean only (UEMN_CONS)?

To save time and energy, there is no need to do the tc-pair for individual ensemble members (UE00, UE01, etc).
And also, the output (al132020_CONSENSUS.tcst) does not need include the data for (UE00, UE01, etc).

Best regards,
Jiayi

[JohnHalleyGotway]

Listing @JiayiPeng-NOAA on the issue as the advising scientist.

[JohnHalleyGotway]

Recommend adding a new write_members = TRUE/FALSE config option. See this sample TC-Pairs config file.

consensus = [
   {
      name     = "HCCA_CONS";
      members  = [ "AEMI", "GFSI", "CTCI", "DSHP", "EGRI", "EMN2", "EMXI", "HWFI", "LGEM" ];
      required = [];
      min_req  = 8;
      write_members = TRUE; // Default = TRUE
   },
   {
      name     = "GFEX_CONS";
      members  = [ "GFSI", "EMXI" ];
      required = [];
      min_req  = 2;
      write_members = FALSE;
   },

What to do when a member appears in both lists? Which should take precedence... write_members = TRUE or write_members = FALSE?

Recommend letting FALSE take precedence over TRUE so that we keep track of the model names to be EXCLUDED from the output. That makes the logic much easier to implement rather then keeping track of the model names to be included.

Relevant files:

• Add new entry for write_members in src/basic/vx_config/config_constants.h.
• Update consensus parsing in src/tools/tc_utils/tc_pairs/tc_pairs_conf_info.cc and store the new boolean in the ConsensusInfo struct.
• Update the ConsensusInfo struct in src/tools/tc_utils/tc_pairs/tc_pairs_conf_info.h to add this boolean.
• Make sure the boolean it TRUE by default to maintain backward compatibility.

      Consensus[i].WriteMembers = (*dict)[i]->dict_value()->lookup_bool(conf_key_write_members);
...
      // If required is empty, default to 0
      if(!Consensus[i].WriteMembers) {
         SkipConsensusMembers.add(Consensus[i].Members); // could do this fancier to prevent duplicates
     }

Use StringArray::has(string) to determine if the model name appears in the list to be skipped.

Once we have our StringArray list of SkipConsensusMembers to be skipped, we should throw those tracks away ASAP rather than actually processing them. Update the existing filter_tracks() function to discard any consensus members that should be skipped.

Add a StringArray object to keep track of which models should be omitted from the output across all consensus definitions.

[JiayiPeng-NOAA]

Hi John, Thanks for the hard work! Please see the previous email (attached below). We have two key points for the TC-pair new code: (1) there is no need to do the tc-pair for individual ensemble members (UE00, UE01, etc). (2) there is no need to output the tc-pair data for individual ensemble members. In this new code update (#2429), you add the option "write_members = FALSE". Does this mean the new code is still calculating TC-pair for individual ensemble members ? If the answer is yes, please add another option "TCpair_members=FALSE" in the new code setting to save CPU time for verification jobs. Thanks, Jiayi -----------------------below : the previous email -------------------------------------- Hi John, I downloaded the tar file from your email and saved on Hera: /scratch2/NCEPDEV/ensemble/save/Jiayi.Peng/bak/tc_pairs_consensus_example From the configure file: /scratch2/NCEPDEV/ensemble/save/Jiayi.Peng/bak/tc_pairs_consensus_example/TCPairsConfig_CONSENSUS, it shows the UK ensemble consensus setting: name = "UEMN_CONS"; members = [ "UE00", "UE01", "UE02", "UE03", "UE04", "UE05", "UE06", "UE07", "UE08", "UE09", "UE10", "UE11", "UE12", "UE13", "UE14", "UE15", "UE16", "UE17", "UE18", "UE19", "UE20", "UE21", "UE22", "UE23", "UE24", "UE25", "UE26", "UE27", "UE28", "UE29", "UE30", "UE31", "UE32", "UE33", "UE34", "UE35" ]; required = []; min_req = 36; The tc-pair output file (al132020_CONSENSUS.tcst) includes UEMN_CONS and the 36 ensemble members (UE00, UE01,... UE35). How can I change the configure file (the setting) in order to do the tc-pair for the ensemble mean only (UEMN_CONS)? To save time and energy, there is no need to do the tc-pair for individual ensemble members (UE00, UE01, etc). And also, the output (al132020_CONSENSUS.tcst) does not need include the data for (UE00, UE01, etc). Best regards, Jiayi

…

On Fri, Feb 24, 2023 at 1:48 PM John Halley Gotway ***@***.***> wrote: Closed #2429 <#2429> as completed. — Reply to this email directly, view it on GitHub <#2429 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALRTVNLGCG76Y37PGQOASPDWZD677ANCNFSM6AAAAAAUPD2SGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[JiayiPeng-NOAA]

Hi John, When you get a chance, could you please respond the following email. Thanks, Jiayi

[JohnHalleyGotway]

Sorry Jiayi, I'd missed this one. The newly added consensus.write_members option in TC-Pairs is doing what you'd want it to do.

In the process_adecks() function, it...

1. Reads all the input ADeck track data files.
2. Derives any consensus tracks defined in the configuration file.
3. Filters tracks to discard any that shouldn't be retained.

When write_members = FALSE, the members tracks are discarded in the filtering step 3 above.

And that occurs PRIOR TO the matching with the Best track data. So TC-Pairs is NOT wasting any extra time processing the individual ensemble member tracks.

[JiayiPeng-NOAA]

Hi John, Could you please send me the new code with the test data? I would like to have a test. Thanks, Jiayi

[JohnHalleyGotway]

Jiayi, Julie will eventually install the MET-11.1.0-beta1 release ( https://github.com/dtcenter/MET/releases/tag/v11.1.0-beta1) on Hera for exactly this type of testing. But she’s currently focused on the MET-11.0.1 bugfix release. We can update this during the METplus NOAA telecon on Monday. John

[JiayiPeng-NOAA]

Hi John and Julie, I did test the consensus function by adding "write_members = FALSE;" on Hera, it works well. However, in my use case for 2022 Atlantic TC verification, I set up the TC_pair configure file (see below attached). The default setting of write_members is "TRUE", I try to modify it as "FALSE" through: TC_PAIRS_CONSENSUS1_write_members = FALSE TC_PAIRS_CONSENSUS2_write_members = FALSE TC_PAIRS_CONSENSUS3_write_members = FALSE But it does not work, The TC_pair is still doing the job to match the individual members with the best track. There may be a bug in MetPlus 5.1.0-beta1 wrapper (or somewhere else) to be fixed. Thanks, Jiayi

…

-------------Attached --------------------------------------------------------- TC_PAIRS_CONSENSUS1_NAME = MD01 TC_PAIRS_CONSENSUS1_MEMBERS = AP01,AP02,AP03,AP04,AP05,AP06,AP07,AP08,AP09,AP10,AP11,AP12,AP13,AP14,AP15,AP16,AP17,AP18,AP19,AP20,AP21,AP22,AP23,AP24,AP25,AP26,AP27,AP28,AP29,AP30 TC_PAIRS_CONSENSUS1_REQUIRED = false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false TC_PAIRS_CONSENSUS1_MIN_REQ = 12 TC_PAIRS_CONSENSUS1_write_members = FALSE TC_PAIRS_CONSENSUS2_NAME = MD02 TC_PAIRS_CONSENSUS2_MEMBERS = EP01,EP02,EP03,EP04,EP05,EP06,EP07,EP08,EP09,EP10,EP11,EP12,EP13,EP14,EP15,EP16,EP17,EP18,EP19,EP20,EP21,EP22,EP23,EP24,EP25,EN01,EN02,EN03,EN04,EN05,EN06,EN07,EN08,EN09,EN10,EN11,EN12,EN13,EN14,EN15,EN16,EN17,EN18,EN19,EN20,EN21,EN22,EN23,EN24,EN25 TC_PAIRS_CONSENSUS2_REQUIRED = false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false TC_PAIRS_CONSENSUS2_MIN_REQ = 20 TC_PAIRS_CONSENSUS2_write_members = FALSE TC_PAIRS_CONSENSUS3_NAME = MD03 TC_PAIRS_CONSENSUS3_MEMBERS = CP01,CP02,CP03,CP04,CP05,CP06,CP07,CP08,CP09,CP10,CP11,CP12,CP13,CP14,CP15,CP16,CP17,CP18,CP19,CP20 TC_PAIRS_CONSENSUS3_REQUIRED = false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false TC_PAIRS_CONSENSUS3_MIN_REQ = 8 TC_PAIRS_CONSENSUS3_write_members = FALSE -------------

[JiayiPeng-NOAA]

Hi John, Here is the error message to prove that the following setting does not work. The setting in TC-pair configure file: TC_PAIRS_CONSENSUS1_write_members = FALSE TC_PAIRS_CONSENSUS2_write_members = FALSE TC_PAIRS_CONSENSUS3_write_members = FALSE The error messages: WARNING: WARNING: TCPairsConfInfo::process_config() -> "consensus.write_members" is missing. Using default value of true. WARNING: WARNING: WARNING: TCPairsConfInfo::process_config() -> "consensus.write_members" is missing. Using default value of true. WARNING: WARNING: WARNING: TCPairsConfInfo::process_config() -> "consensus.write_members" is missing. Using default value of true. Thanks, Jiayi

[JiayiPeng-NOAA]

Hi Jiayi, Thanks for testing the new functionality. I'm added Logan and Geoff for their awareness. Take care, Alicia

[georgemccabe]

Hi @JiayiPeng-NOAA , the METplus config variable names should be in all caps, e.g. TC_PAIRS_CONSENSUS1_WRITE_MEMBERS, TC_PAIRS_CONSENSUS2_WRITE_MEMBERS, etc.