add option to restrict single-Path jobs of hltIntegrationTests to a subset of triggers [14_0_X]

[missirol]

backport of #44005

PR description:

From the description of #44005:

This PRs adds a functionality to the utility hltIntegrationTests.

Currently, given a HLT menu with N Paths (excluding HLTriggerFinalPath), hltIntegrationTests runs N+1 jobs: one job running the full HLT menu in question, and N jobs each running only one of the N Paths of the menu standalone. This PR adds the option to run the 1-Path-standalone jobs for only a subset M of the Paths in the menu (M < N). This can be useful when testing an update which is known to modify only M Paths, where one normally wants to run (a) the full HLT menu, and (b) standalone jobs only for the M Paths in question, given that the other N-M Paths are unchanged.

This is implemented by adding the command-line argument --paths to hltIntegrationTests. In analogy to the --paths argument of hltGetConfiguration, one can specify patterns to select certain Paths using a comma-separated list. One can also specify Paths to be ignored/dropped by prepending the character - to a given pattern (e.g. --paths "*HLT*,-HLTriggerFinalPath" to ignore the Path HLTriggerFinalPath). The script hltListPaths is updated accordingly (technically, changing its CLI in a non-backward-compatible way). More details are provided in the help messages of the scripts updated in this PR.

Note.

• hltIntegrationTests [MENU] -x "--paths A,B,C" (before and after this PR) corresponds to running 4 jobs: one with a menu containing only 3 Paths (A, B, C), plus 1-Path jobs for each of those 3 Paths (A, B, C).
• hltIntegrationTests [MENU] --paths A,B,C (possible only with this PR) also corresponds to running 4 jobs: one with a 'full' menu containing all its Paths (possibly many more than A, B and C), plus 1-Path jobs only for the 3 Paths specified via --paths (i.e. A, B and C).

Minor code improvements are included as well.

Merely technical. No changes expected.

PR validation:

None beyond the checks done for #44005.

If this PR is a backport, please specify the original PR and why you need to backport that PR. If this PR will be backported, please specify to which release cycle the backport is meant for:

#44005

Backporting to 14_0_X for HLT-menu development in 2024.

[cmsbuild]

A new Pull Request was created by @missirol for CMSSW_14_0_X.

It involves the following packages:

• HLTrigger/Configuration (hlt)

@cmsbuild, @Martin-Grunewald, @mmusich can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @silviodonato this is something you requested to watch as well.
@sextonkennedy, @antoniovilela, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from add option to restrict single-Path jobs of hltIntegrationTests to a subset of triggers #44005

[cmsbuild]

cms-bot internal usage

[Martin-Grunewald]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323d0b/37547/summary.html
COMMIT: 043b424
CMSSW: CMSSW_14_0_X_2024-02-18-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44007/37547/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 63 lines to the logs
• Reco comparison results: 48 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3248626
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3248601
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 200 log files, 161 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[Martin-Grunewald]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_14_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_14_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

[mmusich]

urgent

• needed for trigger development

[antoniovilela]

+1