Remove 12450.* wfs (2023, ZMuMu) from GPU RelVals

[missirol]

PR description:

#41354 added Run-3 wfs with 2023 conditions to the list of GPU RelVals.

After that PR was merged, the 12450.* wfs failed in the relevant IB (i.e. CMSSW_13_1_GPU_X_2023-04-21-2300), likely because the corresponding GEN-SIM input sample does not exist (see #41354 (comment)).

This PR disables those GPU RelVals to avoid IB failures, until the situation is clarified.

PR validation:

None.

If this PR is a backport, please specify the original PR and why you need to backport that PR. If this PR will be backported, please specify to which release cycle the backport is meant for:

If accepted, this PR will be backported in #41371.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41386/35252

• This PR adds an extra 24KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/relval_gpu.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255
  • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Adding all geometries and the FullRecoGPU for Patatrack wfs for Phase2  #41011, Guard for No Digis for Phase2 and Patatrack NuGun #41255, Add run3 tracking low pu era #33532, Displaced regional tracking iteration #40643

[cmsbuild]

A new Pull Request was created by @missirol (Marino Missiroli) for master.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv, upgrade)

@bbilin, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @sunilUIET can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @fabiocos, @slomeo, @kpedro88 this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41386/35253

• This PR adds an extra 12KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/relval_gpu.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255

[cmsbuild]

Pull request #41386 was updated. @bbilin, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @sunilUIET can you please check and sign again.

[missirol]

test parameters:

• enable = gpu

[missirol]

please test

@cms-sw/orp-l2 , this PR 'fixes' the 6 failures in the CMSSW_13_1_GPU_X_2023-04-21-2300 IB by removing the problematic wfs, see #41354 (comment).

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-41accc/32090/summary.html
COMMIT: cfa4fa3
CMSSW: CMSSW_13_1_X_2023-04-22-1100/el8_amd64_gcc11
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41386/32090/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 11 lines from the logs
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3460915
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3460893
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 207 log files, 159 edm output root files, 48 DQM output files
• TriggerResults: no differences found

GPU Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 4
• DQMHistoTests: Total histograms compared: 19870
• DQMHistoTests: Total failures: 10
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 19860
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
• Checked 12 log files, 9 edm output root files, 4 DQM output files
• TriggerResults: no differences found

[AdrianoDee]

+1
Signing as a quick fix to avoid IB failures.

[srimanob]

Hi @missirol @AdrianoDee

The line that control processing string is
https://github.com/cms-sw/cmssw/blob/master/Configuration/PyReleaseValidation/python/relval_steps.py#L3920

It tries to look datasets with processing string CMSSW_12_5_0_pre4-124X_mcRun3_2023_realistic_v11_BS2022-v1.

Most of 12_5_0_pre4 were produced with v10 (see DAS). Those datasets were created on 14-15 Aug 22.

For v11 (default GT of 2023 in 12_5_0_pre4), datasets seem to be produced few days after, but only TTbar and MinBias (see DAS). Maybe resubmission happened just for these two when #39041 (commit) was prepared.

If GPU 12450.* wfs are needed anyways, I think we should not merge this PR, but @cms-sw/pdmv-l2 to submit ZMM using 12_5_0_pre4. You should get input for those workflows. It will take only few days, maybe we can keep the error for now, to avoid re-opening in the next few days.

[AdrianoDee]

@srimanob totally agree that the preferred solution would be to have the adequate sample produced. Didn't intend to be hasty, I wanted to give way to @cms-sw/orp-l2 to choose if this is something we can live with for now or they prefer to a quick fix now (-:

On a side note: I imagine #41225 is failing for the same reason, isn't it?

[missirol]

(I guess Adriano meant #41255, not #41225).

@cms-sw/pdmv-l2 , we need your feedback.

Do you plan to produce the missing sample? If so, how long would that take? Do you think this PR should be merged in the meantime?

This is holding the integration of #41371 in 13_0_X, which would be useful to HLT.

[missirol]

@kskovpen @sunilUIET @bbilin , please review this PR (and see the question above for PdmV).

[sunilUIET]

I think we can go ahead without missing samples. In the meantime, we will also submit the required samples.

[sunilUIET]

+pdmv

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

I think we can go ahead without missing samples. In the meantime, we will also submit the required samples.

Do you have a time estimate for the production of the missing samples? Of course, the preferred option would be to have the tests running, and if it can be achieved in just a few days I'd rather avoid merging this PR

[missirol]

Do you have a time estimate for the production of the missing samples? Of course, the preferred option would be to have the tests running, and if it can be achieved in just a few days I'd rather avoid merging this PR

@sunilUIET , please see the question above by ORP. Prompt feedback is appreciated, so #41371 could also move forward.

[sunilUIET]

https://dmytro.web.cern.ch/dmytro/cmsprodmon/requests.php?campaign=CMSSW_12_5_0_pre4__fullsim_noPU_2023_14TeV-1682483673 samples are done. I hope test should be fine now. Please let us know if something is missing

[missirol]

Thanks @sunilUIET. Closing this PR.