Add D88 wfs to the short matrix #36833

srimanob · 2022-01-29T16:11:46Z

PR description:

As foreseen on move relvals to D88 in 12_3_0_pre5, this PR adds D88 workflows to the short matrix. We then can remove D86 and D77 in the later PR after migration to D88 is complete.

In addition, two workflows (39496.0,39500.0) of CloseByPGun are added to the short matrix as discussed in #36832

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

Not a backport, and no need of backport

cmsbuild · 2022-01-29T16:20:53Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36833/28011

This PR adds an extra 20KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_2026.py modified in PR(s): Phase2 tracker: add T30 (geometry with more realistic TFPX) #36660

cmsbuild · 2022-01-29T16:21:13Z

A new Pull Request was created by @srimanob (Phat Srimanobhas) for master.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)

@jordan-martins, @bbilin, @wajidalikhan, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @Martin-Grunewald, @missirol, @fabiocos, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

srimanob · 2022-01-29T16:22:16Z

@cmsbuild please test with #36753, #36728

cmsbuild · 2022-01-29T17:53:12Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-da3bae/22075/summary.html
COMMIT: bf5fb39
CMSSW: CMSSW_12_3_X_2022-01-29-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36833/22075/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

RelVals-INPUT

12034.0DAS Error
12434.0DAS Error

cmsbuild · 2022-01-30T01:09:36Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36833/28016

This PR adds an extra 20KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_2026.py modified in PR(s): Phase2 tracker: add T30 (geometry with more realistic TFPX) #36660

srimanob · 2022-02-01T07:33:38Z

@cmsbuild please test

cmsbuild · 2022-02-01T11:09:09Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-da3bae/22113/summary.html
COMMIT: 2a7d29a
CMSSW: CMSSW_12_3_X_2022-01-31-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36833/22113/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-da3bae/39434.0_TTbar_14TeV+2026D88+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-da3bae/39496.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-da3bae/39500.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 43
DQMHistoTests: Total histograms compared: 3449612
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3449590
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 42 files compared)
Checked 181 log files, 42 edm output root files, 43 DQM output files
TriggerResults: no differences found

srimanob · 2022-02-01T11:38:57Z

+Upgrade

The PR introduce D88 workflows to the short matrix. We will have another PR to remove D77/D86 when the relvals migration is complete. PR test run fine, no change is expected.

I think we should start running D88 in a short matrix with PR test for a while before we cut the release and produce relvals. Note to @cms-sw/reconstruction-l2 to prepare for comparison config.

srimanob · 2022-02-01T17:28:42Z

@cms-sw/pdmv-l2 This PR is good to review. The premixing wf will be added later, together with cleanup when we complete the migration.

kskovpen · 2022-02-01T17:54:01Z

+pdmv

Looks good to us

cmsbuild · 2022-02-01T17:54:18Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-02-02T08:53:49Z

Configuration/PyReleaseValidation/python/relval_steps.py

@@ -3464,6 +3464,7 @@ def gen2021HiMix(fragment,howMuch):
 defaultDataSets['2026D49']='CMSSW_12_0_0_pre4-113X_mcRun4_realistic_v7_2026D49noPU-v'
 defaultDataSets['2026D76']='CMSSW_12_0_0_pre4-113X_mcRun4_realistic_v7_2026D76noPU-v'
 defaultDataSets['2026D77']='CMSSW_12_1_0_pre2-113X_mcRun4_realistic_v7_2026D77noPU-v'
+#defaultDataSets['2026D88']='CMSSW_12_2_0_pre3-122X_mcRun4_realistic_v4_2026D88noPU-v1'


Why is this line commented out?

I first wanted to add also premix workflow to the short matrix test as other Phase-2 defaults. However, I think it is better to include it when we have updated MinBias from the first relvals in 12_3_0_pre5. So, this line will be updated and uncomment.

If you need a clean PR, I can remove it for now.

perrotta · 2022-02-02T09:43:54Z

+1

srimanob · 2022-02-02T09:51:07Z

FYI @cms-sw/reconstruction-l2

We include 3 workflows to the short matrix, including

39434.0, #2026D88 ttbar
39496.0, #CE_E_Front_120um D88
39500.0, #CE_H_Coarse_Scint D88
In case you would like to update the comparison list. Thanks.

qliphy · 2022-02-03T01:37:16Z

@srimanob @cms-sw/pdmv-l2 There appear several wf failures in IB after this PR gets merged.

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc10/CMSSW_12_3_X_2022-02-02-1100/pyRelValMatrixLogs/run/39634.999_TTbar_14TeV+2026D88PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU/step2_TTbar_14TeV+2026D88PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU.log#/29-29

the query is file dataset = /RelValMinBias_14TeV/1/GEN-SIM
DAS succeeded after 1 attempts 0
found files: []

Maybe we should update the mixing dataset?

srimanob · 2022-02-03T02:33:43Z

@qliphy
Why IB runs also premix workflow? Is it because I put it in relvals_2026?
My intention was to update the premix workflows when we have minbias from 12_3_0_pre5. Should I disable it for now in relvals_2026?

kpedro88 · 2022-02-03T02:44:40Z

@srimanob yes, anything in relvals_2026 can run in IBs.

srimanob · 2022-02-03T02:51:44Z

Thanks @kpedro88 for clarification. I disable it for now in #36871

Norraphat added 2 commits January 29, 2022 17:04

add D88 to short matrix

274d0b5

remove unused space

7f8b540

cmsbuild added this to the CMSSW_12_3_X milestone Jan 29, 2022

cmsbuild added code-checks-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Jan 29, 2022

add D86 back for the moment

bf5fb39

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 29, 2022

cmsbuild added requires-external tests-started and removed tests-pending labels Jan 29, 2022

cmsbuild added tests-rejected and removed tests-started labels Jan 29, 2022

correct wf number

3bdb59e

cmsbuild added code-checks-pending tests-pending and removed tests-rejected requires-external code-checks-approved labels Jan 30, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 30, 2022

cmsbuild added tests-started and removed tests-pending labels Feb 1, 2022

cmsbuild added tests-approved and removed tests-started labels Feb 1, 2022

cmsbuild added upgrade-approved and removed upgrade-pending labels Feb 1, 2022

cmsbuild mentioned this pull request Feb 1, 2022

Phase2 tracker: add T30 (geometry with more realistic TFPX) #36660

Merged

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Feb 1, 2022

perrotta reviewed Feb 2, 2022

View reviewed changes

cmsbuild added orp-approved and removed orp-pending labels Feb 2, 2022

cmsbuild merged commit a53fa96 into cms-sw:master Feb 2, 2022

cmsbuild mentioned this pull request Feb 2, 2022

properly checkout full cmssw for non-master IBs cms-sw/cms-bot#1705

Merged

srimanob mentioned this pull request Feb 3, 2022

Disable D88 PU wfs #36871

Merged

slava77 mentioned this pull request Feb 3, 2022

reco comparisons: add D88 workflows to the reco comparisons map cms-sw/cms-bot#1707

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add D88 wfs to the short matrix #36833

Add D88 wfs to the short matrix #36833

srimanob commented Jan 29, 2022 •

edited

Loading

cmsbuild commented Jan 29, 2022

cmsbuild commented Jan 29, 2022

srimanob commented Jan 29, 2022

cmsbuild commented Jan 29, 2022

cmsbuild commented Jan 30, 2022

srimanob commented Feb 1, 2022

cmsbuild commented Feb 1, 2022

srimanob commented Feb 1, 2022

srimanob commented Feb 1, 2022

kskovpen commented Feb 1, 2022

cmsbuild commented Feb 1, 2022

perrotta Feb 2, 2022

srimanob Feb 2, 2022

srimanob Feb 2, 2022

perrotta commented Feb 2, 2022

srimanob commented Feb 2, 2022

qliphy commented Feb 3, 2022

srimanob commented Feb 3, 2022

kpedro88 commented Feb 3, 2022

srimanob commented Feb 3, 2022

Add D88 wfs to the short matrix #36833

Add D88 wfs to the short matrix #36833

Conversation

srimanob commented Jan 29, 2022 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jan 29, 2022

cmsbuild commented Jan 29, 2022

srimanob commented Jan 29, 2022

cmsbuild commented Jan 29, 2022

RelVals

RelVals-INPUT

cmsbuild commented Jan 30, 2022

srimanob commented Feb 1, 2022

cmsbuild commented Feb 1, 2022

Comparison Summary

srimanob commented Feb 1, 2022

srimanob commented Feb 1, 2022

kskovpen commented Feb 1, 2022

cmsbuild commented Feb 1, 2022

perrotta Feb 2, 2022

Choose a reason for hiding this comment

srimanob Feb 2, 2022

Choose a reason for hiding this comment

srimanob Feb 2, 2022

Choose a reason for hiding this comment

perrotta commented Feb 2, 2022

srimanob commented Feb 2, 2022

qliphy commented Feb 3, 2022

srimanob commented Feb 3, 2022

kpedro88 commented Feb 3, 2022

srimanob commented Feb 3, 2022

srimanob commented Jan 29, 2022 •

edited

Loading