Move Phase2 wf in the short matrix to D77 #35862

srimanob · 2021-10-27T09:18:29Z

PR description:

As title said, we move PR test workflows of Phase-2 to D77 following the green light of PdmV, to move the D77 relvals from CMSSW_12_1_0_pre5. I use this chance also to define missing PU200 D76, in case it will be used for any testing in PR test.

https://its.cern.ch/jira/projects/PDMVRELVALS/issues/PDMVRELVALS-139

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

No need of backport.

cmsbuild · 2021-10-27T09:25:07Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35862/26241

This PR adds an extra 20KB to repository

cmsbuild · 2021-10-27T09:25:30Z

A new Pull Request was created by @srimanob (Phat Srimanobhas) for master.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)

@jordan-martins, @bbilin, @wajidalikhan, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @Martin-Grunewald, @missirol, @fabiocos, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

srimanob · 2021-10-27T09:25:56Z

@cmsbuild please test

cmsbuild · 2021-10-27T10:36:46Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22165d/19975/summary.html
COMMIT: a3e441b
CMSSW: CMSSW_12_1_X_2021-10-26-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35862/19975/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

35234.999

----- Begin Fatal Exception 27-Oct-2021 12:16:02 CEST-----------------------
An exception of category 'NoSecondaryFiles' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
Exception Message:
RootEmbeddedFileSequence no input files specified for secondary input source.
----- End Fatal Exception -------------------------------------------------

kpedro88 · 2021-10-27T14:18:58Z

@srimanob you need to provide a default (minbias) dataset for D77 here:

cmssw/Configuration/PyReleaseValidation/python/relval_steps.py

Line 3341 in c2d7d66

    
           defaultDataSets['2026D76']='CMSSW_12_0_0_pre4-113X_mcRun4_realistic_v7_2026D76noPU-v'

srimanob · 2021-10-27T14:26:01Z

@srimanob you need to provide a default (minbias) dataset for D77 here:

cmssw/Configuration/PyReleaseValidation/python/relval_steps.py

Line 3341 in c2d7d66

defaultDataSets['2026D76']='CMSSW_12_0_0_pre4-113X_mcRun4_realistic_v7_2026D76noPU-v'

Thanks, @kpedro88
I just check with the last D77 for validation relvals. I will add it.

cmsbuild · 2021-10-27T14:45:15Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35862/26247

This PR adds an extra 48KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Use pdigi_hi_nogen for HI MC wfs #35028, Add DQM Matrix to RelVals #35678

cmsbuild · 2021-10-27T14:45:39Z

Pull request #35862 was updated. @jordan-martins, @bbilin, @wajidalikhan, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen can you please check and sign again.

srimanob · 2021-10-27T15:03:56Z

@cmsbuild please test

cmsbuild · 2021-10-27T17:59:58Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22165d/19988/summary.html
COMMIT: 871e4ea
CMSSW: CMSSW_12_1_X_2021-10-27-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35862/19988/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-22165d/11634.914_TTbar_14TeV+2021_DDDDB+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-22165d/35034.0_TTbar_14TeV+2026D77+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-22165d/35234.999_TTbar_14TeV+2026D77PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-22165d/38634.0_TTbar_14TeV+2026D86+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 40
DQMHistoTests: Total histograms compared: 2693134
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2693112
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 39 files compared)
Checked 168 log files, 32 edm output root files, 40 DQM output files
TriggerResults: no differences found

srimanob · 2021-10-27T22:51:34Z

@smuzaffar
There is failure in DAS query of new workflow which calls /RelValTTbar_14TeV/CMSSW_12_1_0_pre2-113X_mcRun4_realistic_v7_2026D77noPU-v1/GEN-SIM

Do we need to trigger it to be at CERN? Also MinBias for PR test? Thanks.

# in: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/dasqueries/run going to execute cd 35034.0_TTbar_14TeV+2026D77+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal
 dasgoclient --limit 0 --query 'file dataset=/RelValTTbar_14TeV/CMSSW_12_1_0_pre2-113X_mcRun4_realistic_v7_2026D77noPU-v1/GEN-SIM site=T2_CH_CERN' | ibeos-lfn-sort -u > step1_dasquery.log  2>&1

https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22165d/19988/dasqueries/run.txt

srimanob · 2021-10-27T22:54:43Z

+Upgrade

This is a technical PR to migrate to D77 in the short matrix. No changes are expected for other workflows. Comparison can't be done on D86 (as removed), D77 (as new). Comparison does not run also in 11634.914 which is new Run-3 DDD which is not about this workflow, but new (introduced in #35761)

smuzaffar · 2021-10-28T06:17:19Z

@srimanob , there is no need to transfer the data set to CERN. CI/CD system will cache it automatically.

srimanob · 2021-10-31T15:44:35Z

@cms-sw/pdmv-l2 Could you please consider?

Note that, you can use https://github.com/cms-sw/cmssw/pulls?q=is%3Apr+label%3Apdmv-pending+is%3Aopen to follow what is pending for PdmV.

bbilin · 2021-10-31T15:59:54Z

+1

cmsbuild · 2021-10-31T16:00:15Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2021-11-01T08:57:12Z

Configuration/PyReleaseValidation/python/relval_2026.py

 numWFIB.extend([34634.21,34834.21,34834.9921]) #2026D76 prodlike, prodlike PU, prodlike premix stage1+stage2
 numWFIB.extend([35034.0]) #2026D77
+numWFIB.extend([35234.99,35234.999]) #2026D77 premixing combined stage1+stage2 (ttbar+PU200, ttbar+PU50 for PR test)


(In the next iteration) one could reorder in ascending wf number

perrotta · 2021-11-01T08:57:30Z

+1

move phase2 short matric to D77

a3e441b

cmsbuild added this to the CMSSW_12_1_X milestone Oct 27, 2021

cmsbuild added code-checks-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Oct 27, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 27, 2021

cmsbuild added tests-started and removed tests-pending labels Oct 27, 2021

cmsbuild added tests-rejected and removed tests-started labels Oct 27, 2021

add default D77 minbias

871e4ea

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Oct 27, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 27, 2021

cmsbuild added tests-started and removed tests-pending labels Oct 27, 2021

cmsbuild added tests-approved and removed tests-started labels Oct 27, 2021

cmsbuild added upgrade-approved and removed upgrade-pending labels Oct 27, 2021

smuzaffar mentioned this pull request Oct 28, 2021

[Do not merge] PR to test other cms PRs cms-sw/cms-bot#1655

Closed

This was referenced Oct 28, 2021

introduce COPYPASTER3 and use it as step2 in wf 138.3 #35890

Merged

Fixed the PPS alignment relval - 1042 #35874

Merged

smuzaffar modified the milestones: CMSSW_12_1_X, CMSSW_12_2_X Oct 29, 2021

cmsbuild mentioned this pull request Oct 29, 2021

restrict RunHI2011 to 182124:40, used in the short matrix wf 140.53 #35912

Merged

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Oct 31, 2021

slava77 mentioned this pull request Oct 31, 2021

reco comparisons: add a minimal set of wf maps for D77 cms-sw/cms-bot#1657

Merged

perrotta reviewed Nov 1, 2021

View reviewed changes

cmsbuild added orp-approved and removed orp-pending labels Nov 1, 2021

cmsbuild merged commit ddc7bf8 into cms-sw:master Nov 1, 2021

cmsbuild mentioned this pull request Nov 1, 2021

rootmodule: Updated root to tip of branch master cms-sw/cmsdist#7421

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move Phase2 wf in the short matrix to D77 #35862

Move Phase2 wf in the short matrix to D77 #35862

srimanob commented Oct 27, 2021 •

edited

Loading

cmsbuild commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

srimanob commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

kpedro88 commented Oct 27, 2021

srimanob commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

srimanob commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

srimanob commented Oct 27, 2021

srimanob commented Oct 27, 2021 •

edited

Loading

smuzaffar commented Oct 28, 2021

srimanob commented Oct 31, 2021

bbilin commented Oct 31, 2021

cmsbuild commented Oct 31, 2021

perrotta Nov 1, 2021

perrotta commented Nov 1, 2021

Move Phase2 wf in the short matrix to D77 #35862

Move Phase2 wf in the short matrix to D77 #35862

Conversation

srimanob commented Oct 27, 2021 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

srimanob commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

RelVals

kpedro88 commented Oct 27, 2021

srimanob commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

srimanob commented Oct 27, 2021

cmsbuild commented Oct 27, 2021

Comparison Summary

srimanob commented Oct 27, 2021

srimanob commented Oct 27, 2021 • edited Loading

smuzaffar commented Oct 28, 2021

srimanob commented Oct 31, 2021

bbilin commented Oct 31, 2021

cmsbuild commented Oct 31, 2021

perrotta Nov 1, 2021

Choose a reason for hiding this comment

perrotta commented Nov 1, 2021

srimanob commented Oct 27, 2021 •

edited

Loading

srimanob commented Oct 27, 2021 •

edited

Loading