[12_4_X] Update Run3 MC GTs + swap update Vertex smearing + update offline Run3 GTs #39019

francescobrivio · 2022-08-10T09:50:07Z

PR description:

Backport of:

Update BeamSpot in Run3 MC GTs + swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing #38760 - update BeamSpot and VtxSmearing to Run3 values
Update SiStrip and SiPixel bad components for Run 3 MC GTs #39645 - update SiStrip and SiPixel bad components in MC GTs
Update Run3 offline and offline_relval GTs #39823 - update Run3 offline data GT (and offline relval GT) to the latest conditions prepared for the 2022ABCD re-reco (see this CMSTalk post)
Add MC GT for 2022 postEE leak #40101 - add phase1_2022_realistic_postEE key in autoCond for the postEE (Run3Summer22EE) MC campaign

This PR is meant to generally update the GTs in 12_4_X to make them even with respect to 12_5_X and 12_6_X.
The PR includes the swap of Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision smearing so that the tests also use the correct new vertex smearing (GS file added in #39548)

New tags included:

Tags for auto:phase1_2022_realistic:
- BeamSpotObjects_Realistic25ns_13p6TeVCollisions_Early2022_v3_mc
- BeamSpotOnlineObjects_Realistic25ns_13p6TeVCollisions_Early2022_v2_mc
- SiStripBadComponents_realisticMC_for2022_v2_mc
- SiPixelQuality_phase1_2022_v2_mc
- SiPixelQuality_forDigitizer_phase1_2022_v2_mc with label forDigitizer`
- SiPixelQuality_phase1_2022_forRawToDigi_v0 with label forRawToDigi`
Tags for auto:phase1_2022_cosmics:
- BeamSpotObjects_Realistic25ns_13p6TeVCollisions_Early2022_v3_mc
- BeamSpotOnlineObjects_Realistic25ns_13p6TeVCollisions_Early2022_v2_mc
- SiStripBadComponents_realisticMC_for2022_v2_mc
- SiPixelQuality_phase1_2022_v2_mc
- SiPixelQuality_forDigitizer_phase1_2022_v2_mc with label forDigitizer
Tags for auto:phase1_2022_realistic_hi:
- SiStripBadComponents_realisticMC_for2022_v2_mc
- SiPixelQuality_phase1_2022_v2_mc
- SiPixelQuality_forDigitizer_phase1_2022_v2_mc with label forDigitizer
Tags for auto:phase1_2023_realistic:
- SiStripBadComponents_realisticMC_for2022_v2_mc
- SiPixelQuality_phase1_2023_v2_mc
- SiPixelQuality_forDigitizer_phase1_2023_v2_mc with label forDigitizer
Tags for auto:phase1_2024_realistic:
- SiStripBadComponents_realisticMC_for2022_v2_mc
- SiPixelQuality_phase1_2024_v2_mc
- SiPixelQuality_forDigitizer_phase1_2024_v2_mc with label forDigitizer
Tags for auto:run3_data
- too many changes to be listed here, see diff below
Tags for auto:run3_data_relval:
- identical to auto:run3_data with fixed L1 menu

GT Diffs:

Offline Run3: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_v9/124X_dataRun3_v11
Offline relval Run3: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_relval_v8/124X_dataRun3_relval_v11
Diff between new offline and offline relval GTs: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_v11/124X_dataRun3_relval_v11
Run3 realistic MC: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2022_realistic_v10/124X_mcRun3_2022_realistic_v12
Run3 cosmic MC: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2022cosmics_realistic_deco_v11/124X_mcRun3_2022cosmics_realistic_deco_v13
Run3 HI MC https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2022_realistic_HI_v10/124X_mcRun3_2022_realistic_HI_v12
Run3 2023 MC https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2023_realistic_v11/124X_mcRun3_2023_realistic_v13
Run3 2024 MC https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2024_realistic_v11/124X_mcRun3_2024_realistic_v13

PR validation:

Code compiles - ran the tests in the master PR

Backport:

Backport of #38760, #39645 and #39823

cmsbuild · 2022-08-10T09:50:31Z

A new Pull Request was created by @francescobrivio for CMSSW_12_4_X.

It involves the following packages:

Configuration/AlCa (alca)
Configuration/PyReleaseValidation (pdmv, upgrade)
EventFilter/CSCRawToDigi (reconstruction)
Geometry/CMSCommonData (geometry, upgrade)
Geometry/HcalCommonData (geometry)
Geometry/TrackerCommonData (geometry)
Geometry/VeryForwardData (geometry)
SimG4CMS/Calo (simulation)
SimG4CMS/ShowerLibraryProducer (simulation)
SimG4Core/Configuration (simulation)

@malbouis, @civanch, @yuanchao, @jordan-martins, @bsunanda, @makortel, @bbilin, @saumyaphor4252, @ianna, @mdhildreth, @cmsbuild, @AdrianoDee, @srimanob, @Dr15Jones, @clacaputo, @kskovpen, @jpata, @tvami, @ChrisMisan, @francescobrivio can you please review it and eventually sign? Thanks.
@VourMa, @felicepantaleo, @kpedro88, @ghugo83, @Martin-Grunewald, @bsunanda, @grzanka, @trtomei, @slomeo, @venturia, @vargasa, @makortel, @JanFSchulte, @missirol, @simonepigazzini, @thomreis, @beaucero, @barvic, @fabferro, @rovere, @VinInn, @ptcox, @tocheng, @mmusich, @mtosi, @fabiocos this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

mmusich · 2022-08-10T11:29:33Z

Who'll be the primary user of this?

tvami · 2022-08-10T11:33:00Z

Who'll be the primary user of this?

HLT studies

Martin-Grunewald · 2022-08-10T11:42:32Z

Sorry, but I suppose this is for the 12_4 MC production, no?

tvami · 2022-08-10T11:45:29Z

Sorry, but I suppose this is for the 12_4 MC production, no?

Yes, this is used in the 12_4_X MC production. On the top of this change, we'll prove you @Martin-Grunewald et al special GTs that are including the JEC + PF for your autoCondHLT

tvami · 2022-08-10T11:46:25Z

@cmsbuild , please test

tvami · 2022-08-10T11:46:48Z

@cmsbuild , please abort

I forgot that we need input from PdmV

mmusich · 2022-08-10T11:46:57Z

Sorry, but I suppose this is for the 12_4 MC production, no?

Why does it need to be in release? Production does not care

mmusich · 2022-08-10T11:47:29Z

hold

until someone convinces me otherwise

cmsbuild · 2022-08-10T11:47:36Z

Pull request has been put on hold by @mmusich
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

tvami · 2022-08-10T11:51:12Z

Marco, this was agreed by PPD L1s, and ORP yesterday. Sal, Jordan, Kaori should comment...

mmusich · 2022-08-10T11:57:41Z

Please let's have a written justification. IIUC there are no minutes of the ORP meeting

fwyzard · 2022-08-10T12:23:48Z

As far as I kow the ORP meeting yesterday had been cancelled:

From: Qiang Li [email protected]
To: [email protected]
Subject: ORP[2022/08/09] cancelled

Dear all,

As the title says, we cancel this week's ORP meeting as many people are on
vacations.

If you have any urgent requests, please fell free write to let us know.
Otherwise we can also discuss relevant stuff at the joint Ops meeting this
week.

Kindly reminds you the next milestone
CMSSW_12_5_0_pre5: 2022/08/23 (last open pre-release)

Best,
Andrea, Qiang

rappoccio · 2022-08-10T13:08:22Z

@mmusich we didn't have an ORP yesterday but we discussed privately with @qliphy and this was agreed to add for posterity and ease of use.

mmusich · 2022-08-10T13:15:13Z

@rappoccio,

ease of use

use for what reason? Why are we changing the physics output of a closed release?

mandrenguyen · 2022-11-29T21:43:26Z

@bsunanda @mandrenguyen @civanch now that #39548 is merged, to be consistent this should be merged as well , would you please review and sign? thanks!

For reco, @clacaputo was looking at this one.

clacaputo · 2022-11-30T10:15:52Z

+reconstruction

bsunanda · 2022-11-30T14:18:27Z

+geometry

tvami · 2022-11-30T15:23:21Z

@bsunanda maybe you could also sign for simulation?

bsunanda · 2022-12-02T01:13:12Z

+1

bsunanda · 2022-12-02T01:14:58Z

@civanch @tvami I believe Vladimir or Mike (Hildredth) can sign for simulation

civanch · 2022-12-02T07:18:48Z

+1

cmsbuild · 2022-12-02T07:19:13Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_0_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

tvami · 2022-12-02T16:17:26Z

hi @cms-sw/orp-l2
I had this comment earlier

now that #39548 is merged, to be consistent this should be merged as well

in the last few IBs we had inconsistencies so it would be nice to merge this as soon as possible, thanks!

perrotta · 2022-12-02T18:46:03Z

please test
(9 days old tests... just check that there is no inconsistency with #39548 or any other PR merged since then: the plan is to merge as soon as the tests succeed, quite likely sometime tomorrow morning)

cmsbuild · 2022-12-03T03:41:23Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fd62c0/29433/summary.html
COMMIT: fcb9b1b
CMSSW: CMSSW_12_4_X_2022-11-29-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39019/29433/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 22728 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3709306
DQMHistoTests: Total failures: 50066
DQMHistoTests: Total nulls: 3
DQMHistoTests: Total successes: 3659215
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 5.189 KiB( 49 files compared)
DQMHistoSizes: changed ( 11834.0 ): 5.189 KiB SiStrip/MechanicalView
Checked 208 log files, 161 edm output root files, 50 DQM output files
TriggerResults: found differences in 7 / 49 workflows

perrotta · 2022-12-03T07:12:43Z

+1

missirol · 2022-12-03T15:43:15Z

Configuration/PyReleaseValidation/python/relval_steps.py

@@ -34,7 +34,7 @@
 # step1 gensim: for 2018 HI prod
 step1Up2018HiProdDefaults = merge ([{'--eventcontent':'RAWSIM'},step1Up2018HiDefaults])
 # step1 gensim: for 2021 HI prod
-step1Up2021HiProdDefaults = merge ([{'--conditions':'auto:phase1_2022_realistic_hi','--era':'Run3_pp_on_PbPb','--beamspot':'Run3RoundOptics25ns13TeVLowSigmaZ','--eventcontent':'RAWSIM','--geometry':'DB:Extended'},step1Up2018HiDefaults])
+step1Up2021HiProdDefaults = merge ([{'--conditions':'auto:phase1_2022_realistic_hi','--era':'Run3_pp_on_PbPb','--beamspot':'Realistic25ns13p6TeVEarly2022Collision','--eventcontent':'RAWSIM','--geometry':'DB:Extended'},step1Up2018HiDefaults])


@cms-sw/alca-l2

I'm having trouble understanding why this change is needed (and two other ones below).

After this PR, the MC GTs for 2022_hi, 2023, and 2024 [1] still use the 'old' beamspot tags, i.e. Run3RoundOptics25ns13TeVLowSigmaZ (iiuc, this is because the PU samples of the corresponding RelVals were produced with the 'old' beamspot). At the same time, this PR changes the beamspot parameters used in the GEN-SIM step of those workflows to the 'new' one, i.e. Realistic25ns13p6TeVEarly2022Collision.

Taking wf 12434.0 as an example (pp 2023, no PU), step-1 is now as in [2], and this plot from the PR tests looks odd.

Am I missing something?

[1]

phase1_2022_realistic_hi: https://cms-conddb.cern.ch/cmsDbBrowser/list/Prod/gts/124X_mcRun3_2022_realistic_HI_v12

phase1_2023_realistic: https://cms-conddb.cern.ch/cmsDbBrowser/list/Prod/gts/124X_mcRun3_2023_realistic_v13

phase1_2024_realistic: https://cms-conddb.cern.ch/cmsDbBrowser/list/Prod/gts/124X_mcRun3_2024_realistic_v13

[2]

cmsDriver.py TTbar_14TeV_TuneCP5_cfi -s GEN,SIM -n 10 \ --conditions auto:phase1_2023_realistic --beamspot Realistic25ns13p6TeVEarly2022Collision \ --datatier GEN-SIM --eventcontent FEVTDEBUG --geometry DB:Extended --era Run3 --relval 9000,100

This looks like a mistake.
In the master version of this PR: #38760 the >= 2022 scenarios got updated because the corresponding PR updating the PU libraries has changed the later Run-3 years as well (see #39041) but in this case this is not appropriate.
I suggest this bug-fix PR #40238 to resolve the issue.
Alternatively we'd need updates to GlobalTags as well as PU libraries.

missirol · 2022-12-03T15:43:35Z

Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py

@@ -2002,15 +2002,15 @@ def condition(self, fragment, stepList, key, hasHarvest):
        'GT' : 'auto:phase1_2023_realistic',
        'HLTmenu': '@relval2022',
        'Era' : 'Run3',
-        'BeamSpot': 'Run3RoundOptics25ns13TeVLowSigmaZ',
+        'BeamSpot': 'Realistic25ns13p6TeVEarly2022Collision',


See first comment.

missirol · 2022-12-03T15:43:48Z

Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py

        'ScenToRun' : ['GenSim','Digi','RecoNano','HARVESTNano','ALCA'],
    },
    '2024' : {
        'Geom' : 'DB:Extended',
        'GT' : 'auto:phase1_2024_realistic',
        'HLTmenu': '@relval2022',
        'Era' : 'Run3',
-        'BeamSpot': 'Run3RoundOptics25ns13TeVLowSigmaZ',
+        'BeamSpot': 'Realistic25ns13p6TeVEarly2022Collision',


See first comment.

cmsbuild added this to the CMSSW_12_4_X milestone Aug 10, 2022

cmsbuild added alca-pending geometry-pending orp-pending pdmv-pending pending-signatures reconstruction-pending simulation-pending tests-pending upgrade-pending labels Aug 10, 2022

cmsbuild added tests-started and removed tests-pending labels Aug 10, 2022

cmsbuild added tests-pending and removed tests-started labels Aug 10, 2022

cmsbuild added the hold label Aug 10, 2022

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Nov 30, 2022

cmsbuild added geometry-approved and removed geometry-pending labels Nov 30, 2022

cmsbuild added fully-signed simulation-approved and removed simulation-pending pending-signatures labels Dec 2, 2022

cmsbuild added tests-started and removed tests-approved labels Dec 2, 2022

cmsbuild added tests-approved and removed tests-started labels Dec 3, 2022

cmsbuild added orp-approved and removed orp-pending labels Dec 3, 2022

cmsbuild merged commit c7f7797 into cms-sw:CMSSW_12_4_X Dec 3, 2022

missirol reviewed Dec 3, 2022

View reviewed changes

missirol mentioned this pull request Dec 3, 2022

Update of HLT-specific MC GT for frozen "V1.4" HLT menu to "postEE" scenario [12_4_X only] #40233

Merged

mmusich mentioned this pull request Dec 5, 2022

revert change of vertex smearing for 2022 HI , 2023 and 2024 relvals #40238

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[12_4_X] Update Run3 MC GTs + swap update Vertex smearing + update offline Run3 GTs #39019

[12_4_X] Update Run3 MC GTs + swap update Vertex smearing + update offline Run3 GTs #39019

francescobrivio commented Aug 10, 2022 •

edited

Loading

cmsbuild commented Aug 10, 2022

mmusich commented Aug 10, 2022

tvami commented Aug 10, 2022

Martin-Grunewald commented Aug 10, 2022

tvami commented Aug 10, 2022 •

edited

Loading

tvami commented Aug 10, 2022

tvami commented Aug 10, 2022 •

edited

Loading

mmusich commented Aug 10, 2022

mmusich commented Aug 10, 2022

cmsbuild commented Aug 10, 2022

tvami commented Aug 10, 2022

mmusich commented Aug 10, 2022

fwyzard commented Aug 10, 2022

rappoccio commented Aug 10, 2022

mmusich commented Aug 10, 2022

mandrenguyen commented Nov 29, 2022

clacaputo commented Nov 30, 2022

bsunanda commented Nov 30, 2022

tvami commented Nov 30, 2022

bsunanda commented Dec 2, 2022

bsunanda commented Dec 2, 2022

civanch commented Dec 2, 2022

cmsbuild commented Dec 2, 2022

tvami commented Dec 2, 2022

perrotta commented Dec 2, 2022

cmsbuild commented Dec 3, 2022

perrotta commented Dec 3, 2022

missirol Dec 3, 2022

mmusich Dec 5, 2022 •

edited

Loading

missirol Dec 3, 2022

missirol Dec 3, 2022

[12_4_X] Update Run3 MC GTs + swap update Vertex smearing + update offline Run3 GTs #39019

[12_4_X] Update Run3 MC GTs + swap update Vertex smearing + update offline Run3 GTs #39019

Conversation

francescobrivio commented Aug 10, 2022 • edited Loading

PR description:

PR validation:

Backport:

cmsbuild commented Aug 10, 2022

mmusich commented Aug 10, 2022

tvami commented Aug 10, 2022

Martin-Grunewald commented Aug 10, 2022

tvami commented Aug 10, 2022 • edited Loading

tvami commented Aug 10, 2022

tvami commented Aug 10, 2022 • edited Loading

mmusich commented Aug 10, 2022

mmusich commented Aug 10, 2022

cmsbuild commented Aug 10, 2022

tvami commented Aug 10, 2022

mmusich commented Aug 10, 2022

fwyzard commented Aug 10, 2022

rappoccio commented Aug 10, 2022

mmusich commented Aug 10, 2022

mandrenguyen commented Nov 29, 2022

clacaputo commented Nov 30, 2022

bsunanda commented Nov 30, 2022

tvami commented Nov 30, 2022

bsunanda commented Dec 2, 2022

bsunanda commented Dec 2, 2022

civanch commented Dec 2, 2022

cmsbuild commented Dec 2, 2022

tvami commented Dec 2, 2022

perrotta commented Dec 2, 2022

cmsbuild commented Dec 3, 2022

Comparison Summary

perrotta commented Dec 3, 2022

missirol Dec 3, 2022

Choose a reason for hiding this comment

mmusich Dec 5, 2022 • edited Loading

Choose a reason for hiding this comment

missirol Dec 3, 2022

Choose a reason for hiding this comment

missirol Dec 3, 2022

Choose a reason for hiding this comment

francescobrivio commented Aug 10, 2022 •

edited

Loading

tvami commented Aug 10, 2022 •

edited

Loading

tvami commented Aug 10, 2022 •

edited

Loading

mmusich Dec 5, 2022 •

edited

Loading