Fix ZMu skimming code for 2024 PbPb data taking

[flodamas]

PR description:

Urgent fix of the ZMu skimming code which is responsible for the crashes observed during the Replay of the HIPhysicsRawPrime data with CMSSW_14_1_4_patch4 as documented in this post.

PR validation:

Tested with runTheMatrix.py -l 142.0 -n -e modified to run the PbPbZMu skimming code. No crash observed and output produced successfully.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Will be backported urgently in 14_1_X for PbPb data taking.

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46584/42478

[cmsbuild]

A new Pull Request was created by @flodamas for master.

It involves the following packages:

• Configuration/Skimming (pdmv)

@AdrianoDee, @cmsbuild, @DickyChant, @miquork can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @fabiocos, @missirol, @youyingli this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[AdrianoDee]

please test workflow 142.0

[AdrianoDee]

urgent
(see #46458 (review))

[mmusich]

please test workflow 142.0
[...]

Tested with runTheMatrix.py -l 142.0 -n -e modified to run the PbPbZMu skimming code. No crash observed and output produced successfully.

is there a relval test that actually tests these skims?

[mmusich]

type bug-fix

[AdrianoDee]

is there a relval test that actually tests these skims?

I had misinterpreted the modified to run the PbPbZMu skimming code, my bad. I think in general no, but let me tag @youyingli that should know better than me. To be honest I don't know why we don't have RelVals to test any skim and if this is something just missing or practically not doable. I suspect the former.

[mmusich]

To be honest I don't know why we don't have RelVals to test any skim and if this is something just missing or practically not doable. I suspect the former.

wf 1000.0 runs skims for pp:

cmssw/Configuration/PyReleaseValidation/python/relval_production.py

Line 12 in 7d3d80e

workflows[1000] = [ '',['RunMinBias2011A','TIER0','SKIMD','HARVESTDfst2','ALCASPLIT']]

cmssw/Configuration/PyReleaseValidation/python/relval_steps.py

Line 4160 in 7d3d80e

steps['SKIMD']={'-s':'SKIM:all',

can we have something for PbPb too?

[mmusich]

FWIW the issue reported at #46458 (review) is reproducible with:

#!/bin/bash -ex                                                                                                                                                                                            

scram p CMSSW CMSSW_14_1_4_patch4
cd CMSSW_14_1_4_patch4/src/
eval `scramv1 runtime -sh`
git cms-addpkg Configuration/DataProcessing
python3 Configuration/DataProcessing/test/RunPromptReco.py \
        --scenario=ppEra_Run3_pp_on_PbPb_approxSiStripClusters_2024 \
        --reco \
        --global-tag 141X_dataRun3_Prompt_v3 \
        --lfn=root://eoscms.cern.ch//eos/cms/tier0/store/backfill/1/hidata/Tier0_HIREPLAY_2024/HIPhysicsRawPrime15/RAW/v1194758/000/374/951/00000/cd04c2a1-c933-479b-9fbb-1db653fb6236.root \
        --PhysicsSkims=@HIPhysicsRawPrime15 \
        --dqmSeq=@commonSiStripZeroBias+@ecal+@hcal+@muon+@jetmet+@egamma
edmConfigDump RunPromptRecoCfg.py > dump.py
echo 'process.source.skipEvents=cms.untracked.uint32(407)' >> dump.py
cmsRun -e dump.py

if I modify the recipe to cherry-pick this commit with:

#!/bin/bash -ex

scram p CMSSW CMSSW_14_1_4_patch4
cd CMSSW_14_1_4_patch4/src/
eval `scramv1 runtime -sh`
git cms-addpkg Configuration/DataProcessing Configuration/Skimming
git cms-init; git remote add flodamas [email protected]:flodamas/cmssw.git; git fetch flodamas
git cherry-pick 3cfdab7cd9549e5b76fa68aadb97652ca4aa8a6f
scram b -j 20
eval `scramv1 runtime -sh`
python3 Configuration/DataProcessing/test/RunPromptReco.py \
	--scenario=ppEra_Run3_pp_on_PbPb_approxSiStripClusters_2024 \
	--reco \
	--global-tag 141X_dataRun3_Prompt_v3 \
	--lfn=root://eoscms.cern.ch//eos/cms/tier0/store/backfill/1/hidata/Tier0_HIREPLAY_2024/HIPhysicsRawPrime15/RAW/v1194758/000/374/951/00000/cd04c2a1-c933-479b-9fbb-1db653fb6236.root \
	--PhysicsSkims=@HIPhysicsRawPrime15 \
	--dqmSeq=@commonSiStripZeroBias+@ecal+@hcal+@muon+@jetmet+@egamma
edmConfigDump RunPromptRecoCfg.py > dump.py
echo 'process.source.skipEvents=cms.untracked.uint32(407)' >> dump.py
cmsRun -e dump.py

the process runs fine.

+1 from ORM.

[mandrenguyen]

There is code to test the PbPb skims, but not yet with this years trigger menu. For this particular skim, the HLT path might have been the same, but for others the path was new.
Anyway, the point is of course well taken. As soon as we get some data we'll set up a workflow with the skims so we don't run into this problem next time (assuming we don't again skim on new triggers. )

[mandrenguyen]

There is a relval that tests PbPb skims, but not yet with this years trigger menu. For this particular skim, the HLT path might have been the same, but for others the path was new. Anyway, the point is of course well taken. As soon as we get some data we'll set up a workflow with the skims so we don't run into this problem next time (assuming we don't again skim on new triggers. )

[mmusich]

Anyway, the point is of course well taken. As soon as we get some data we'll set up a workflow with the skims so we don't run into this problem next time (assuming we don't again skim on new triggers. )

I opened #46586 - feel free to follow-up there.

[mandrenguyen]

+1

[cmsbuild]

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-274cbf/42551/summary.html
COMMIT: 80f761e
CMSSW: CMSSW_14_2_X_2024-11-01-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46584/42551/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 3 lines from the logs
• Reco comparison results: 12 differences found in the comparisons
• DQMHistoTests: Total files compared: 47
• DQMHistoTests: Total histograms compared: 3632331
• DQMHistoTests: Total failures: 434
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3631876
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.008 KiB( 46 files compared)
• DQMHistoSizes: changed ( 140.56 ): 0.004 KiB MessageLogger/Errors
• DQMHistoSizes: changed ( 140.56 ): 0.004 KiB MessageLogger/Warnings
• Checked 205 log files, 174 edm output root files, 47 DQM output files
• TriggerResults: no differences found

[AdrianoDee]

+pdmv

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.