Update nano relval matrix

[hqucms]

PR description:

Updated NANO relval matrix:

• updated workflows to 140X
• added Run2UL reMINI + reNANO workflows
• moved NANOGEN from relval_generator to relval_nano

Also added the support for NANOGEN with the autoNANO syntax:

• from GEN inputs: -s NANO:@GEN --eventcontent NANOAOD
• from MiniAOD: -s NANO:@GENFromMini --eventcontent NANOAOD

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Before submitting your pull requests, make sure you followed this checklist:

• verify that the PR is really intended for the chosen branch
• verify that changes follow CMS Naming, Coding, And Style Rules
• verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

[cmsbuild]

cms-bot internal usage

[hqucms]

enable nano

[hqucms]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45419/40864

• There are other open Pull requests which might conflict with changes you have proposed:
  • File Configuration/PyReleaseValidation/python/relval_nano.py modified in PR(s): Add ScoutingMuonNoVtx and ScoutingMuonVtx in Run3Scouting Nano starting from 2024 data taking #44970
  • File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): 2024 Data RelVals and InputInfo Events Skimming (actually) Working #45055, Update heavy-ion eras for 2024 + add MC RelVal workflows #45307
  • File PhysicsTools/NanoAOD/python/nanogen_cff.py modified in PR(s): Fix NanoGen with Rivet4 #45411

[cmsbuild]

A new Pull Request was created by @hqucms for master.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv, upgrade)
• PhysicsTools/NanoAOD (xpog)

@AdrianoDee, @ftorrresd, @hqucms, @kskovpen, @miquork, @srimanob, @subirsarkar, @sunilUIET, @vlimant can you please review it and eventually sign? Thanks.
@AnnikaStein, @Martin-Grunewald, @fabiocos, @gpetruc, @makortel, @missirol, @slomeo this is something you requested to watch as well.
@antoniovilela, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

-1

Failed Tests: RelVals RelVals-INPUT RelVals-NANO
Size: This PR adds an extra 68KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-565cc2/40322/summary.html
COMMIT: 3d0536d
CMSSW: CMSSW_14_1_X_2024-07-09-2300/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45419/40322/install.sh to create a dev area with all the needed externals and cmssw changes.

• DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals

ValueError: Undefined workflows: 2500.4

RelVals-INPUT

• 2500.2372500.237_ScoutingNANOdata140Xrun3/step2_ScoutingNANOdata140Xrun3.log

RelVals-NANO

• 2500.2372500.237_ScoutingNANOdata140Xrun3/step2_ScoutingNANOdata140Xrun3.log
• 2500.9112500.911_DYToLL_M-50_13TeV_pythia8/step2_DYToLL_M-50_13TeV_pythia8.log
• 2500.9132500.913_TTbar_Pow_LHE_13TeV/step3_TTbar_Pow_LHE_13TeV.log

Expand to see more relval errors ...

• 2500.912

[vlimant]

N.B : should we update ConfigBulder to hard remove NANOGEN since you include the support here ?

[hqucms]

N.B : should we update ConfigBulder to hard remove NANOGEN since you include the support here ?

Done in e193249 now.

[hqucms]

@cms-sw/pdmv-l2 @cms-sw/upgrade-l2 @cms-sw/operations-l2
Could you please review and sign? We would like to get this in asap. Thanks!

[srimanob]

Hi @hqucms
As you change the way to call NanoGEN, would it be useful to add a workflow on it? I see you only remove the workflow of NanoGEN, but no new workflow on the new way. Thx.

[hqucms]

@srimanob The NanoGEN workflows are moved from relval_generator to relval_nano now so we can test them with all other nano workflows together.

[srimanob]

@srimanob The NanoGEN workflows are moved from relval_generator to relval_nano now so we can test them with all other nano workflows together.

Is it moved in this PR? Thx.

[hqucms]

@srimanob The NanoGEN workflows are moved from relval_generator to relval_nano now so we can test them with all other nano workflows together.

Is it moved in this PR? Thx.

Yes - in these lines: https://github.com/cms-sw/cmssw/pull/45419/files#diff-aaca4437268e5c684a1042abc90e2cdd651a08b8c35f03cdd32060e8918f41e5R359-R367

[srimanob]

+Upgrade

[AdrianoDee]

+pdmv

[vlimant]

@cms-sw/operations-l2 please sign

[rappoccio]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

[hqucms]

@cms-sw/core-l2 I noticed that the updated wf 2500.245 is failing because the input file [1] has disappeared from disk (now only on tape).

I would expect it to be automatically cached by ibeos but it seems no? How is the ibeos caching supposed to work for new input files?

[1] /store/data/Run2024F/TestEnablesEcalHcal/RAW/Express-v1/000/383/173/00000/1088fbe0-c256-4752-992c-0e535b804bef.root

[makortel]

(@smuzaffar please correct me) I believe the ibeos caching works automatically for files that are on disk. I don't think we have any automated way to recall a file from tape (but I believe we could shovel the file to the cache by hand if that's what is needed).

In any case someone has to recall the file from tape. If the recall is done such that the file is visible e.g. via the xrootd redirectors (e.g. for a week or two), I'd expect the automatic caching to pick the file from there.

[smuzaffar]

/store/data/Run2024F/TestEnablesEcalHcal/RAW/Express-v1/000/383/173/00000/1088fbe0-c256-4752-992c-0e535b804bef.root is now in ibeos cache

[hqucms]

Thanks a lot @smuzaffar @makortel !