DQM unit tests massive failures in CMSSW_12_6_X_2022-08-30-2300

[mmusich]

Starting from CMSSW_12_6_X_2022-08-30-2300 we observe massive DQM unit test failures.
See log: https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc10/CMSSW_12_6_X_2022-08-30-2300/unitTestLogs/DQM/Integration#/

Trying to compile in local checkout:

$ python3 beamhlt_dqm_sourceclient-live_cfg.py unitTest=True
Querying DAS for files...
the query is file run=355380 dataset=/ExpressPhysics/Run2022B-Express-v1/FEVT lumi=19
DAS succeeded after 1 attempts 0
found files:  ['/store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/b8a57fc4-5656-42b4-9b7b-2e647baf65e8.root']
edmFileUtil --catalog file:/cvmfs/cms-ib.cern.ch/SITECONF/local/PhEDEx/storage.xml?protocol=xrootd --events /store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/b8a57fc4-5656-42b4-9b7b-2e647baf65e8.root | tail -n +9 | head -n -5 | awk '{ print $3 }'
Traceback (most recent call last):
  File "/tmp/musich/CMSSW_12_6_X_2022-08-30-2300/src/DQM/Integration/python/clients/beamhlt_dqm_sourceclient-live_cfg.py", line 42, in <module>
    process.load("DQM.Integration.config.unittestinputsource_cfi")
  File "/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_6_X_2022-08-30-2300/src/FWCore/ParameterSet/python/Config.py", line 744, in load
    module = __import__(moduleName)
  File "/tmp/musich/CMSSW_12_6_X_2022-08-30-2300/src/DQM/Integration/python/config/unittestinputsource_cfi.py", line 113, in <module>
    eventsToProcess.append("%s:%s:%s-%s:%s:%s" % (options.runNumber, ls, events[0], options.runNumber, ls, events[-1]))
IndexError: list index out of range

[mmusich]

assign dqm

[cmsbuild]

New categories assigned: dqm

@jfernan2,@ahmad3213,@micsucmed,@rvenditti,@emanueleusai,@pmandrik you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

A new Issue was created by @mmusich Marco Musich.

@Dr15Jones, @perrotta, @dpiparo, @rappoccio, @makortel, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[smuzaffar]

I think this is related to #37278 change. dqm tests are explicitly passing --catalog file:/cvmfs/cms-ib.cern.ch/SITECONF/local/PhEDEx/storage.xml?protocol=xrootd . I think simple fix is to just drop the --catalog option at https://github.com/cms-sw/cmssw/blob/master/DQM/Integration/python/config/unittestinputsource_cfi.py#L105

[nhduongvn]

Yes, if the SITECONFIG_PATH is set correctly, --catalog file:/cvmfs/cms-ib.cern.ch/SITECONF/local/PhEDEx/storage.xml?protocol=xrootd should be dropped.

[smuzaffar]

yes SITECONFIG_PATH is set properly for PR and IB tests

[mmusich]

I confirm that by dropping the --catalog option one of the unit tests runs fine.
I've opened #39266

[makortel]

Thanks @mmusich!