XML and FedId configuration update for PPS #37147

ChrisMisan · 2022-03-04T19:33:12Z

PR description:

This PR updates the xml mappings and FedId configuration for PPS run3. Backward compatibility is sustained.

PR validation:

This PR was validated with relvals 136.793, 136.874 (which run PPS reconstruction). New run3 xml's were parsed without issue. We lack the run3 data to conduct more sophisticated tests. For the run2 data, there should be no changes as to the results.

if this PR is a backport please specify the original PR and why you need to backport that PR:

no

cmsbuild · 2022-03-04T19:40:21Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37147/28689

This PR adds an extra 24KB to repository

cmsbuild · 2022-03-04T19:40:41Z

A new Pull Request was created by @ChrisMisan (Christopher) for master.

It involves the following packages:

CondFormats/PPSObjects (alca)
EventFilter/CTPPSRawToDigi (reconstruction)

@malbouis, @yuanchao, @clacaputo, @cmsbuild, @slava77, @jpata, @tvami, @francescobrivio can you please review it and eventually sign? Thanks.
@fabferro, @tocheng, @Martin-Grunewald, @missirol, @grzanka, @mmusich, @seemasharmafnal this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

tvami · 2022-03-04T23:33:01Z

Hi @ChrisMisan has it been considered to move these into a DB object? Having this IOV structure that you have in the config file is exactly what the multi-IOV tags are for.

tvami · 2022-03-04T23:33:30Z

test parameters:

workflows = 136.793, 136.874

tvami · 2022-03-04T23:34:38Z

@cmsbuild , please test

cmsbuild · 2022-03-05T01:48:09Z

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7b2143/22862/summary.html
COMMIT: fa44abe
CMSSW: CMSSW_12_3_X_2022-03-04-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37147/22862/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

138.4

----- Begin Fatal Exception 05-Mar-2022 01:46:54 CET-----------------------
An exception of category 'TotemDAQMappingESSourceXML' occurred while
   [0] Processing  Event run: 346512 lumi: 250 event: 243042266 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Prefetching for module DQMMessageLogger/'DQMMessageLogger'
   [3] Prefetching for module LogErrorHarvester/'logErrorHarvester'
   [4] Prefetching for module CTPPSDiamondLocalTrackFitter/'ctppsDiamondLocalTracks'
   [5] Prefetching for module CTPPSDiamondRecHitProducer/'ctppsDiamondRecHits'
   [6] Prefetching for module TotemVFATRawToDigi/'ctppsDiamondRawToDigi'
   [7] Calling method for EventSetup module TotemDAQMappingESSourceXML/'totemDAQMappingESSourceXML_TimingDiamond'
Exception Message:
Node station not allowed within station block.
----- End Fatal Exception -------------------------------------------------

140.56

----- Begin Fatal Exception 05-Mar-2022 01:50:39 CET-----------------------
An exception of category 'TotemDAQMappingESSourceXML::setIntervalFor' occurred while
   [0] Calling EventProcessor::runToCompletion (which does almost everything after beginJob and before endJob)
Exception Message:
No configuration for event run: 326479 lumi: 0 event: 0
----- End Fatal Exception -------------------------------------------------

139.001

----- Begin Fatal Exception 05-Mar-2022 01:53:52 CET-----------------------
An exception of category 'TotemDAQMappingESSourceXML' occurred while
   [0] Processing  Event run: 346512 lumi: 250 event: 243042266 stream: 0
   [1] Running path 'dqmoffline_6_step'
   [2] Prefetching for module LogMessageMonitor/'TrackFinderLogMessageMonMB'
   [3] Prefetching for module LogErrorHarvester/'logErrorHarvester'
   [4] Prefetching for module CTPPSDiamondLocalTrackFitter/'ctppsDiamondLocalTracks'
   [5] Prefetching for module CTPPSDiamondRecHitProducer/'ctppsDiamondRecHits'
   [6] Prefetching for module TotemVFATRawToDigi/'ctppsDiamondRawToDigi'
   [7] Calling method for EventSetup module TotemDAQMappingESSourceXML/'totemDAQMappingESSourceXML_TimingDiamond'
Exception Message:
Node station not allowed within station block.
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

140.56140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log
138.4138.4_PromptCollisions+RunMinimumBias2021+ALCARECOPROMPTR3+HARVESTDPROMPTR3/step2_PromptCollisions+RunMinimumBias2021+ALCARECOPROMPTR3+HARVESTDPROMPTR3.log
140.57140.57_RunHI2018Reduced+RunHI2018Reduced+RECOHID18+HARVESTDHI18/step2_RunHI2018Reduced+RunHI2018Reduced+RECOHID18+HARVESTDHI18.log

AddOn Tests

hlt_data_GRun

----- Begin Fatal Exception 05-Mar-2022 01:43:13 CET-----------------------
An exception of category 'TotemDAQMappingESSourceXML' occurred while
   [0] Processing  Event run: 346304 lumi: 378 event: 367545818 stream: 1
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module TotemVFATRawToDigi/'ctppsDiamondRawToDigi'
   [4] Calling method for EventSetup module TotemDAQMappingESSourceXML/'totemDAQMappingESSourceXML_TimingDiamond'
Exception Message:
Node station not allowed within station block.
----- End Fatal Exception -------------------------------------------------

hlt_data_HIon

----- Begin Fatal Exception 05-Mar-2022 01:42:16 CET-----------------------
An exception of category 'TotemDAQMappingESSourceXML' occurred while
   [0] Processing  Event run: 346304 lumi: 378 event: 367545818 stream: 3
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module TotemVFATRawToDigi/'ctppsDiamondRawToDigi'
   [4] Calling method for EventSetup module TotemDAQMappingESSourceXML/'totemDAQMappingESSourceXML_TimingDiamond'
Exception Message:
Node station not allowed within station block.
----- End Fatal Exception -------------------------------------------------

hlt_data_PIon

----- Begin Fatal Exception 05-Mar-2022 01:42:22 CET-----------------------
An exception of category 'TotemDAQMappingESSourceXML' occurred while
   [0] Processing  Event run: 346304 lumi: 378 event: 367545818 stream: 0
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module TotemVFATRawToDigi/'ctppsDiamondRawToDigi'
   [4] Calling method for EventSetup module TotemDAQMappingESSourceXML/'totemDAQMappingESSourceXML_TimingDiamond'
Exception Message:
Node station not allowed within station block.
----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

hlt_data_PRef

ChrisMisan · 2022-03-05T03:02:08Z

Both issues with relvals should be resolved shortly:
Node station not allowed within station block. is due to minor error in one of the xmls, it was parsed fine but the hierarchy is missing one tag.

No configuration for event run: 326479 lumi: 0 event: 0 it's true - it's outside our run2 interval, maybe interval is actually different then what we have in the config

@tvami To answer your question, I know this has been discussed, I guess @grzanka and @vavati can better state the reasons for the current configuration.

grzanka · 2022-03-05T08:09:20Z

Hi @ChrisMisan has it been considered to move these into a DB object? Having this IOV structure that you have in the config file is exactly what the multi-IOV tags are for.

Hi @tvami . We can consider moving into a DB objects, I agree it is a cleaner solution. But again we are under time pressure before Run3. Migrating to DB would require some significant changes in our RAW2DATA modules, proper testing etc...

cmsbuild · 2022-03-05T10:23:40Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37147/28692

This PR adds an extra 24KB to repository

cmsbuild · 2022-03-08T22:01:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37147/28750

This PR adds an extra 28KB to repository

cmsbuild · 2022-03-08T22:01:44Z

Pull request #37147 was updated. @malbouis, @yuanchao, @clacaputo, @cmsbuild, @slava77, @jpata, @tvami, @francescobrivio can you please check and sign again.

tvami · 2022-03-08T22:02:37Z

@cmsbuild , please test

cmsbuild · 2022-03-09T01:23:14Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7b2143/22953/summary.html
COMMIT: 5888760
CMSSW: CMSSW_12_3_X_2022-03-08-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37147/22953/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3985573
DQMHistoTests: Total failures: 10
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3985541
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0 KiB( 0 files compared)
Checked 0 log files, 0 edm output root files, 49 DQM output files

jpata · 2022-03-09T13:50:11Z

+reconstruction

new PPS fedIds for Run3, corresponding reco changes
no changes for Run2 otherwise

tvami · 2022-03-09T13:57:03Z

+alca

with the remark that these object might be moved to DB in the future

cmsbuild · 2022-03-09T13:57:28Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-03-09T14:18:38Z

+1

Christopher added 3 commits March 3, 2022 23:25

Add PPS 2022 xml mapping

b764df6

Add PPS run3 fedids

a5d8a3e

Xml files update

fa44abe

cmsbuild added this to the CMSSW_12_3_X milestone Mar 4, 2022

cmsbuild added alca-pending code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Mar 4, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 4, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 4, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 5, 2022

Fix diamond xml structure and validity range for run2

3da0471

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Mar 5, 2022

fix config syntax

5888760

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Mar 8, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 8, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 8, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 9, 2022

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 9, 2022

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Mar 9, 2022

cmsbuild added orp-approved and removed orp-pending labels Mar 9, 2022

cmsbuild merged commit 8ea6464 into cms-sw:master Mar 9, 2022

This was referenced Mar 9, 2022

[G4 VECGEOM] Geant4 update to 11.0.1 patch cms-sw/cmsdist#7678

Merged

[G4] Geant4 update to 11.0.1 patch cms-sw/cmsdist#7679

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

XML and FedId configuration update for PPS #37147

XML and FedId configuration update for PPS #37147

ChrisMisan commented Mar 4, 2022

cmsbuild commented Mar 4, 2022

cmsbuild commented Mar 4, 2022

tvami commented Mar 4, 2022 •

edited

Loading

tvami commented Mar 4, 2022

tvami commented Mar 4, 2022

cmsbuild commented Mar 5, 2022

ChrisMisan commented Mar 5, 2022 •

edited

Loading

grzanka commented Mar 5, 2022

cmsbuild commented Mar 5, 2022

cmsbuild commented Mar 8, 2022

cmsbuild commented Mar 8, 2022

tvami commented Mar 8, 2022

cmsbuild commented Mar 9, 2022

jpata commented Mar 9, 2022 •

edited

Loading

tvami commented Mar 9, 2022

cmsbuild commented Mar 9, 2022

perrotta commented Mar 9, 2022

XML and FedId configuration update for PPS #37147

XML and FedId configuration update for PPS #37147

Conversation

ChrisMisan commented Mar 4, 2022

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Mar 4, 2022

cmsbuild commented Mar 4, 2022

tvami commented Mar 4, 2022 • edited Loading

tvami commented Mar 4, 2022

tvami commented Mar 4, 2022

cmsbuild commented Mar 5, 2022

RelVals

RelVals-INPUT

AddOn Tests

ChrisMisan commented Mar 5, 2022 • edited Loading

grzanka commented Mar 5, 2022

cmsbuild commented Mar 5, 2022

cmsbuild commented Mar 8, 2022

cmsbuild commented Mar 8, 2022

tvami commented Mar 8, 2022

cmsbuild commented Mar 9, 2022

Comparison Summary

jpata commented Mar 9, 2022 • edited Loading

tvami commented Mar 9, 2022

cmsbuild commented Mar 9, 2022

perrotta commented Mar 9, 2022

tvami commented Mar 4, 2022 •

edited

Loading

ChrisMisan commented Mar 5, 2022 •

edited

Loading

jpata commented Mar 9, 2022 •

edited

Loading