Update Run3 GEM to use the eMap from the DB #29194

watson-ij · 2020-03-12T06:07:05Z

PR description:

Fix for GEM crashing in Run3 data due to the emap not being read from the database. This patch instructs run3 to also use dbEmaps, as was done in slice test (its a minimal change as currently only slice test and run 3 emaps are in the db).

PR validation:

I tested on one of the files [1] and it no longer crashes. There are still warnings of the form

%MSG-e BadDigiInput: GEMRecHitProducer:gemRecHits 11-Mar-2020 11:39:34
CET Run: 335670 Event: 4434
Failed to find GEMEtaPartition for ID Re -1 Ri 1 St 1 La 2 Ch 25 Ro 8
%MSG

which is due to currently only the slice test geometry being in the DB, which is being worked on.

I haven't checked on MC though.

[1] cmsDriver.py GEM --step=RAW2DIGI,L1Reco,RECO --filein=/store/data/Commissioning2020/MinimumBias/RAW/v1/000/335/670/00000/5F3ECF4A-0637-844F-AE67-6BB424F0BF80.root --data --conditions 110X_dataRun3_Prompt_v3 --no_exec --era Run3

@jshlee @hyunyong

cmsbuild · 2020-03-12T07:24:14Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-03-12T07:43:35Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29194/14168

This PR adds an extra 12KB to repository

perrotta · 2020-03-12T11:16:53Z

please test

cmsbuild · 2020-03-12T11:17:15Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5155/console Started: 2020/03/12 12:20

cmsbuild · 2020-03-12T11:17:16Z

A new Pull Request was created by @watson-ij (Ian J. Watson) for master.

It involves the following packages:

EventFilter/GEMRawToDigi

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@jshlee, @Martin-Grunewald this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2020-03-12T12:33:00Z

+1
Tested at: 0400cce
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a9ce50/5155/summary.html
CMSSW: CMSSW_11_1_X_2020-03-11-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-03-12T12:33:03Z

Comparison job queued.

cmsbuild · 2020-03-12T14:04:33Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a9ce50/5155/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 343 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2680593
DQMHistoTests: Total failures: 620
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2679654
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

slava77 · 2020-03-12T16:24:53Z

Reco comparison results: 343 differences found in the comparisons

it looks like run3 and phase2 MC is broken for GEMs using the database for emap.
Do we have a good payload to replace at least the Run3 version for the MC global tag (IIUC it's 111X_mcRun3_2021_realistic_v1 now).
For phase2 I can imagine that we still need a "hardcoded" emap.

watson-ij · 2020-03-13T07:01:47Z

We have a candidate GT for GEM eMap in MC. If I run the ttbar 2021 workflow with the auto GT and then with the following:
111X_mcRun3_2021_realistic_Candidate_2020_03_13_03_56_33
I see that the auto GT doesn't have GEM rechits (as seen in the comparison plots), but they're there for the candidate GT.

slava77 · 2020-03-13T12:27:48Z

assign alca

I'm adding alca to keep track of the GT update.
I prefer that we have the GEM emap at least for the Run3 workflows.

@watson-ij
is it practical to also make a GT payload for emap for phase2?
If not, please add phase2_GEM modifier and apply it in
https://github.com/cms-sw/cmssw/blob/master/Configuration/Eras/python/Era_Phase2_cff.py
while removing in copyAndExclude the run3_GEM currently inherited from Run3.

cmsbuild · 2020-03-13T12:28:09Z

New categories assigned: alca

@christopheralanwest,@franzoni,@tlampen,@pohsun,@tocheng you have been requested to review this Pull request/Issue and eventually sign? Thanks

perrotta · 2020-03-18T08:42:33Z

@cmsbuild please test

cmsbuild · 2020-03-18T08:42:57Z

The tests are being triggered in jenkins.
Tested with other pull request(s) #29210
Test Parameters:

MATRIX_EXTRAS = 159,159.1,159.2,159.3
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5242/console Started: 2020/03/18 09:59

cmsbuild · 2020-03-18T10:25:03Z

+1
Tested at: 0fd5cc6
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0d2e15/5242/summary.html
CMSSW: CMSSW_11_1_X_2020-03-17-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-03-18T10:25:10Z

Comparison job queued.

cmsbuild · 2020-03-18T11:56:14Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0d2e15/5242/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0d2e15/159.0_HydjetQ_B12_5020GeV_2021_ppReco+HydjetQ_B12_5020GeV_2021_ppReco+DIGIHI2021PPRECO+RECOHI2021PPRECO+ALCARECOHI2021PPRECO+HARVESTHI2021PPRECO
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0d2e15/159.1_QCD_Pt_80_120_14_HI_2021+QCD_Pt_80_120_14_HI_2021+DIGIHI2021PPRECO+RECOHI2021PPRECO+HARVESTHI2021PPRECO
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0d2e15/159.2_PhotonJets_Pt_10_14_HI_2021+PhotonJets_Pt_10_14_HI_2021+DIGIHI2021PPRECO+RECOHI2021PPRECO+HARVESTHI2021PPRECO
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0d2e15/159.3_ZMM_14_HI_2021+ZMM_14_HI_2021+DIGIHI2021PPRECO+RECOHI2021PPRECO+HARVESTHI2021PPRECO

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2692437
DQMHistoTests: Total failures: 14
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2692104
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

slava77 · 2020-03-21T21:36:25Z

+1

for #29194 0fd5cc6

code changes are in line with the PR description and the follow up review
jenkins tests pass and comparisons with the baseline show no differences
this PR is a follow up to the problem processing data at T0 https://hypernews.cern.ch/HyperNews/CMS/get/recoDevelopment/1669/1/1/1/1/1/1/1/1.html . I checked that after adding process.muonGEMDigis.useDBEMap = True to the T0 config, the reco job in run 335670 completes OK.

slava77 · 2020-03-21T21:38:51Z

@silviodonato
this PR is to be merged with #29210

slava77 · 2020-03-21T21:40:03Z

unassign alca
the feature needed for this PR is in a separate PR #29210 , no need to have the extra signature here

christopheralanwest · 2020-03-21T22:07:53Z

Why are there single-bin differences in the L1T muon DQM plots for workflow 25202.0, which is a 2016 scenario? For example:

https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_11_1_X_2020-03-17-2300+0d2e15/35524/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15_PU25/L1TEMU__L1TdeStage2uGMT_ugmtMuonhwPt.png

silviodonato · 2020-03-23T09:24:31Z

@christopheralanwest the difference is related to #29237

silviodonato · 2020-03-23T09:25:01Z

+operations

cmsbuild · 2020-03-23T09:25:27Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @silviodonato, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-03-23T09:26:23Z

waiting for #29210

silviodonato · 2020-03-23T20:40:53Z

+1

Update Run3 GEM to use the eMap from the DB

0400cce

watson-ij mentioned this pull request Mar 12, 2020

Update Run3 GEM to use the eMap from the DB #29195

Merged

cmsbuild added this to the CMSSW_11_1_X milestone Mar 12, 2020

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Mar 12, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 12, 2020

cmsbuild added tests-started and removed tests-pending labels Mar 12, 2020

cmsbuild added comparison-available tests-approved and removed comparison-pending tests-started labels Mar 12, 2020

cmsbuild added the alca-pending label Mar 13, 2020

cmsbuild added tests-started and removed tests-pending labels Mar 18, 2020

cmsbuild added tests-approved and removed tests-started labels Mar 18, 2020

cmsbuild added comparison-available and removed comparison-pending labels Mar 18, 2020

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 21, 2020

cmsbuild removed the alca-pending label Mar 21, 2020

cmsbuild added fully-signed operations-approved and removed operations-pending pending-signatures labels Mar 23, 2020

cmsbuild added orp-approved and removed orp-pending labels Mar 23, 2020

cmsbuild merged commit 97d4788 into cms-sw:master Mar 23, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Run3 GEM to use the eMap from the DB #29194

Update Run3 GEM to use the eMap from the DB #29194

watson-ij commented Mar 12, 2020 •

edited

Loading

cmsbuild commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

perrotta commented Mar 12, 2020

cmsbuild commented Mar 12, 2020 •

edited

Loading

cmsbuild commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

slava77 commented Mar 12, 2020

watson-ij commented Mar 13, 2020

slava77 commented Mar 13, 2020

cmsbuild commented Mar 13, 2020

perrotta commented Mar 18, 2020

cmsbuild commented Mar 18, 2020 •

edited

Loading

cmsbuild commented Mar 18, 2020

cmsbuild commented Mar 18, 2020

cmsbuild commented Mar 18, 2020

slava77 commented Mar 21, 2020

slava77 commented Mar 21, 2020

slava77 commented Mar 21, 2020

christopheralanwest commented Mar 21, 2020

silviodonato commented Mar 23, 2020

silviodonato commented Mar 23, 2020

cmsbuild commented Mar 23, 2020

silviodonato commented Mar 23, 2020

silviodonato commented Mar 23, 2020

Update Run3 GEM to use the eMap from the DB #29194

Update Run3 GEM to use the eMap from the DB #29194

Conversation

watson-ij commented Mar 12, 2020 • edited Loading

PR description:

PR validation:

cmsbuild commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

perrotta commented Mar 12, 2020

cmsbuild commented Mar 12, 2020 • edited Loading

cmsbuild commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

slava77 commented Mar 12, 2020

watson-ij commented Mar 13, 2020

slava77 commented Mar 13, 2020

cmsbuild commented Mar 13, 2020

perrotta commented Mar 18, 2020

cmsbuild commented Mar 18, 2020 • edited Loading

cmsbuild commented Mar 18, 2020

cmsbuild commented Mar 18, 2020

cmsbuild commented Mar 18, 2020

slava77 commented Mar 21, 2020

slava77 commented Mar 21, 2020

slava77 commented Mar 21, 2020

christopheralanwest commented Mar 21, 2020

silviodonato commented Mar 23, 2020

silviodonato commented Mar 23, 2020

cmsbuild commented Mar 23, 2020

silviodonato commented Mar 23, 2020

silviodonato commented Mar 23, 2020

watson-ij commented Mar 12, 2020 •

edited

Loading

cmsbuild commented Mar 12, 2020 •

edited

Loading

cmsbuild commented Mar 18, 2020 •

edited

Loading