Data and MC global tags for 2018 UL #27923

christopheralanwest · 2019-09-03T21:51:02Z

PR description:

This PR includes global tags for the 2018 ultra-legacy reprocessing that are intended to be final. An unrelated change to the Run 3 prompt GT is included; this change is purely technical and will not affect any workflow.

Offline GT
Complete set of conditions for 2018UL. Includes updates to beamspot [1-2], tracker alignment [3-5], muon alignment [6-7], ECAL calibrations [8] and a fix for pixel conditions for a single fill with modified HV settings [9].

Diffs:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_v5/110X_dataRun2_v6
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_relval_v5/110X_dataRun2_relval_v6

[1] https://indico.cern.ch/event/840920/#35-ul-2018-sign-off-of-beamspo
[2] https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4213/1/1/1.html
[3] https://indico.cern.ch/event/837619/#10-ul-2018-tracker-alignment-s
[4] https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4205.html
[5] https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4205/1/1.html
[6] https://indico.cern.ch/event/839963/#33-global-position-record-for
[7] https://indico.cern.ch/event/840920/#34-ul-2018-sign-off-for-muon-a
[8] https://indico.cern.ch/event/840920/#38-ecal-sign-off-for-ultra-leg
[9] https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4197.html

Prompt-like GTs

Updated PPS conditions for consistency with 2018 UL. The prompt-like GTs are updated as well because the PPS diamond was introduced after the completion of 2018 prompt reco. A payload is needed to avoid crashes in prompt-like workflows and so we have chosen to use payloads consistent with the offline GT.

Diffs:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_PromptLike_HEfail_v3/110X_dataRun2_PromptLike_HEfail_v5
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_PromptLike_v3/110X_dataRun2_PromptLike_v5
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_PromptLike_HI_v3/110X_dataRun2_PromptLike_HI_v5

2018 MC
Complete set of conditions for 2018UL. Updated tracker alignment scenario and APEs [1]

[1] https://indico.cern.ch/event/840920/#36-ul-2018-sign-off-for-mc-tra

Diffs:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_upgrade2018_realistic_v3/110X_upgrade2018_realistic_v4
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_upgrade2018_realistic_HEfail_v3/110X_upgrade2018_realistic_HEfail_v4

Run 3 prompt
Change to EcalADCToGeVConstantRcd tag populated by O2O [1]

[1] https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4191.html

Diff:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun3_Prompt_v2/106X_dataRun3_Prompt_v3

PR validation:

For physics validation, please see above. Also, as a technical test, ran runTheMatrix.py -l limited -i all --ibeos

if this PR is a backport please specify the original PR:

Not a backport but will be backported to 10_6_X

…ions and a fix for pixel conditions for a single fill.

cmsbuild · 2019-09-03T21:51:31Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-09-03T21:56:55Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27923/11753

This PR adds an extra 12KB to repository

christopheralanwest · 2019-09-03T21:57:05Z

please test

cmsbuild · 2019-09-03T21:57:23Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2342/console Started: 2019/09/03 23:57

cmsbuild · 2019-09-03T21:57:25Z

A new Pull Request was created by @christopheralanwest for master.

It involves the following packages:

Configuration/AlCa

@christopheralanwest, @tocheng, @cmsbuild, @franzoni, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2019-09-03T23:34:53Z

-1

Tested at: 48a23b2

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-47e69c/2342/summary.html

I found follow errors while testing this PR

Failed tests: AddOn

AddOn:

I found errors in the following addon tests:

cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Wed Sep 4 01:20:13 2019-date Wed Sep 4 01:13:54 2019 s - exit: 16640
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Wed Sep 4 01:20:26 2019-date Wed Sep 4 01:14:15 2019 s - exit: 16640
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Wed Sep 4 01:25:55 2019-date Wed Sep 4 01:14:34 2019 s - exit: 16640

cmsbuild · 2019-09-03T23:35:00Z

Comparison job queued.

christopheralanwest · 2019-09-04T00:47:34Z

There is a problem with the tag PPSDiamondTimingCalibration_2017_2018_prompt_v1 in the promptlike global tag. The errors in the three failing tests are equivalent:

----- Begin Fatal Exception 04-Sep-2019 01:19:56 CEST-----------------------
An exception of category 'WrongNumParameters' occurred while
   [0] Processing  Event run: 295606 lumi: 1 event: 37589 stream: 3
   [1] Running path 'reconstruction_step'
   [2] Calling method for module CTPPSDiamondRecHitProducer/'ctppsDiamondRecHits'
Exception Message:
FormulaEvaluator expected at least 4 but was passed only 0
----- End Fatal Exception -------------------------------------------------

I will follow up with PPS experts.

cmsbuild · 2019-09-04T02:30:53Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-47e69c/2342/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 5454 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2955700
DQMHistoTests: Total failures: 5852
DQMHistoTests: Total nulls: 7
DQMHistoTests: Total successes: 2949500
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.016 KiB( 33 files compared)
DQMHistoSizes: changed ( 136.85 ): 0.016 KiB JetMET/SUSYDQM
Checked 145 log files, 15 edm output root files, 34 DQM output files

cmsbuild · 2019-09-04T14:18:49Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-09-04T14:26:21Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27923/11761

This PR adds an extra 12KB to repository

cmsbuild · 2019-09-04T14:26:49Z

Pull request #27923 was updated. @christopheralanwest, @tocheng, @cmsbuild, @franzoni, @tlampen, @pohsun can you please check and sign again.

christopheralanwest · 2019-09-04T14:27:09Z

please test workflow 138.1

cmsbuild · 2019-09-04T14:27:14Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2358/console Started: 2019/09/04 16:28

cmsbuild · 2019-09-04T16:37:31Z

+1
Tested at: 18f28bb
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-47e69c/2358/summary.html

cmsbuild · 2019-09-04T16:37:34Z

Comparison job queued.

cmsbuild · 2019-09-04T19:09:41Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-47e69c/2358/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-47e69c/138.1_RunCosmics2019+RunCosmics2019+RECOCOSDRUN3+ALCACOSDRUN3+HARVESTDCRUN3

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 5406 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2955700
DQMHistoTests: Total failures: 5676
DQMHistoTests: Total nulls: 6
DQMHistoTests: Total successes: 2949677
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.016 KiB( 33 files compared)
DQMHistoSizes: changed ( 136.85 ): 0.016 KiB JetMET/SUSYDQM
Checked 145 log files, 15 edm output root files, 34 DQM output files

christopheralanwest · 2019-09-04T19:17:38Z

+1

Comparison test differences seen only in 2018 data and MC workflows
Payloads have been validated individually by subdetector experts, as described in the introduction to this PR

@fabiocos @kpedro88 I suggest merging this PR prior to building 11_0_0_pre7 so that the global tag updates that complete the 2018 UL preparation enter the standard release validation

cmsbuild · 2019-09-04T19:18:12Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-09-04T20:12:47Z

+1

christopheralanwest added 4 commits September 3, 2019 23:25

Updates to beamspot, tracker alignment, muon alignment, ECAL calibrat…

c7f72b4

…ions and a fix for pixel conditions for a single fill.

Updated PPS conditions for consistency with 2018 UL

7b37ea5

Updated 2018 MC tracker alignment scenario and APEs

8c00bc1

Change to EcalADCToGeVConstantRcd tag populated by O2O

48a23b2

cmsbuild added this to the CMSSW_11_0_X milestone Sep 3, 2019

cmsbuild added alca-pending code-checks-pending comparison-pending orp-pending labels Sep 3, 2019

cmsbuild added pending-signatures tests-pending labels Sep 3, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 3, 2019

cmsbuild added tests-started and removed tests-pending labels Sep 3, 2019

cmsbuild added tests-rejected and removed tests-started labels Sep 3, 2019

cmsbuild added comparison-available and removed comparison-pending labels Sep 4, 2019

Make PPSDiamondTimingCalibration tag consistent with tag in offline GT.

18f28bb

cmsbuild removed the code-checks-approved label Sep 4, 2019

cmsbuild added comparison-pending tests-pending labels Sep 4, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 4, 2019

cmsbuild added tests-started and removed tests-pending labels Sep 4, 2019

cmsbuild added tests-approved and removed tests-started labels Sep 4, 2019

cmsbuild added comparison-available and removed comparison-pending labels Sep 4, 2019

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Sep 4, 2019

cmsbuild added orp-approved and removed orp-pending labels Sep 4, 2019

cmsbuild merged commit 1ec1f25 into cms-sw:master Sep 4, 2019

cmsbuild mentioned this pull request Sep 5, 2019

DD4hep: Tracker Builder, Numbering and GeometricDet pre-migration #27926

Merged

christopheralanwest mentioned this pull request Sep 5, 2019

Data and MC global tags for 2018 UL [10_6_X] #27934

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Data and MC global tags for 2018 UL #27923

Data and MC global tags for 2018 UL #27923

christopheralanwest commented Sep 3, 2019 •

edited

Loading

cmsbuild commented Sep 3, 2019

cmsbuild commented Sep 3, 2019

christopheralanwest commented Sep 3, 2019

cmsbuild commented Sep 3, 2019 •

edited

Loading

cmsbuild commented Sep 3, 2019

cmsbuild commented Sep 3, 2019

cmsbuild commented Sep 3, 2019

christopheralanwest commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

christopheralanwest commented Sep 4, 2019

cmsbuild commented Sep 4, 2019 •

edited

Loading

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

christopheralanwest commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

fabiocos commented Sep 4, 2019

Data and MC global tags for 2018 UL #27923

Data and MC global tags for 2018 UL #27923

Conversation

christopheralanwest commented Sep 3, 2019 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR:

cmsbuild commented Sep 3, 2019

cmsbuild commented Sep 3, 2019

christopheralanwest commented Sep 3, 2019

cmsbuild commented Sep 3, 2019 • edited Loading

cmsbuild commented Sep 3, 2019

cmsbuild commented Sep 3, 2019

cmsbuild commented Sep 3, 2019

christopheralanwest commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

christopheralanwest commented Sep 4, 2019

cmsbuild commented Sep 4, 2019 • edited Loading

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

christopheralanwest commented Sep 4, 2019

cmsbuild commented Sep 4, 2019

fabiocos commented Sep 4, 2019

christopheralanwest commented Sep 3, 2019 •

edited

Loading

cmsbuild commented Sep 3, 2019 •

edited

Loading

cmsbuild commented Sep 4, 2019 •

edited

Loading