Global tags for Run 2 UL reprocessing #28524

christopheralanwest · 2019-12-03T04:11:44Z

PR description:

This autoCond update finalizes the global tags for the Run 2 UL reprocessing, including cosmic GTs (the 2017 cosmic GT has already been updated in PR #28079). A separate GT will be used for the early 2016 MC scenario (the "pre-VFP" era) and a new autoCond entry auto:run2_mc_pre_vfp has been created.

The offline GTs remove an record of type HcalRespCorrsRcd with label "bugged". As this record will be obsolete with the UL reprocessing, commit c2349b3 removes a customization from 2016 workflows that makes use of this labeled record. @abdoulline @mariadalfonso @mazarkin

GT diffs

2016 pp collision MC
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_mcRun2_asymptotic_v4/110X_mcRun2_asymptotic_v5

2016 pp collision MC for "pre-VFP" era (new autoCond key)
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_mcRun2_asymptotic_v4/110X_mcRun2_asymptotic_preVFP_v1

which differs from the standard 2016 pp collision MC GT as follows;

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_mcRun2_asymptotic_v5/110X_mcRun2_asymptotic_preVFP_v1

2016 cosmic MC
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_mcRun2cosmics_startup_deco_v3/110X_mcRun2cosmics_startup_deco_v4

Offline data
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_v8/110X_dataRun2_v9
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_relval_v8/110X_dataRun2_relval_v9

Prompt-like data
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_PromptLike_HEfail_v7/110X_dataRun2_PromptLike_HEfail_v9
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_PromptLike_v7/110X_dataRun2_PromptLike_v9
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_dataRun2_PromptLike_HI_v7/110X_dataRun2_PromptLike_HI_v9

2018 cosmic GTs
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_upgrade2018cosmics_realistic_deco_v4/110X_upgrade2018cosmics_realistic_deco_v5
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/110X_upgrade2018cosmics_realistic_peak_v5/110X_upgrade2018cosmics_realistic_peak_v6

PR validation:

Details of subsystem validation of the conditions can be found at [1]-[15] below. In additional, a technical test was performed: runTheMatrix.py -l limited,7.2,7.4,7.21,7.22 --ibeos

8 July 2019:
HCAL [1]: https://indico.cern.ch/event/832785/contributions/3489397/attachments/1876358/3089810/Hcal_UL_2016_2018_July8_v1.pdf, followup at https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4188.html

22 July 2019:
Pixel [2]: https://indico.cern.ch/event/835589/contributions/3513525/attachments/1887574/3114665/20190729_AlCaDB_UL16.pdf; See also #27600

8 Aug 2018:
HCAL negative energy filter [3]: PPD meeting

2 Sep 2019:
PF [3]:
https://indico.cern.ch/event/843260/#39-sign-off-pf-calibration-for
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4215/1/1.html

30 Sep 2019:
PPS [4]:
https://indico.cern.ch/event/848325/#2-ul-2016-sign-off-pps-alignme
Tracker alignment + APE [5]:
https://indico.cern.ch/event/848325/#3-ul-2016-sign-off-tracker-ali
https://hypernews.cern.ch/HyperNews/CMS/get/tk-alignment/1875.html

7 Oct 2019
Tracker alignment + APE HN announcement [6]:
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4228/1.html

21 Oct 2019:
Beamspot [7]:
https://indico.cern.ch/event/854617/#7-beamspot-for-ultra-legacy-20
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4234.html
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4234/1.html

28 Oct 2019:
PPS details[8]
https://indico.cern.ch/event/854618/#3-ul-2016-sign-off-update-pps
SiStrip G2 calibration[9]
https://indico.cern.ch/event/854618/#4-updated-sistrip-g2-calibrati
Tracker MC misalignment scenario[10]
https://indico.cern.ch/event/854618/#11-tracker-mc-misalignment-sce

4 Nov 2019:
PPS, update of runs 303817-303819 of 2017E , virtual presentation [11]:
https://indico.cern.ch/event/859478/#4-ul-2016-2018-updated-pps-ali

11 Nov 2019:
ECAL [12]:
https://indico.cern.ch/event/859480/#2-ul-2016-sign-off-ecal-data-a
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4241.html
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4241/1.html
Muon + GRP[13]:
https://indico.cern.ch/event/859480/#3-ul-2016-sign-off-muongpr

18 Nov 2019:
EGamma, partial sign-off, data tags and PR still needed [14]:
https://indico.cern.ch/event/862728/#3-ul-2016-sign-off-egamma-mc-a

21 Nov 2019:
EGamma final sign-off [15]:
https://indico.cern.ch/event/860471/#12-ul-16-egm-regression-sign-o

if this PR is a backport please specify the original PR:

This PR is not a backport but all updates will be backported to 10_6_X.

…ltra-legacy reprocessing

cmsbuild · 2019-12-03T04:12:13Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-12-03T04:18:29Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28524/12993

This PR adds an extra 48KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Phase out old pre-GED GsfElectron sequence #28429, ECAL for Time dependent MC. Premix option C. Handle for APD/PN tag for time dependence #28214, Add Patatrack process modifiers and workflows #28522, Use PREMIX in the name of the .97 workflow #28470

christopheralanwest · 2019-12-03T04:18:56Z

please test workflow 7.2,7.4,7.21,7.22

cmsbuild · 2019-12-03T04:19:00Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/3740/console Started: 2019/12/03 05:19

cmsbuild · 2019-12-03T04:19:03Z

A new Pull Request was created by @christopheralanwest for master.

It involves the following packages:

Configuration/AlCa
Configuration/PyReleaseValidation

@cmsbuild, @chayanit, @zhenhu, @christopheralanwest, @tocheng, @pgunnell, @franzoni, @kpedro88, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2019-12-03T06:01:01Z

+1
Tested at: c2349b3
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b6b3e5/3740/summary.html

cmsbuild · 2019-12-03T06:01:05Z

Comparison job queued.

cmsbuild · 2019-12-03T07:22:09Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b6b3e5/3740/summary.html

There are some workflows for which there are errors in the baseline:
25202.0 step 2
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-b6b3e5/7.21_Cosmics_UP17+Cosmics_UP17+DIGICOS_UP17+RECOCOS_UP17+ALCACOS_UP17+HARVESTCOS_UP17
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-b6b3e5/7.22_Cosmics_UP16+Cosmics_UP16+DIGICOS_UP16+RECOCOS_UP16+ALCACOS_UP16+HARVESTCOS_UP16

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 5662 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 2691485
DQMHistoTests: Total failures: 4721
DQMHistoTests: Total nulls: 5
DQMHistoTests: Total successes: 2686432
DQMHistoTests: Total skipped: 327
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.008 KiB( 32 files compared)
DQMHistoSizes: changed ( 136.731 ): 0.008 KiB JetMET/SUSYDQM
Checked 144 log files, 12 edm output root files, 33 DQM output files

cmsbuild · 2019-12-06T20:32:30Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28524/13071

This PR adds an extra 48KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/AlCa/python/autoCond.py modified in PR(s): Introducing SiPixelCalSingleMuon ALCARECO #28535
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Phase out old pre-GED GsfElectron sequence #28429, ECAL for Time dependent MC. Premix option C. Handle for APD/PN tag for time dependence #28214, Use PREMIX in the name of the .97 workflow #28470, Introducing SiPixelCalSingleMuon ALCARECO #28535

cmsbuild · 2019-12-06T20:32:55Z

Pull request #28524 was updated. @cmsbuild, @chayanit, @zhenhu, @christopheralanwest, @tocheng, @pgunnell, @franzoni, @kpedro88, @tlampen, @pohsun can you please check and sign again.

cmsbuild · 2019-12-06T20:47:22Z

+1
Tested at: d602e6c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b6b3e5/3853/summary.html

cmsbuild · 2019-12-06T20:47:26Z

Comparison job queued.

christopheralanwest · 2019-12-06T20:48:23Z

please abort

cmsbuild · 2019-12-06T22:44:42Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b6b3e5/3853/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-b6b3e5/7.21_Cosmics_UP17+Cosmics_UP17+DIGICOS_UP17+RECOCOS_UP17+ALCACOS_UP17+HARVESTCOS_UP17
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-b6b3e5/7.22_Cosmics_UP16+Cosmics_UP16+DIGICOS_UP16+RECOCOS_UP16+ALCACOS_UP16+HARVESTCOS_UP16

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 7684 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2800084
DQMHistoTests: Total failures: 8731
DQMHistoTests: Total nulls: 5
DQMHistoTests: Total successes: 2791007
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.008 KiB( 33 files compared)
DQMHistoSizes: changed ( 136.731 ): 0.008 KiB JetMET/SUSYDQM
Checked 147 log files, 16 edm output root files, 34 DQM output files

christopheralanwest · 2019-12-06T23:33:48Z

please test workflow 7.2,7.4,7.21,7.22

cmsbuild · 2019-12-06T23:34:48Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/3856/console Started: 2019/12/07 00:36

cmsbuild · 2019-12-07T01:12:49Z

+1
Tested at: d7b3f97
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b6b3e5/3856/summary.html

cmsbuild · 2019-12-07T01:12:53Z

Comparison job queued.

cmsbuild · 2019-12-07T02:51:06Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b6b3e5/3856/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-b6b3e5/7.21_Cosmics_UP17+Cosmics_UP17+DIGICOS_UP17+RECOCOS_UP17+ALCACOS_UP17+HARVESTCOS_UP17
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-b6b3e5/7.22_Cosmics_UP16+Cosmics_UP16+DIGICOS_UP16+RECOCOS_UP16+ALCACOS_UP16+HARVESTCOS_UP16

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 7684 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2800084
DQMHistoTests: Total failures: 8731
DQMHistoTests: Total nulls: 5
DQMHistoTests: Total successes: 2791007
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.008 KiB( 33 files compared)
DQMHistoSizes: changed ( 136.731 ): 0.008 KiB JetMET/SUSYDQM
Checked 147 log files, 16 edm output root files, 34 DQM output files

christopheralanwest · 2019-12-07T20:06:54Z

+1

fabiocos · 2019-12-07T21:56:46Z

@chayanit @kpedro88 you had already signed the PyRelVal update, I integrate the PR as it is the basis for the next 10_6_X build

fabiocos · 2019-12-07T21:56:51Z

+1

fabiocos · 2019-12-07T21:56:56Z

merge

christopheralanwest added 5 commits December 3, 2019 04:40

Complete set of updates for ultra-legacy reprocessing of Run 2 data

d0b5df9

Updates of EGM regression and PPS calibrations introduced for Run 2 u…

cbb5334

…ltra-legacy reprocessing

Updates to 2016 MC GTs for ultra-legacy reprocessing

0e5fe47

Updates of 2018 cosmic GTs for consistency with pp collision GT

355e6a4

Remove obsolete customization

c2349b3

cmsbuild added this to the CMSSW_11_1_X milestone Dec 3, 2019

cmsbuild added alca-pending code-checks-pending comparison-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Dec 3, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Dec 3, 2019

cmsbuild added tests-started and removed tests-pending labels Dec 3, 2019

cmsbuild added tests-approved and removed tests-started labels Dec 3, 2019

cmsbuild added comparison-available and removed comparison-pending labels Dec 3, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Dec 6, 2019

christopheralanwest mentioned this pull request Dec 6, 2019

Global tags for Run 2 UL reprocessing [10_6_X] #28577

Merged

cmsbuild added tests-started and removed tests-pending labels Dec 6, 2019

cmsbuild added tests-approved and removed tests-started labels Dec 7, 2019

cmsbuild added comparison-available and removed comparison-pending labels Dec 7, 2019

cmsbuild added alca-approved and removed alca-pending labels Dec 7, 2019

cmsbuild added orp-approved and removed orp-pending labels Dec 7, 2019

cmsbuild merged commit bf5212e into cms-sw:master Dec 7, 2019

christopheralanwest mentioned this pull request Dec 10, 2019

Global tags for Run 2 UL reprocessing [11_0_X] #28603

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Global tags for Run 2 UL reprocessing #28524

Global tags for Run 2 UL reprocessing #28524

christopheralanwest commented Dec 3, 2019 •

edited

Loading

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 3, 2019

christopheralanwest commented Dec 3, 2019

cmsbuild commented Dec 3, 2019 •

edited

Loading

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 6, 2019

cmsbuild commented Dec 6, 2019

cmsbuild commented Dec 6, 2019

cmsbuild commented Dec 6, 2019

christopheralanwest commented Dec 6, 2019

cmsbuild commented Dec 6, 2019

christopheralanwest commented Dec 6, 2019

cmsbuild commented Dec 6, 2019 •

edited

Loading

cmsbuild commented Dec 7, 2019

cmsbuild commented Dec 7, 2019

cmsbuild commented Dec 7, 2019

christopheralanwest commented Dec 7, 2019

fabiocos commented Dec 7, 2019

fabiocos commented Dec 7, 2019

fabiocos commented Dec 7, 2019

Global tags for Run 2 UL reprocessing #28524

Global tags for Run 2 UL reprocessing #28524

Conversation

christopheralanwest commented Dec 3, 2019 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR:

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 3, 2019

christopheralanwest commented Dec 3, 2019

cmsbuild commented Dec 3, 2019 • edited Loading

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 3, 2019

cmsbuild commented Dec 6, 2019

cmsbuild commented Dec 6, 2019

cmsbuild commented Dec 6, 2019

cmsbuild commented Dec 6, 2019

christopheralanwest commented Dec 6, 2019

cmsbuild commented Dec 6, 2019

christopheralanwest commented Dec 6, 2019

cmsbuild commented Dec 6, 2019 • edited Loading

cmsbuild commented Dec 7, 2019

cmsbuild commented Dec 7, 2019

cmsbuild commented Dec 7, 2019

christopheralanwest commented Dec 7, 2019

fabiocos commented Dec 7, 2019

fabiocos commented Dec 7, 2019

fabiocos commented Dec 7, 2019

christopheralanwest commented Dec 3, 2019 •

edited

Loading

cmsbuild commented Dec 3, 2019 •

edited

Loading

cmsbuild commented Dec 6, 2019 •

edited

Loading