Final global tags for 2017 UL reprocessing

[christopheralanwest]

PR description:

This PR updates the global tags relevant for the 2017 UL reprocessing, as well as a fix for PPS conditions in prompt-like global tags. Both data and MC global tags are updated.

The changes are as follows:

106X_dataRun2_v17:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_v17/106X_dataRun2_v13

106X_dataRun2_relval_v16:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_relval_v16/106X_dataRun2_relval_v12

106X_dataRun2_PromptLike_HEfail_v9:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_PromptLike_HEfail_v9/106X_dataRun2_PromptLike_HEfail_v8

106X_dataRun2_PromptLike_v10:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_PromptLike_v10/106X_dataRun2_PromptLike_v9

106X_dataRun2_PromptLike_HI_v10:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_PromptLike_HI_v10/106X_dataRun2_PromptLike_HI_v9

106X_mc2017_realistic_v7:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_mc2017_realistic_v7/106X_mc2017_realistic_v5

PR validation:

An enormous amount of work went into validating the conditions for this PR. Details can be found at the sign-off talks listed at:

• Data: https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4163.html
• MC: https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4159.html

As a technical check, I ran runTheMatrix.py -l limited -i all --ibeos

if this PR is a backport please specify the original PR:

This PR is not a backport but a backport will be prepared for 10_6_X.

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27342/10605

• This PR adds an extra 12KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/AlCa/python/autoCond.py modified in PR(s): Adding Run 330098 (2019 MWGR3) in data relvals #27341, Update 2021 global tags to include new pixel conditions #27334

[cmsbuild]

A new Pull Request was created by @christopheralanwest for master.

It involves the following packages:

Configuration/AlCa

@christopheralanwest, @tocheng, @cmsbuild, @franzoni, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[christopheralanwest]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1161/console Started: 2019/06/26 05:19

[cmsbuild]

+1
Tested at: 95a0cac
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-470899/1161/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-470899/1161/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 10052 differences found in the comparisons
• DQMHistoTests: Total files compared: 33
• DQMHistoTests: Total histograms compared: 3254598
• DQMHistoTests: Total failures: 2630
• DQMHistoTests: Total nulls: 28
• DQMHistoTests: Total successes: 3251606
• DQMHistoTests: Total skipped: 334
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.008 KiB( 32 files compared)
• DQMHistoSizes: changed ( 4.53,... ): -0.012 KiB JetMET/SUSYDQM
• DQMHistoSizes: changed ( 1000.0 ): 0.004 KiB JetMET/SUSYDQM
• DQMHistoSizes: changed ( 136.85 ): 0.016 KiB JetMET/SUSYDQM
• DQMHistoSizes: changed ( 136.731 ): -0.004 KiB JetMET/SUSYDQM
• Checked 137 log files, 14 edm output root files, 33 DQM output files

[christopheralanwest]

The differences in pre-2017 workflows are not expected. They appear to only affect EcalRecHitsSorted collections and downstream quantities. The differences arise because a change has been made to the pre-2017 history of the EcalPFRecHitThresholdsRcd tag:

lxplus793>conddb diff EcalPFRecHitThresholds_UL_Run1_Run2_2017_v1 EcalPFRecHitThresholds_2018_v1_offline
[2019-06-26 14:49:33,059] INFO: Connecting to pro [frontier://PromptProd/CMS_CONDITIONS]
Range               pro::EcalPFRecHitThresholds_UL_Run1_Run2_2017_v1 Payload  pro::EcalPFRecHitThresholds_2018_v1_offline Payload  
------------------  --------------------------------------------------------  ---------------------------------------------------  
[1, 265475)         9fdf3a78064032372f6107368c8da1c1a44a94f2                  9f22e391bcdcabe1f9f13608961bbed938c8a05c             
[265475, 286694)    0d6c3b61b14fec97f10bfba826350bc7f2a8f413                  9f22e391bcdcabe1f9f13608961bbed938c8a05c             
[286694, Infinity)  f6ae126e7828a2b6f89869d65bdfd1902f9574f7                  9f22e391bcdcabe1f9f13608961bbed938c8a05c             

I will follow up with ECAL experts offline to have this fixed.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27342/10716

• This PR adds an extra 16KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/AlCa/python/autoCond.py modified in PR(s): Adding Run 330098 (2019 MWGR3) in data relvals #27341

[cmsbuild]

Pull request #27342 was updated. @christopheralanwest, @tocheng, @cmsbuild, @franzoni, @tlampen, @pohsun can you please check and sign again.

[christopheralanwest]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1287/console Started: 2019/07/01 18:38

[christopheralanwest]

The last update introduced two changes:

• The beamspot tag used by the offline data GT was updated by appending the BeamSpotObjects_post2017_LumiBased_v1 to a copy of the BeamSpotObjects_2017UL_LumiBased_v1 tag. This was done to ensure that the 2018 history is unchanged.
• A purely technical change to the 2017 realistic MC GT: an obsolete, unused record was removed.

[cmsbuild]

+1
Tested at: ca5fbe7
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-470899/1287/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-470899/1287/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 6874 differences found in the comparisons
• DQMHistoTests: Total files compared: 33
• DQMHistoTests: Total histograms compared: 3290686
• DQMHistoTests: Total failures: 9266
• DQMHistoTests: Total nulls: 6
• DQMHistoTests: Total successes: 3281080
• DQMHistoTests: Total skipped: 334
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.012 KiB( 32 files compared)
• DQMHistoSizes: changed ( 136.788 ): -0.012 KiB JetMET/SUSYDQM
• Checked 137 log files, 14 edm output root files, 33 DQM output files

[christopheralanwest]

+1

• Changes seen only in 2017 workflows, except for PPS quantities in 2018 data. PPS is a special case as its reconstruction software was developed in parallel to the 2017 UL preparation and therefore there is no requirement to leave the 2018 PPS quantities unchanged during the 2017 UL preparation.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[fabiocos]

+1