Backport FastSim2017 PU and pmx wfs to 10_2_X

[ssekmen]

backport of #24829

[cmsbuild]

A new Pull Request was created by @ssekmen (Sezen Sekmen) for CMSSW_10_2_X.

It involves the following packages:

Configuration/PyReleaseValidation

@pgunnell, @zhenhu, @prebello, @cmsbuild, @kpedro88 can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

• Backported from fastsim 2017 PU and pmx wfs #24829

[ssekmen]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/32164/console Started: 2018/12/13 11:38

[cmsbuild]

-1

Tested at: 36d8989

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25490/32164/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

• RelVals:

[cmsbuild]

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

[cmsbuild]

+1
Tested at: 8ad1b41
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25490/32479/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25490/32479/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-25490/25400.17_ZEE_13+FS_ZEE_13_UP17_PU50+HARVESTUP17FS+MINIAODMCUP17FS

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 31
• DQMHistoTests: Total histograms compared: 3007400
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3007207
• DQMHistoTests: Total skipped: 190
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
• Checked 129 log files, 14 edm output root files, 31 DQM output files

[ssekmen]

@zhenhu , @prebello , as this is becoming very urgent, can pdmv please check and sign? Thanks.

[zhenhu]

+1

[ssekmen]

Thanks a lot!

[ssekmen]

@kpedro88 , @fabiocos , seems like we can merge this PR.

[kpedro88]

+upgrade

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_10_5_X is complete. This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[fabiocos]

+1

[fabiocos]

@ssekmen @zhenhu this PR is still not good, it has broken the IB:

https://cmssdt.cern.ch/SDT/html/cmssdt-ib/#/relVal/CMSSW_10_2/2019-01-10-2300?selectedArchs=slc7_amd64_gcc700&selectedArchs=slc6_amd64_gcc700&selectedFlavors=X&selectedStatus=failed

Apparently from a quick look there are 2 issues:

• the change of input files for tests 2540X.17 has broken the corresponding 2540X.0 ;
• the input for 25040X.17 is not good

[zhenhu]

Hi @ssekmen , since this is a backport of #24829, it's better to exactly follow the changes in the original PR. For example, in #24829, we added two new strings [17] [18] for fastSim2017 to avoid this issue
https://github.com/cms-sw/cmssw/pull/24829/files#diff-819af2ec677272fddf1fef50e788a88bR620
Can you try to do a similar change here?

[fabiocos]

in these backports it would be good to test all the updates, as we are clearly missing problems at integration time. I will revert this PR so as it can be further updated without leaving the IB broken