Add relval steps for Charmonium PD and MuonPOGSkim (Restart from newer IB)

[kfjack]

Add relval steps for MuonPOGSkim and MuonPOGJPsiSkim, restart from newer IB.

MuonPOGSkim is added to the SingleMuon PD relvals
Add new workflow for Charmonium PD and MuonPOGJPsiSkim
For testing MuonPOGSkim with 2017F relval: 136.835
For testing MuonPOGJPsiSkim with 2017F relval: 136.848

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22665/4022

[cmsbuild]

A new Pull Request was created by @kfjack (Kai-Feng Chen) for master.

It involves the following packages:

Configuration/PyReleaseValidation

@GurpreetSinghChahal, @cmsbuild, @prebello, @kpedro88, @fabozzi can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @ebrondol, @ghellwig, @Martin-Grunewald this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[prebello]

please test workflow 136.835,136.848

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/26949/console Started: 2018/03/19 14:14

[cmsbuild]

-1

Tested at: 90f1ad1

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22665/26949/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

• RelVals:

When I ran the RelVals I found an error in the following worklfows:
136.848 step2

runTheMatrix-results/136.848_RunCharmonium2017F+RunCharmonium2017F+HLTDR2_2017+RECODR2_2017reHLT_skimCharmonium_Prompt+HARVEST2017/step2_RunCharmonium2017F+RunCharmonium2017F+HLTDR2_2017+RECODR2_2017reHLT_skimCharmonium_Prompt+HARVEST2017.log

[cmsbuild]

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

[kfjack]

Somehow the input raw files for Charmonium PD are not available from the servers. Could we retest it again? Or try the workflows for other era, e.g. 136.844 for 2017B, 136.845 for 2017C, 136.846 for 2017D, 136.847 for 2017E?

[fabozzi]

We should check if they are available at CERN T2 (if not, we have to request a transfer to disk)

[kfjack]

Ok let me check!
Update: I think we can test with 136.845 first since 2017C is partially available on CERN T2.

[fabozzi]

please test workflow 136.835,136.845

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/27018/console Started: 2018/03/21 14:24

[cmsbuild]

+1
Tested at: 90f1ad1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22665/27018/summary.html

[cmsbuild]

Comparison job queued.

[fabozzi]

@kfjack can you ask the skim contacts to check the output of this test before we sign off?

[kfjack]

@fabozzi Sure!
@sscruz Could you help to check if the output of this relval test fits your needs? Many thanks. (Another mail sent to the original discussion thread.)

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22665/27018/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22665/136.835_RunSingleMu2017F+RunSingleMu2017F+HLTDR2_2017+RECODR2_2017reHLT_skimSingleMu_Prompt_Lumi+HARVEST2017_L1TMuDQM
• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22665/136.845_RunCharmonium2017C+RunCharmonium2017C+HLTDR2_2017+RECODR2_2017reHLT_skimCharmonium_Prompt+HARVEST2017

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 29
• DQMHistoTests: Total histograms compared: 2497880
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2497703
• DQMHistoTests: Total skipped: 176
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 1.43000000014 KiB( 22 files compared)
• Checked 118 log files, 9 edm output root files, 29 DQM output files

[kfjack]

@fabozzi I received the confirmation, the skim content is okay.

[fabozzi]

+1

[kpedro88]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[fabiocos]

@fabozzi @prebello are all the needed input files available now? Deploying tests that systematically fail for several days is not very useful.

@smuzaffar FYI

[fabozzi]

We have tested on the datasets available at CERN T2. Should we ask a Phedex transfer also for the IB?

[fabiocos]

+1