[13_0_X] EMTF add DQM plots for unconstrained pT and dxy #42168

lutuck · 2023-06-30T12:08:23Z

PR description:

This PR adds DQM plots using unconstrained pT and dxy. These plots will be useful for monitoring now that the neural network is working at P5.

This is a backport of #42155

PR validation:

Validated by runTheMatrix.py -l limited -i all --ibeos and everything looks good.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

@eyigitba

cmsbuild · 2023-06-30T12:08:48Z

A new Pull Request was created by @lutuck (Lucas Faria de S Tucker) for CMSSW_13_0_X.

It involves the following packages:

DQM/L1TMonitor (dqm)

@nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@missirol this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from EMTF add DQM plots for unconstrained pT and dxy #42155

emanueleusai · 2023-07-03T05:38:57Z

type l1t

emanueleusai · 2023-07-03T05:39:03Z

please test

cmsbuild · 2023-07-03T06:52:46Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d9c869/33512/summary.html
COMMIT: f37ffdf
CMSSW: CMSSW_13_0_X_2023-07-02-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42168/33512/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

10224.0

----- Begin Fatal Exception 03-Jul-2023 08:13:43 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root
      [c] Input file root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root?tried=+1213xrootd-redic.pi.infn.it,&xrdcl.requuid=7af6dace-69a8-4e42-9fb5-4f62a825c665
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

23.023.0_JpsiMM/step2_JpsiMM.log
134.808134.808_RunSingleMuPrpt2015C/step2_RunSingleMuPrpt2015C.log
134.806134.806_RunMuonEG2015C/step2_RunMuonEG2015C.log

Expand to see more relval errors ...

perrotta · 2023-07-04T05:15:20Z

backport of #42155

emanueleusai · 2023-07-05T07:12:13Z

please test

cmsbuild · 2023-07-05T09:39:01Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d9c869/33555/summary.html
COMMIT: f37ffdf
CMSSW: CMSSW_13_0_X_2023-07-04-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42168/33555/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 2 lines from the logs
Reco comparison results: 9 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3286334
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3286303
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1129.1800000000005 KiB( 48 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 56.459 KiB L1T/L1TStage2EMTF
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

emanueleusai · 2023-07-07T05:01:18Z

testing 13_0 at p5

emanueleusai · 2023-07-11T06:00:28Z

+1

p5 test ok

emanueleusai · 2023-08-16T22:41:41Z

please test

cmsbuild · 2023-08-17T04:20:34Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d9c869/34332/summary.html
COMMIT: 8472f30
CMSSW: CMSSW_13_0_X_2023-08-16-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42168/34332/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 19 lines from the logs
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3281270
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3281239
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1129.1800000000005 KiB( 48 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 56.459 KiB L1T/L1TStage2EMTF
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

cmsbuild · 2023-08-19T02:08:58Z

Pull request #42168 was updated. @nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please check and sign again.

emanueleusai · 2023-08-22T16:59:28Z

please test

emanueleusai · 2023-08-22T17:02:06Z

+1

last commit just removed comments, no changes expected in tests

cmsbuild · 2023-08-22T19:24:20Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d9c869/34422/summary.html
COMMIT: 654a9f5
CMSSW: CMSSW_13_0_X_2023-08-22-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42168/34422/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 228 lines from the logs
Reco comparison results: 15 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3281270
DQMHistoTests: Total failures: 983
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3280265
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1129.1800000000005 KiB( 48 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 56.459 KiB L1T/L1TStage2EMTF
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

perrotta · 2023-08-23T20:59:41Z

unhold

cmsbuild · 2023-08-23T21:00:06Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-08-23T21:00:14Z

+1

Identical to the master

lutuck added 2 commits June 30, 2023 14:05

Add Unconstrained Plots

3ca8337

Corrected formatting of plot names

f37ffdf

cmsbuild added this to the CMSSW_13_0_X milestone Jun 30, 2023

cmsbuild added dqm-pending pending-signatures tests-pending orp-pending labels Jun 30, 2023

cmsbuild added tests-started l1t and removed tests-pending labels Jul 3, 2023

cmsbuild added tests-rejected and removed tests-started labels Jul 3, 2023

cmsbuild added the backport label Jul 4, 2023

cmsbuild added tests-started and removed tests-rejected labels Jul 5, 2023

cmsbuild added tests-approved and removed tests-started labels Jul 5, 2023

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Jul 11, 2023

cmsbuild added tests-started and removed tests-pending labels Aug 16, 2023

cmsbuild added tests-approved and removed tests-started labels Aug 17, 2023

removed comments

654a9f5

cmsbuild added tests-pending and removed tests-approved labels Aug 19, 2023

cmsbuild added backport-ok and removed backport labels Aug 21, 2023

cmsbuild added tests-started and removed tests-pending labels Aug 22, 2023

cmsbuild added dqm-approved and removed dqm-pending labels Aug 22, 2023

cmsbuild added tests-approved and removed tests-started labels Aug 22, 2023

cmsbuild added fully-signed and removed pending-signatures hold labels Aug 23, 2023

cmsbuild added orp-approved and removed orp-pending labels Aug 23, 2023

cmsbuild merged commit b99cd4d into cms-sw:CMSSW_13_0_X Aug 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[13_0_X] EMTF add DQM plots for unconstrained pT and dxy #42168

[13_0_X] EMTF add DQM plots for unconstrained pT and dxy #42168

lutuck commented Jun 30, 2023

cmsbuild commented Jun 30, 2023 •

edited

Loading

emanueleusai commented Jul 3, 2023

emanueleusai commented Jul 3, 2023

cmsbuild commented Jul 3, 2023

perrotta commented Jul 4, 2023

emanueleusai commented Jul 5, 2023

cmsbuild commented Jul 5, 2023

emanueleusai commented Jul 7, 2023

emanueleusai commented Jul 11, 2023

emanueleusai commented Aug 16, 2023

cmsbuild commented Aug 17, 2023

cmsbuild commented Aug 19, 2023

emanueleusai commented Aug 22, 2023

emanueleusai commented Aug 22, 2023

cmsbuild commented Aug 22, 2023

perrotta commented Aug 23, 2023

cmsbuild commented Aug 23, 2023

perrotta commented Aug 23, 2023

[13_0_X] EMTF add DQM plots for unconstrained pT and dxy #42168

[13_0_X] EMTF add DQM plots for unconstrained pT and dxy #42168

Conversation

lutuck commented Jun 30, 2023

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Jun 30, 2023 • edited Loading

emanueleusai commented Jul 3, 2023

emanueleusai commented Jul 3, 2023

cmsbuild commented Jul 3, 2023

RelVals

RelVals-INPUT

perrotta commented Jul 4, 2023

emanueleusai commented Jul 5, 2023

cmsbuild commented Jul 5, 2023

Comparison Summary

emanueleusai commented Jul 7, 2023

emanueleusai commented Jul 11, 2023

emanueleusai commented Aug 16, 2023

cmsbuild commented Aug 17, 2023

Comparison Summary

cmsbuild commented Aug 19, 2023

emanueleusai commented Aug 22, 2023

emanueleusai commented Aug 22, 2023

cmsbuild commented Aug 22, 2023

Comparison Summary

perrotta commented Aug 23, 2023

cmsbuild commented Aug 23, 2023

perrotta commented Aug 23, 2023

cmsbuild commented Jun 30, 2023 •

edited

Loading