[13_1_X] EMTF add DQM plots for unconstrained pT and dxy #42167

lutuck · 2023-06-30T12:00:47Z

PR description:

This PR adds DQM plots using unconstrained pT and dxy. These plots will be useful for monitoring now that the neural network is working at P5.

This is a backport of #42155

PR validation:

Validated by runTheMatrix.py -l limited -i all --ibeos and everything looks good.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

@eyigitba

cmsbuild · 2023-06-30T12:01:10Z

A new Pull Request was created by @lutuck (Lucas Faria de S Tucker) for CMSSW_13_1_X.

It involves the following packages:

DQM/L1TMonitor (dqm)

@nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@missirol this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from EMTF add DQM plots for unconstrained pT and dxy #42155

emanueleusai · 2023-07-03T05:36:35Z

type l1t

emanueleusai · 2023-07-03T05:36:39Z

please test

cmsbuild · 2023-07-03T06:58:52Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-76acd3/33511/summary.html
COMMIT: f72c3c0
CMSSW: CMSSW_13_1_X_2023-07-02-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42167/33511/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

10224.0

----- Begin Fatal Exception 03-Jul-2023 08:17:45 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root
      [c] Input file root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.�
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root?tried=+1213xrootd-cms-redir-int.cr.cnaf.infn.it,&xrdcl.requuid=20e17530-10e4-4eac-a176-87c581214307
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

23.023.0_JpsiMM/step2_JpsiMM.log
134.806134.806_RunMuonEG2015C/step2_RunMuonEG2015C.log
4.294.29_RunMinBias2011B/step2_RunMinBias2011B.log

Expand to see more relval errors ...

perrotta · 2023-07-04T05:16:59Z

backport of #42155

emanueleusai · 2023-07-05T07:11:54Z

please test

cmsbuild · 2023-07-05T09:50:22Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-76acd3/33554/summary.html
COMMIT: f72c3c0
CMSSW: CMSSW_13_1_X_2023-07-04-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42167/33554/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 5 lines to the logs
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3222791
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3222763
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1129.1800000000005 KiB( 47 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 56.459 KiB L1T/L1TStage2EMTF
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

emanueleusai · 2023-07-07T05:01:05Z

testing 13_0 backport at p5

emanueleusai · 2023-07-11T06:00:20Z

+1

p5 test ok

emanueleusai · 2023-08-16T22:41:22Z

please test

cmsbuild · 2023-08-17T03:39:29Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-76acd3/34331/summary.html
COMMIT: 77a492b
CMSSW: CMSSW_13_1_X_2023-08-16-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42167/34331/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 22 lines from the logs
Reco comparison results: 13 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3186945
DQMHistoTests: Total failures: 12
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3186911
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1129.1800000000005 KiB( 47 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 56.459 KiB L1T/L1TStage2EMTF
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

cmsbuild · 2023-08-19T02:07:42Z

Pull request #42167 was updated. @nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please check and sign again.

emanueleusai · 2023-08-22T16:59:21Z

please test

emanueleusai · 2023-08-22T17:02:03Z

+1

last commit just removed comments, no changes expected in tests

cmsbuild · 2023-08-22T19:21:04Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-76acd3/34421/summary.html
COMMIT: 8742579
CMSSW: CMSSW_13_1_X_2023-08-22-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42167/34421/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 5 lines from the logs
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3186945
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3186920
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1129.1800000000005 KiB( 47 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 56.459 KiB L1T/L1TStage2EMTF
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

perrotta · 2023-08-23T21:00:28Z

unhold

perrotta · 2023-08-23T21:00:33Z

+1

Identical to the master

cmsbuild · 2023-08-23T21:00:49Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_3_X is complete. This pull request will be automatically merged.

lutuck added 2 commits June 30, 2023 13:58

Add Unconstrained Plots

378f821

Corrected formatting of plot names

f72c3c0

cmsbuild added this to the CMSSW_13_1_X milestone Jun 30, 2023

cmsbuild added dqm-pending pending-signatures tests-pending orp-pending labels Jun 30, 2023

cmsbuild added tests-started l1t and removed tests-pending labels Jul 3, 2023

cmsbuild added tests-rejected and removed tests-started labels Jul 3, 2023

cmsbuild added the backport label Jul 4, 2023

cmsbuild added tests-started and removed tests-rejected labels Jul 5, 2023

cmsbuild added tests-approved and removed tests-started labels Jul 5, 2023

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Jul 11, 2023

cmsbuild added tests-started and removed tests-pending labels Aug 16, 2023

cmsbuild added tests-approved and removed tests-started labels Aug 17, 2023

removed comments

8742579

cmsbuild added tests-pending and removed tests-approved labels Aug 19, 2023

cmsbuild added backport-ok and removed backport labels Aug 21, 2023

cmsbuild added tests-started and removed tests-pending labels Aug 22, 2023

cmsbuild added dqm-approved and removed dqm-pending labels Aug 22, 2023

cmsbuild added tests-approved and removed tests-started labels Aug 22, 2023

cmsbuild added fully-signed orp-approved and removed pending-signatures orp-pending hold labels Aug 23, 2023

cmsbuild merged commit cfb756f into cms-sw:CMSSW_13_1_X Aug 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[13_1_X] EMTF add DQM plots for unconstrained pT and dxy #42167

[13_1_X] EMTF add DQM plots for unconstrained pT and dxy #42167

lutuck commented Jun 30, 2023 •

edited

Loading

cmsbuild commented Jun 30, 2023 •

edited

Loading

emanueleusai commented Jul 3, 2023

emanueleusai commented Jul 3, 2023

cmsbuild commented Jul 3, 2023

perrotta commented Jul 4, 2023

emanueleusai commented Jul 5, 2023

cmsbuild commented Jul 5, 2023

emanueleusai commented Jul 7, 2023

emanueleusai commented Jul 11, 2023

emanueleusai commented Aug 16, 2023

cmsbuild commented Aug 17, 2023

cmsbuild commented Aug 19, 2023

emanueleusai commented Aug 22, 2023

emanueleusai commented Aug 22, 2023

cmsbuild commented Aug 22, 2023

perrotta commented Aug 23, 2023

perrotta commented Aug 23, 2023

cmsbuild commented Aug 23, 2023

[13_1_X] EMTF add DQM plots for unconstrained pT and dxy #42167

[13_1_X] EMTF add DQM plots for unconstrained pT and dxy #42167

Conversation

lutuck commented Jun 30, 2023 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Jun 30, 2023 • edited Loading

emanueleusai commented Jul 3, 2023

emanueleusai commented Jul 3, 2023

cmsbuild commented Jul 3, 2023

RelVals

RelVals-INPUT

perrotta commented Jul 4, 2023

emanueleusai commented Jul 5, 2023

cmsbuild commented Jul 5, 2023

Comparison Summary

emanueleusai commented Jul 7, 2023

emanueleusai commented Jul 11, 2023

emanueleusai commented Aug 16, 2023

cmsbuild commented Aug 17, 2023

Comparison Summary

cmsbuild commented Aug 19, 2023

emanueleusai commented Aug 22, 2023

emanueleusai commented Aug 22, 2023

cmsbuild commented Aug 22, 2023

Comparison Summary

perrotta commented Aug 23, 2023

perrotta commented Aug 23, 2023

cmsbuild commented Aug 23, 2023

lutuck commented Jun 30, 2023 •

edited

Loading

cmsbuild commented Jun 30, 2023 •

edited

Loading