EMTF add DQM plots for unconstrained pT and dxy #42155

lutuck · 2023-06-29T17:51:48Z

PR description:

This PR adds DQM plots using unconstrained pT and dxy. These plots will be useful for monitoring now that the neural network is working at P5.

This PR will be back-ported to 13_0_X and 13_1_X to be used online.

PR validation:

Validated by runTheMatrix.py -l limited -i all --ibeos and everything looks good.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild · 2023-06-29T17:57:52Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42155/36144

This PR adds an extra 24KB to repository

emanueleusai · 2023-07-03T05:36:13Z

type l1t

emanueleusai · 2023-07-03T05:36:18Z

please test

cmsbuild · 2023-07-03T05:36:39Z

A new Pull Request was created by @lutuck (Lucas Faria de Sá Tucker) for master.

It involves the following packages:

DQM/L1TMonitor (dqm)

@nothingface0, @emanueleusai, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@missirol this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2023-07-03T06:57:03Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3f4d2d/33510/summary.html
COMMIT: ada07ca
CMSSW: CMSSW_13_2_X_2023-07-02-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42155/33510/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

10224.0

----- Begin Fatal Exception 03-Jul-2023 08:11:43 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root
      [c] Input file root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.�
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root?tried=+1213xrootd-redic.pi.infn.it,&xrdcl.requuid=7b465282-3733-4303-9016-5a8935a568c4
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

23.023.0_JpsiMM/step2_JpsiMM.log
134.806134.806_RunMuonEG2015C/step2_RunMuonEG2015C.log
134.808134.808_RunSingleMuPrpt2015C/step2_RunSingleMuPrpt2015C.log

Expand to see more relval errors ...

perrotta · 2023-07-04T05:15:51Z

please test

cmsbuild · 2023-07-04T07:50:00Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3f4d2d/33536/summary.html
COMMIT: ada07ca
CMSSW: CMSSW_13_2_X_2023-07-03-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42155/33536/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 3 lines from the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3193832
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3193807
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1129.1800000000005 KiB( 47 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 56.459 KiB L1T/L1TStage2EMTF
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

emanueleusai · 2023-07-05T07:12:24Z

+1

cmsbuild · 2023-08-19T02:37:00Z

Pull request #42155 was updated. @nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please check and sign again.

perrotta · 2023-08-19T06:02:35Z

@lutuck I still see some unanswered comment in https://github.com/cms-sw/cmssw/pull/42155/files
Can't you also see them?

cmsbuild · 2023-08-19T12:58:13Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42155/36660

This PR adds an extra 20KB to repository

cmsbuild · 2023-08-19T12:58:38Z

Pull request #42155 was updated. @nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please check and sign again.

syuvivida · 2023-08-21T08:43:51Z

+1

cmsbuild · 2023-08-21T08:44:17Z

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2023-08-21T14:07:02Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3f4d2d/34372/summary.html
COMMIT: 1961347
CMSSW: CMSSW_13_3_X_2023-08-20-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42155/34372/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 11 lines to the logs
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3152915
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3152887
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1129.1800000000005 KiB( 47 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 56.459 KiB L1T/L1TStage2EMTF
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

antoniovilela · 2023-08-21T15:38:11Z

+1

syuvivida · 2023-09-19T20:56:04Z

@lutuck
We don't seem to find the 13_2_X version of this PR. Could you create a backport to 13_2_X? For the time being, we are adding this PR on top of 13_2_3 and 13_2_4 but it would be easier for DQM if this PR is merged in one of the 13_2_X releases in the coming future. Thanks!!

eyigitba · 2023-09-21T16:41:49Z

@syuvivida , I think @lutuck won't be able to help with this. I can handle the backport to 13_2_X tomorrow.

lutuck added 2 commits June 28, 2023 22:07

Add Unconstrained Plots

8da5ba4

Corrected formatting of plot names

ada07ca

cmsbuild added this to the CMSSW_13_2_X milestone Jun 29, 2023

cmsbuild added dqm-pending pending-signatures tests-pending orp-pending code-checks-pending labels Jun 29, 2023

This was referenced Jun 30, 2023

[13_1_X] EMTF add DQM plots for unconstrained pT and dxy #42167

Merged

[13_0_X] EMTF add DQM plots for unconstrained pT and dxy #42168

Merged

cmsbuild added tests-started code-checks-approved l1t and removed tests-pending code-checks-pending labels Jul 3, 2023

cmsbuild added tests-rejected and removed tests-started labels Jul 3, 2023

cmsbuild added tests-started and removed tests-rejected labels Jul 4, 2023

cmsbuild added tests-approved and removed tests-started labels Jul 4, 2023

cmsbuild removed the dqm-pending label Jul 5, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 19, 2023

removed additional comments

1961347

cmsbuild added code-checks-pending and removed code-checks-approved labels Aug 19, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 19, 2023

cmsbuild added dqm-approved fully-signed tests-started and removed dqm-pending pending-signatures tests-pending labels Aug 21, 2023

cmsbuild added tests-approved and removed tests-started labels Aug 21, 2023

cmsbuild added orp-approved and removed orp-pending labels Aug 21, 2023

cmsbuild merged commit 505e9de into cms-sw:master Aug 21, 2023

cmsbuild mentioned this pull request Aug 22, 2023

FKDTree_t: avoid Wmaybe-uninitialized warning in CPP20 builds #42626

Merged

lutuck mentioned this pull request Sep 26, 2023

[13_2_X] EMTF add DQM plots for unconstrained pT and dxy #42864

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EMTF add DQM plots for unconstrained pT and dxy #42155

EMTF add DQM plots for unconstrained pT and dxy #42155

lutuck commented Jun 29, 2023

cmsbuild commented Jun 29, 2023

emanueleusai commented Jul 3, 2023

emanueleusai commented Jul 3, 2023

cmsbuild commented Jul 3, 2023

cmsbuild commented Jul 3, 2023

perrotta commented Jul 4, 2023

cmsbuild commented Jul 4, 2023

emanueleusai commented Jul 5, 2023

cmsbuild commented Aug 19, 2023

perrotta commented Aug 19, 2023

cmsbuild commented Aug 19, 2023

cmsbuild commented Aug 19, 2023

syuvivida commented Aug 21, 2023

cmsbuild commented Aug 21, 2023

cmsbuild commented Aug 21, 2023

antoniovilela commented Aug 21, 2023

syuvivida commented Sep 19, 2023

eyigitba commented Sep 21, 2023

EMTF add DQM plots for unconstrained pT and dxy #42155

EMTF add DQM plots for unconstrained pT and dxy #42155

Conversation

lutuck commented Jun 29, 2023

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Jun 29, 2023

emanueleusai commented Jul 3, 2023

emanueleusai commented Jul 3, 2023

cmsbuild commented Jul 3, 2023

cmsbuild commented Jul 3, 2023

RelVals

RelVals-INPUT

perrotta commented Jul 4, 2023

cmsbuild commented Jul 4, 2023

Comparison Summary

emanueleusai commented Jul 5, 2023

cmsbuild commented Aug 19, 2023

perrotta commented Aug 19, 2023

cmsbuild commented Aug 19, 2023

cmsbuild commented Aug 19, 2023

syuvivida commented Aug 21, 2023

cmsbuild commented Aug 21, 2023

cmsbuild commented Aug 21, 2023

Comparison Summary

antoniovilela commented Aug 21, 2023

syuvivida commented Sep 19, 2023

eyigitba commented Sep 21, 2023