Add CPPF DQM modules #38564

zhangcg123 · 2022-06-30T09:58:54Z

PR description:

Add CPPF online DQM modules L1TdeStage2CPPF and L1TStage2CPPF.
By comparing the CPPF emulator and unpacker outputs and putting some comparison histograms on the CMS online DQM GUI, shifters can monitor the status of the CPPF hardware.

https://twiki.cern.ch/twiki/bin/viewauth/CMS/CPPFDQMDevelopment
https://indico.cern.ch/event/1096936/contributions/4614848/attachments/2347544/4003548/Ram_CPPFUpdate_20211117.pdf

PR validation:

cmsRun DQM/Integration/python/clients/l1tstage2emulator_dqm_sourceclient-live_cfg.py unitTest=True

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

The PR will have to be backported to 10_4_X

cmsbuild · 2022-06-30T10:08:48Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38564/30811

This PR adds an extra 48KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/L1TMonitor/python/L1TStage2Emulator_cff.py modified in PR(s): Update of GEM & CSC trigger primitive emulators #38373
- File L1Trigger/Configuration/python/L1TRawToDigi_cff.py modified in PR(s): WIP: Unpack uGT testcrate data #37481

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38564/30811/code-checks.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38564/30811/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly
code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38564/30811/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38564/30811/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2022-06-30T10:38:33Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38564/30812

This PR adds an extra 72KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/L1TMonitor/python/L1TStage2Emulator_cff.py modified in PR(s): Update of GEM & CSC trigger primitive emulators #38373
- File L1Trigger/Configuration/python/L1TRawToDigi_cff.py modified in PR(s): WIP: Unpack uGT testcrate data #37481

cmsbuild · 2022-06-30T10:38:57Z

A new Pull Request was created by @zhangcg123 for master.

It involves the following packages:

DQM/L1TMonitor (dqm)
EventFilter/RPCRawToDigi (reconstruction)
L1Trigger/Configuration (l1)
L1Trigger/L1TMuonCPPF (l1)

@epalencia, @emanueleusai, @ahmad3213, @cmsbuild, @rekovic, @jfernan2, @clacaputo, @jpata, @pmandrik, @micsucmed, @cecilecaillol, @rvenditti can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @dinyar, @thomreis, @eyigitba this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

qliphy · 2022-07-01T00:53:19Z

please test

cmsbuild · 2022-07-01T05:54:57Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-daadf4/25926/summary.html
COMMIT: ba76379
CMSSW: CMSSW_12_5_X_2022-06-30-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38564/25926/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 5 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3654771
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3654741
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 2092.8200000000006 KiB( 49 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 102.008 KiB L1TEMU/L1TdeStage2CPPF
DQMHistoSizes: changed ( 10024.0,... ): 2.633 KiB L1T/L1TStage2CPPF
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

emanueleusai · 2022-07-01T06:31:22Z

+1

zhangcg123 · 2022-07-04T07:52:11Z

@vukasinmilosevic

zhangcg123 · 2022-07-07T09:47:27Z

Hi @cecilecaillol, @epalencia, is this ok to go in

cecilecaillol · 2022-07-07T12:31:29Z

@zhangcg123 Can you please replace the couts with edmLogWarning/Error/... (https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideMessageLogger) ?

clacaputo · 2022-07-25T15:48:29Z

+reconstruction

re-sign after rebase

perrotta · 2022-07-26T12:21:48Z

@zhangcg123 if I compare with the version before your latest squash I get the following diff in DQM/L1TMonitor/python/L1TStage2Emulator_cff.py:

diff DQM/L1TMonitor/python/L1TStage2Emulator_cff.py $APREL\DQM/L1TMonitor/python/L1TStage2Emulator_cff.py
71,82d70
< from Configuration.Eras.Modifier_run3_common_cff import run3_common
< run3_common.toModify( valCscStage2Digis,
<                       keepShowers = True,
<                       commonParam = dict(run3 = True,
<                                          runCCLUT_OTMB = True,
<                                          runPhase2 = True,
<                                          runME11Up = True,
<                                          runME21Up = True,
<                                          runME31Up = True,
<                                          runME41Up = True)
< )
< 
84c72
< run3_GEM.toModify( valCscStage2Digis, GEMPadDigiClusterProducer = "valMuonGEMPadDigiClusters" , commonParam = dict(runME11ILT = True) )
---
> run3_GEM.toModify( valCscStage2Digis, GEMPadDigiClusterProducer = "valMuonGEMPadDigiClusters" )
103,104d90
< run3_GEM.toModify( valEmtfStage2Digis, UseRun3CCLUT_OTMB = cms.bool(True), Era = cms.string('Run3_2021'))
<

Could you please check?

zhangcg123 · 2022-07-27T10:40:28Z

@zhangcg123 if I compare with the version before your latest squash I get the following diff in DQM/L1TMonitor/python/L1TStage2Emulator_cff.py:

diff DQM/L1TMonitor/python/L1TStage2Emulator_cff.py $APREL\DQM/L1TMonitor/python/L1TStage2Emulator_cff.py
71,82d70
< from Configuration.Eras.Modifier_run3_common_cff import run3_common
< run3_common.toModify( valCscStage2Digis,
<                       keepShowers = True,
<                       commonParam = dict(run3 = True,
<                                          runCCLUT_OTMB = True,
<                                          runPhase2 = True,
<                                          runME11Up = True,
<                                          runME21Up = True,
<                                          runME31Up = True,
<                                          runME41Up = True)
< )
< 
84c72
< run3_GEM.toModify( valCscStage2Digis, GEMPadDigiClusterProducer = "valMuonGEMPadDigiClusters" , commonParam = dict(runME11ILT = True) )
---
> run3_GEM.toModify( valCscStage2Digis, GEMPadDigiClusterProducer = "valMuonGEMPadDigiClusters" )
103,104d90
< run3_GEM.toModify( valEmtfStage2Digis, UseRun3CCLUT_OTMB = cms.bool(True), Era = cms.string('Run3_2021'))
<

Could you please check?

Hi @perrotta

The new squash was rebased on the top of the newest merged commits at that time. The diff you mentioned is here.

67f988c#diff-958876a581b9e879b842bdfcb4371f7bef929312c19e0d17a8e5c55484708c71

Thanks

perrotta · 2022-07-27T10:57:50Z

Thank you @zhangcg123 : then you confirm that the update 67f988c#diff-958876a581b9e879b842bdfcb4371f7bef929312c19e0d17a8e5c55484708c71 that is in this PR was already there in the previous one?
If you can please confirm so, this one can be merged as it is

cecilecaillol · 2022-07-27T10:59:47Z

+l1

zhangcg123 · 2022-07-27T13:54:21Z

Hi @perrotta

I think these changes is not from our side. When we created this PR and now there are couple of PR which is merged.

So, the change that you mention is not from our side. It is from the other PR.

If you check all the files that we changed here:

https://github.com/cms-sw/cmssw/pull/38564/files

This link does not show any additional line/file.

Let me know if I did not get your message.

perrotta · 2022-07-27T13:54:42Z

@perrotta , I have to say no, the update was not there before the PR was rebased. As you can see the update was merged into the master 8 days ago. When I was asked to squash my commits into one, I rebased this PR on the top of the newest master(the update had been merged). So, the update was included in this PR. It looks like I should not include the update into this PR? If so, sorry for that. and what I should do? Many thanks.

the update#67f988c#diff-958876a581b9e879b842bdfcb4371f7bef929312c19e0d17a8e5c55484708c71

Ah, ok, now I understand. You rebased on top of an older IB than the one in which #38373 was merged
github does not complain about possible conflicts, and this should be one of the cases in which git is smart enough to keep the updates separated, even if they are only a couple of lines far away each other.
We can have a look at that particular file, but I imagine everything will be smooth.

The squeeze was correct, then. And as soon as @cms-sw/dqm-l2 signs, this PR can get merged

zhangcg123 · 2022-07-27T13:58:42Z

@perrotta , we are also getting confused. Thanks for your explanation.

emanueleusai · 2022-07-28T02:37:14Z

+1

cmsbuild · 2022-07-28T02:37:37Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-07-28T04:41:20Z

+1

qliphy · 2022-07-31T01:07:51Z

@zhangcg123 There is a reval error appearing after merging this PR, please have a look and fix soon:

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc10/CMSSW_12_5_X_2022-07-30-1100/pyRelValMatrixLogs/run/136.8561_RunZeroBias_hBStarTk+RunZeroBias_hBStarTk+HLTDR2_2018_hBStar+RECODR2_2018reHLT_Offline_hBStar+HARVEST2018_hBStar/step3_RunZeroBias_hBStarTk+RunZeroBias_hBStarTk+HLTDR2_2018_hBStar+RECODR2_2018reHLT_Offline_hBStar+HARVEST2018_hBStar.log#/1072-1072

An exception of category 'OutOfRange' occurred while
[0] Processing Event run: 314890 lumi: 591 event: 497522872 stream: 0
[1] Running path 'dqmoffline_step'
[2] Prefetching for module L1TdeStage2CPPF/'l1tdeStage2Cppf'
[3] Calling method for module RPCAMCRawToDigi/'rpcCPPFRawToDigi'
Exception Message:
Out-of-range input for RPCAMCLink::bf_set, position 0: 113
----- End Fatal Exception -------------------------------------------------

qliphy · 2022-08-01T01:05:57Z

@zhangcg123 In case you don't know, you can reproduce the error with
runTheMatrix.py -l 136.8561 --job-reports -t 4 --ibeos

The test with CMSSW_12_5_X_2022-07-27-2300 (before your PR merged) is fine,
while not with CMSSW_12_5_X_2022-07-31-0000

zhangcg123 · 2022-08-01T04:26:44Z

@qliphy Many thanks for your instruction, I will try to fix the problem as soon as I can.

cmsbuild added this to the CMSSW_12_5_X milestone Jun 30, 2022

cmsbuild added code-checks-pending dqm-pending l1-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jun 30, 2022

cmsbuild added code-checks-rejected code-checks-pending and removed code-checks-pending code-checks-rejected labels Jun 30, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 30, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 1, 2022

cmsbuild added tests-approved and removed tests-started labels Jul 1, 2022

cmsbuild added dqm-approved and removed dqm-pending labels Jul 1, 2022

cmsbuild removed the dqm-approved label Jul 8, 2022

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Jul 25, 2022

cmsbuild added l1-approved and removed l1-pending labels Jul 27, 2022

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Jul 28, 2022

cmsbuild added orp-approved and removed orp-pending labels Jul 28, 2022

cmsbuild merged commit 9fb18d7 into cms-sw:master Jul 28, 2022

makortel mentioned this pull request Aug 2, 2022

Out of range exception from RPCAMCRawToDigi #38939

Closed

mileva mentioned this pull request Aug 9, 2022

Fix the delay of the CPPF DAQ data #38974

Merged

This was referenced Sep 5, 2022

A possible solution for RPCAMCLink out-of-range issue #39307

Merged

A possible solution for RPCAMCLink out-of-range issue[12_5_X] #39320

Merged

zhangcg123 mentioned this pull request Dec 7, 2022

CPPF DQM modules backport to 12_4_X #40258

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CPPF DQM modules #38564

Add CPPF DQM modules #38564

zhangcg123 commented Jun 30, 2022 •

edited

Loading

cmsbuild commented Jun 30, 2022

cmsbuild commented Jun 30, 2022

cmsbuild commented Jun 30, 2022

qliphy commented Jul 1, 2022

cmsbuild commented Jul 1, 2022

emanueleusai commented Jul 1, 2022

zhangcg123 commented Jul 4, 2022

zhangcg123 commented Jul 7, 2022

cecilecaillol commented Jul 7, 2022

clacaputo commented Jul 25, 2022

perrotta commented Jul 26, 2022

zhangcg123 commented Jul 27, 2022 •

edited

Loading

perrotta commented Jul 27, 2022

cecilecaillol commented Jul 27, 2022

zhangcg123 commented Jul 27, 2022

perrotta commented Jul 27, 2022 •

edited

Loading

zhangcg123 commented Jul 27, 2022

emanueleusai commented Jul 28, 2022

cmsbuild commented Jul 28, 2022

perrotta commented Jul 28, 2022

qliphy commented Jul 31, 2022

qliphy commented Aug 1, 2022

zhangcg123 commented Aug 1, 2022

Add CPPF DQM modules #38564

Add CPPF DQM modules #38564

Conversation

zhangcg123 commented Jun 30, 2022 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Jun 30, 2022

cmsbuild commented Jun 30, 2022

cmsbuild commented Jun 30, 2022

qliphy commented Jul 1, 2022

cmsbuild commented Jul 1, 2022

Comparison Summary

emanueleusai commented Jul 1, 2022

zhangcg123 commented Jul 4, 2022

zhangcg123 commented Jul 7, 2022

cecilecaillol commented Jul 7, 2022

clacaputo commented Jul 25, 2022

perrotta commented Jul 26, 2022

zhangcg123 commented Jul 27, 2022 • edited Loading

perrotta commented Jul 27, 2022

cecilecaillol commented Jul 27, 2022

zhangcg123 commented Jul 27, 2022

perrotta commented Jul 27, 2022 • edited Loading

zhangcg123 commented Jul 27, 2022

emanueleusai commented Jul 28, 2022

cmsbuild commented Jul 28, 2022

perrotta commented Jul 28, 2022

qliphy commented Jul 31, 2022

qliphy commented Aug 1, 2022

zhangcg123 commented Aug 1, 2022

zhangcg123 commented Jun 30, 2022 •

edited

Loading

zhangcg123 commented Jul 27, 2022 •

edited

Loading

perrotta commented Jul 27, 2022 •

edited

Loading