Modify RPC and GEM unpacker blocks in EMTF unpacker to match the new Run 3 format #39388

eyigitba · 2022-09-14T12:51:13Z

PR description:

This PR implements the necessary changes to unpack the new DAQ format that is implemented at P5 a couple of weeks ago. Our tests show that the new DAQ format significantly reduces EMTF dead time at very high PU scenarios. According to the new format 2 RPC (GEM) TPs are packed into 1 RPC (GEM) data block, thus reducing the total number of blocks in EMTF DAQ stream.

Modified EMTFBlockGEM.cc, EMTFBlockRPC.cc to add functionality to create 2 TPs per block depending on the firmware version. Changes in EMTFSetup.cc are made to pass firmware version to the RPC and GEM block unpackers.

A backport of this PR to 12_4_X and 12_5_X is required for upcoming collision runs.

PR validation:

Tested with unpacking data collected last week and running l1tstage2_dqm_sourceclient-live_cfg.py to confirm the DQM plots are as expected.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild · 2022-09-14T12:59:50Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39388/32092

This PR adds an extra 20KB to repository

cmsbuild · 2022-09-14T13:00:19Z

A new Pull Request was created by @eyigitba for master.

It involves the following packages:

EventFilter/L1TRawToDigi (l1)

@epalencia, @cmsbuild, @cecilecaillol, @rekovic can you please review it and eventually sign? Thanks.
@dinyar, @missirol, @thomreis, @Martin-Grunewald this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cecilecaillol · 2022-09-14T13:01:56Z

please test

epalencia · 2022-09-14T13:02:43Z

Please test

rovere · 2022-09-14T13:05:35Z

Since this is needed for data taking after TS1 and LHC recovery, please open a backport to 12_4 and 12_5.
On top of that, to make it even more explicit that this is needed for the coming data-taking, please mark the 12_4 backport as urgent.

Marco(ORM)

eyigitba · 2022-09-14T13:07:25Z

urgent

cmsbuild · 2022-09-14T16:55:43Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-36c7f7/27531/summary.html
COMMIT: 1844fd0
CMSSW: CMSSW_12_6_X_2022-09-13-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39388/27531/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3618326
DQMHistoTests: Total failures: 5
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3618299
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 49 edm output root files, 51 DQM output files
TriggerResults: no differences found

cecilecaillol · 2022-09-14T17:10:30Z

+l1

cmsbuild · 2022-09-14T17:10:50Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-09-14T21:30:03Z

EventFilter/L1TRawToDigi/plugins/implementations_stage2/EMTFBlockRPC.cc

@@ -55,6 +55,15 @@ namespace l1t {
        uint16_t RPCc = payload[2];
        uint16_t RPCd = payload[3];

+        // Run 3 has a different EMTF DAQ output format since August 26th


What is the purpose of this commented out block? Please either add a comment which explains its usage, or remove it

This was my bad. I forgot to remove that part :). Should be fixed now

cmsbuild · 2022-09-15T07:06:28Z

Pull request #39388 was updated. @epalencia, @cmsbuild, @cecilecaillol, @rekovic can you please check and sign again.

cecilecaillol · 2022-09-15T07:31:09Z

please test

cecilecaillol · 2022-09-15T07:31:15Z

+l1

cmsbuild · 2022-09-15T07:31:29Z

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2022-09-15T11:15:27Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-36c7f7/27550/summary.html
COMMIT: 664c88a
CMSSW: CMSSW_12_6_X_2022-09-14-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39388/27550/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3618326
DQMHistoTests: Total failures: 11
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3618293
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 49 edm output root files, 51 DQM output files
TriggerResults: no differences found

perrotta · 2022-09-15T13:52:59Z

+1

perrotta · 2022-09-16T06:37:10Z

@eyigitba I am notincing now that this PR reverts the protections previously added with #38961: is it wanted, or simply you developed in 12_4_X (where that PR was never backported) and copied blindly the same files in 12_5_X and 12_6_X?

I'm holding the backports until this issue is addressed

eyigitba · 2022-09-16T07:11:37Z

@perrotta I can see the protections are still there at https://github.com/cms-sw/cmssw/pull/39388/files#diff-1a6dadb65e6c79d8a2fc124b34a859b8e8536d39e2cdfea6d7d9f05eba4a3bddR262-R269

Are you talking about these?

perrotta · 2022-09-16T08:38:18Z

Ah, ok, thank you for having checked. The confusion was caused by the fact that PR #38961 was never backported in 12_4_X, as its modifications are only there in 12_5_X and 12_6_X

That PR adds the protections in the following files:

EventFilter/L1TRawToDigi/plugins/implementations_stage2/EMTFBlockGEM.cc (*)
EventFilter/L1TRawToDigi/plugins/implementations_stage2/EMTFBlockME.cc
EventFilter/L1TRawToDigi/plugins/implementations_stage2/EMTFBlockRPC.cc (*)
EventFilter/L1TRawToDigi/plugins/implementations_stage2/EMTFBlockSP.cc

The backport of this PR adds those protections in EMTFBlockGEM.cc and EMTFBlockRPC.cc, but as such there is no protection for EMTFBlockME.cc and EMTFBlockSP.cc

I think that they could also be needed in 12_4_X, and therefore a partial backport of #38961 with what's missing should be prepared for 12_4_X: @eyigitba what do you think?

eyigitba · 2022-09-16T08:45:51Z

@perrotta I think that's a good point. I can prepare a partial backport of #38961.

These protections were only made to protect the DQM code from out-of-range BX values that happened in 2021. The DQM PR is not in 12_4_X, so it's probably not urgent. But it's good to have anyway for consistency

Modify RPC and GEM unpacker blocks to match the new Run 3 format

1844fd0

cmsbuild added this to the CMSSW_12_6_X milestone Sep 14, 2022

cmsbuild added code-checks-pending l1-pending orp-pending pending-signatures tests-pending labels Sep 14, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 14, 2022

cmsbuild added tests-started and removed tests-pending labels Sep 14, 2022

eyigitba mentioned this pull request Sep 14, 2022

[12_4_X] Modify RPC and GEM unpacker blocks in EMTF unpacker to match the new Run 3 format #39389

Merged

cmsbuild added the urgent label Sep 14, 2022

eyigitba mentioned this pull request Sep 14, 2022

[12_5_X] Modify RPC and GEM unpacker blocks in EMTF unpacker to match the new Run 3 format #39390

Merged

cmsbuild added tests-approved and removed tests-started labels Sep 14, 2022

cmsbuild added fully-signed l1-approved and removed l1-pending pending-signatures labels Sep 14, 2022

perrotta reviewed Sep 14, 2022

View reviewed changes

cmsbuild added fully-signed l1-approved tests-started and removed l1-pending pending-signatures tests-pending labels Sep 15, 2022

cmsbuild added tests-approved and removed tests-started labels Sep 15, 2022

cmsbuild added orp-approved and removed orp-pending labels Sep 15, 2022

cmsbuild merged commit f3954b5 into cms-sw:master Sep 15, 2022

cmsbuild mentioned this pull request Sep 15, 2022

Add wfs for JetMET PD #39405

Closed

eyigitba mentioned this pull request Sep 16, 2022

[12_4_X] Added protection against out-of-range BX values in EMTF unpacker blocks #39413

Merged

makortel mentioned this pull request Sep 20, 2022

Crashes in workflow 39434.911 #39445

Closed

This was referenced Sep 20, 2022

Bug fix in EMTF GEM and RPC unpacker blocks #39460

Merged

[12_5_X] Bug fix in EMTF GEM and RPC unpacker blocks #39461

Merged

[12_4_X] Bug fix in EMTF GEM and RPC unpacker blocks #39462

Merged

eyigitba mentioned this pull request Oct 24, 2022

Modify RPC and GEM unpacker blocks in EMTF unpacker to match the new Run 3 format cms-l1t-offline/cmssw#1054

Merged

This was referenced Feb 16, 2023

Fix RPC and GEM TP unpacking in EMTF unpacker #40797

Merged

[13_0_X] Fix RPC and GEM TP unpacking in EMTF unpacker #40836

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Modify RPC and GEM unpacker blocks in EMTF unpacker to match the new Run 3 format #39388

Modify RPC and GEM unpacker blocks in EMTF unpacker to match the new Run 3 format #39388

eyigitba commented Sep 14, 2022 •

edited

Loading

cmsbuild commented Sep 14, 2022

cmsbuild commented Sep 14, 2022

cecilecaillol commented Sep 14, 2022

epalencia commented Sep 14, 2022

rovere commented Sep 14, 2022

eyigitba commented Sep 14, 2022

cmsbuild commented Sep 14, 2022

cecilecaillol commented Sep 14, 2022

cmsbuild commented Sep 14, 2022

perrotta Sep 14, 2022

eyigitba Sep 15, 2022

cmsbuild commented Sep 15, 2022

cecilecaillol commented Sep 15, 2022

cecilecaillol commented Sep 15, 2022

cmsbuild commented Sep 15, 2022

cmsbuild commented Sep 15, 2022

perrotta commented Sep 15, 2022

perrotta commented Sep 16, 2022

eyigitba commented Sep 16, 2022

perrotta commented Sep 16, 2022

eyigitba commented Sep 16, 2022

Modify RPC and GEM unpacker blocks in EMTF unpacker to match the new Run 3 format #39388

Modify RPC and GEM unpacker blocks in EMTF unpacker to match the new Run 3 format #39388

Conversation

eyigitba commented Sep 14, 2022 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Sep 14, 2022

cmsbuild commented Sep 14, 2022

cecilecaillol commented Sep 14, 2022

epalencia commented Sep 14, 2022

rovere commented Sep 14, 2022

eyigitba commented Sep 14, 2022

cmsbuild commented Sep 14, 2022

Comparison Summary

cecilecaillol commented Sep 14, 2022

cmsbuild commented Sep 14, 2022

perrotta Sep 14, 2022

Choose a reason for hiding this comment

eyigitba Sep 15, 2022

Choose a reason for hiding this comment

cmsbuild commented Sep 15, 2022

cecilecaillol commented Sep 15, 2022

cecilecaillol commented Sep 15, 2022

cmsbuild commented Sep 15, 2022

cmsbuild commented Sep 15, 2022

Comparison Summary

perrotta commented Sep 15, 2022

perrotta commented Sep 16, 2022

eyigitba commented Sep 16, 2022

perrotta commented Sep 16, 2022

eyigitba commented Sep 16, 2022

eyigitba commented Sep 14, 2022 •

edited

Loading