[14_0_X Backport] Unpacker and DataFormat for L1 Trigger Scouting BMTF source

[Mmiglio]

PR description:

This PR introduces the unpacker and data format needed to process muon stubs collected with the L1 Scouting system from the BMTF Trigger processors.

With respect to the PR in the master branch, I removed from the test script the part testing data format with files from cms-data as they are not available for this release.

PR validation:

PR unpacker has been validated with data collected at p5 with the L1 scouting system, running in local, and comparing data results with L1 Trigger objects reconstructed starting from the cdaq Raw data

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #45262
Reason: collect data from this Trigger Source for the remaining of 2024 pp run.
Adding @epalencia and @aloeliger for the L1 Trigger.

[cmsbuild]

A new Pull Request was created by @Mmiglio for CMSSW_14_0_X.

It involves the following packages:

• DataFormats/L1Scouting (daq)
• EventFilter/L1ScoutingRawToDigi (daq)

@emeschi, @cmsbuild, @smorovic can you please review it and eventually sign? Thanks.
@mmusich, @dinyar, @Martin-Grunewald, @rovere, @missirol this is something you requested to watch as well.
@antoniovilela, @sextonkennedy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

cms-bot internal usage

[smorovic]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a9dc8b/39988/summary.html
COMMIT: 0a9b396
CMSSW: CMSSW_14_0_X_2024-06-19-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45273/39988/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 98 lines to the logs
• Reco comparison results: 144 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3342458
• DQMHistoTests: Total failures: 2736
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3339702
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 202 log files, 165 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[smorovic]

+daq
The only DataFormats touched are from L1 scouting. With unit test stripped from this version, it doesn't need external.
Note: if this is integrated into specific 14_0 release, Tier0, L1Scouting and HLT should all switch to that version at the same time.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_14_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_14_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[rappoccio]

+1

[srimanob]

@davidlange6

Could you please clarify from yesterday ORP that we need to roll back the PR before making CMSSW_14_0_10? Thanks.

[davidlange6]

hi @srimanob - i was involved only as ORM. (and this info came from the previous week, but was agreed by all in the ops meetings the last few weeks). I guess not rolling back just means the release won't be soon used online. Perhaps that was the plan from the ORP side?

[antoniovilela]

hi @srimanob - i was involved only as ORM. (and this info came from the previous week, but was agreed by all in the ops meetings the last few weeks). I guess not rolling back just means the release won't be soon used online. Perhaps that was the plan from the ORP side?

@srimanob @cms-sw/hlt-l2 @cms-sw/daq-l2 @cms-sw/orp-l2
It is still not clear to me how soon a release is needed at Tier-0 or HLT.
The requested release was for HLT menu development, however #45324 was not yet signed.

[mmusich]

It is still not clear to me how soon a release is needed at Tier-0 or HLT.

The TSG request hasn't changed with respect to yesterday meeting. ORP decided to build CMSSW_14_0_10 without #45324 (unclear to me which purpose it is supposed to serve).
In any case I think it should be fine to move (at least) Tier0 to CMSSW_14_0_10 (and profit of bugfixes for minor crashes that happened before the TS and possible problems with memory consumption seems the PR wasn't backported) as Tier-0 doesn't run any unpacker of the L1 Scouting, just repacking of the collections written at P5.
We should simply make sure that L1T scouting doesn't produce new collections from this PR (or at least not send them to Tier0) until we change the era.
@fwyzard @missirol .

[fwyzard]

We should simply make sure that L1T scouting doesn't produce new collections from this PR (or at least not send them to Tier0) until we change the era.

This is the same conclusion we reached with @Mmiglio yesterday evening.

[srimanob]

It is still not clear to me how soon a release is needed at Tier-0 or HLT.

The TSG request hasn't changed with respect to yesterday meeting. ORP decided to build CMSSW_14_0_10 without #45324 (unclear to me which purpose it is supposed to serve). In any case I think it should be fine to move (at least) Tier0 to CMSSW_14_0_10 (and profit of bugfixes for minor crashes that happened before the TS and possible problems with memory consumption) as Tier-0 doesn't run any unpacker of the L1 Scouting, just repacking of the collections written at P5. We should simply make sure that L1T scouting doesn't produce new collections from this PR (or at least not send them to Tier0) until we change the era. @fwyzard @missirol .

Hi, I think the proposal to build 14_0_10 first is to get benefit of bug fixes, and Tier-0 can do replay and move if need. With my understanding from above conversation, Tier-0 is fine to use this and benefit from bug fixes. And we are able to use this release also when we will change the era. Of couse, new release will be required, if Mahi alpaka comes in the same time of new era.

[aloeliger]

It is still not clear to me how soon a release is needed at Tier-0 or HLT.

The TSG request hasn't changed with respect to yesterday meeting. ORP decided to build CMSSW_14_0_10 without #45324 (unclear to me which purpose it is supposed to serve). In any case I think it should be fine to move (at least) Tier0 to CMSSW_14_0_10 (and profit of bugfixes for minor crashes that happened before the TS and possible problems with memory consumption) as Tier-0 doesn't run any unpacker of the L1 Scouting, just repacking of the collections written at P5. We should simply make sure that L1T scouting doesn't produce new collections from this PR (or at least not send them to Tier0) until we change the era. @fwyzard @missirol .

Hi, I think the proposal to build 14_0_10 first is to get benefit of bug fixes, and Tier-0 can do replay and move if need. With my understanding from above conversation, Tier-0 is fine to use this and benefit from bug fixes. And we are able to use this release also when we will change the era.

I think this is my understanding as well from the joint ops meeting yesterday as well. Tier 0 would like this new data-format to be available in a release first, to switch to and test, and then other systems can follow suit. Again, this is my understanding.

[antoniovilela]

It is still not clear to me how soon a release is needed at Tier-0 or HLT.

The TSG request hasn't changed with respect to yesterday meeting. ORP decided to build CMSSW_14_0_10 without #45324 (unclear to me which purpose it is supposed to serve). In any case I think it should be fine to move (at least) Tier0 to CMSSW_14_0_10 (and profit of bugfixes for minor crashes that happened before the TS and possible problems with memory consumption seems the PR wasn't backported) as Tier-0 doesn't run any unpacker of the L1 Scouting, just repacking of the collections written at P5. We should simply make sure that L1T scouting doesn't produce new collections from this PR (or at least not send them to Tier0) until we change the era. @fwyzard @missirol .

@mmusich @fwyzard @srimanob
After the ORP meeting, I had a discussion with Phat, who insisted on 14_0_10 before waiting for the missing PR, for the reasons he wrote. And he actually said he talked to Marco (if I was not lost in translation). I raised the concern that it was not clear the release would actually be deployed, but we agreed on building it.

[mmusich]

And he actually said he talked to Marco (if I was not lost in translation)

That's correct. For the record the reasons for CMSSW_14_0_10 were not discussed. I just stated that as long as we could have soon a release including #45324 as well, such an intermediate release wasn't terribly relevant for TSG but we would not be against it.

[antoniovilela]

And he actually said he talked to Marco (if I was not lost in translation)

That's correct. For the record the reasons for CMSSW_14_0_10 were not discussed. I just stated that as long as we could have soon a release including #45324 as well, such an intermediate release wasn't terribly relevant for TSG but we would not be against it.

Ok. I have warned that 14_0_10 might not be relevant for operations, and it was not my preference, but agreed to build it. Also 14_0_11 once the PR is merged. I understand a full release is needed for this PR.

[fwyzard]

Unfortunately yes, because it includes new DataFormats .