EMTF Primitive Conversion LUT update

[eyigitba]

PR description:

This PR adds options to use the new primitive conversion LUTs in the EMTF emulator. The LUTs were deployed at P5 on October 6th, but they are not used in the emulator yet.

This PR needs the LUTs in cms-data/L1Trigger-L1TMuon#23 to work.

We see improvement in EMTF performance at P5 with these new LUTs, so changes to muon efficiencies etc are expected when testing the re-emulation workflows from recent runs (after October 6th). Otherwise this PR shouldn't change anything in the emulator.

This PR should be backported to 12_4 if there is an opportunity to run this at P5 to improve data/emulator comparisons in DQM. If data taking with 12_5 is expected we should also backport it there.

This has no effect on MC production so MC campaigns do not matter for the purpose of this PR.

PR validation:

runTheMatrix.py -l limited -i all --ibeos has no failures. Also validated by unpacking one of the recent runs to compare LUT outputs and it matches.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

N/A

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39822/32690

• This PR adds an extra 16KB to repository

[cmsbuild]

A new Pull Request was created by @eyigitba for master.

It involves the following packages:

• L1Trigger/L1TMuonEndCap (l1)

@epalencia, @cmsbuild, @cecilecaillol, @rekovic can you please review it and eventually sign? Thanks.
@dinyar, @missirol, @JanFSchulte, @thomreis, @Martin-Grunewald this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[epalencia]

Please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2a40d1/28434/summary.html
COMMIT: 0c22da8
CMSSW: CMSSW_12_6_X_2022-10-21-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39822/28434/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 9 differences found in the comparisons
• Reco comparison had 6 failed jobs
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3378384
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3378353
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 201 log files, 48 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[cecilecaillol]

+l1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1

[eyigitba]

I want to mention that cms-data/L1Trigger-L1TMuon#23 needs to be merged as well for this to work with recently collected data.

The tests are not useful to catch a crash for this change since the firmware was deployed on October 6th and any data before that run will not trigger the changes.

[perrotta]

I want to mention that cms-data/L1Trigger-L1TMuon#23 needs to be merged as well for this to work with recently collected data.

The tests are not useful to catch a crash for this change since the firmware was deployed on October 6th and any data before that run will not trigger the changes.

Thank you for having reminded it @eyigitba

I think that some workflow, or maybe unit test, should be made available in order to test this with the data collected with the new firmware: does it mean that the cmssw software with this update included was never tested with the new firmware? This has to be clarified before we can think at a possible backport to the current data taking releases.

[eyigitba]

Hi @perrotta , I can provide a workflow to test this with the recent data collected.

I ran this test myself before submitting the PR and everything worked fine (the code uses the correct LUTs and matches the online eta/phi values). Since we had a similar issue before I wanted to be more careful this time.

[perrotta]

Hi @perrotta , I can provide a workflow to test this with the recent data collected.

Thank you @eyigitba , yest that would be useful
I think we can merge cms-data/L1Trigger-L1TMuon#23 now in master, but wait for that workflows before backporting this into previous cycles, as it is the intention as far as I understood

I ran this test myself before submitting the PR and everything worked fine (the code uses the correct LUTs and matches the online eta/phi values). Since we had a similar issue before I wanted to be more careful this time.

[mmusich]

at the Joint Operations meeting today, it was mentioned that a backport of this PR would be desirable in the next CMSSW_12_4_X release. Does such a backport exist yet?

[francescobrivio]

@eyigitba any update on opening the backports to 12_4_X and 12_5_X of this PR?
Cheers,
Francesco as ORM

[eyigitba]

Sorry, I couldn't do it yesterday. I'll submit the PRs and a workflow to test these soon.

[eyigitba]

Here is a simple PSET file that runs on recent data: https://github.com/eyigitba/cmssw/blob/d4b832735b26d45e7c74e6ec8a9f0195b1424df7/L1Trigger/L1TMuonEndCap/test/unittests/emtf_run3_test.py

In LogInfo one can see:

%MSG-i L1T:  L1TMuonEndCapTrackProducer:simEmtfDigisData  25-Oct-2022 15:21:04 CEST Run: 360820 Event: 129711316
EMTF updating conditions: pc_lut_ver: 3 pt_lut_ver: 8 fw_ver: 1664468309

which is the new settings added with this PR. When pc_lut_ver = 3 is set, we will need the new LUTs in cms-data/L1Trigger-L1TMuon#23.

Please let me know if this looks good to you.

I'll submit the other 2 PRs now.