[11_2_X] EMTF Emulator Update to Add Displaced Muon pT & Dxy Assignment #32642

eyigitba · 2021-01-12T12:24:33Z

PR description:

This PR updates EMTF emulator by adding a new class (PtAssignmentEngineDxy) that handles vertex unconstrained pT and Dxy assignment. The changes should not affect anything for prompt muons. The main changes are:

Added PtAssignmentEngineDxy and necessary changes to the EMTF algorithm to support it.
Added setters and getters for vertex unconstrained pT and Dxy to EMTFTrack

PR validation:

Validated with:

scram b runtests
runTheMatrix.py -l limited -i all --ibeos

if this PR is a backport please specify the original PR and why you need to backport that PR:

This is a backport of PR #32813, which is done to include this update in Run 3 MC campaigns and upcoming MWGRs.

Notifications: @jiafulow

…ting EMTFTrack object to have the relevant setters and getters.

…ines.

…than the constructor in PtAssignmentEngineDxy

cmsbuild · 2021-01-12T12:25:02Z

A new Pull Request was created by @eyigitba for CMSSW_11_2_X.

It involves the following packages:

DataFormats/L1TMuon
L1Trigger/L1TMuonEndCap

@cmsbuild, @rekovic, @jmduarte can you please review it and eventually sign? Thanks.
@dinyar, @Martin-Grunewald, @rovere, @thomreis this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2021-01-12T12:54:10Z

Pull request #32642 was updated. @cmsbuild, @rekovic, @jmduarte can you please check and sign again.

rekovic · 2021-01-13T14:36:50Z

please remove the data file
L1Trigger/L1TMuonEndCap/data/emtfpp_tf_graphs/model_graph.displ.5.pb
and PRit to external data repostitory https://github.com/cms-l1t-offline/L1Trigger-L1TMuon

…the new data path.

cmsbuild · 2021-01-13T15:53:22Z

Pull request #32642 was updated. @cmsbuild, @rekovic, @jmduarte can you please check and sign again.

silviodonato · 2021-01-14T09:30:31Z

test parameters:

pull_request = Added protobuf file for the NN used in EMTF displaced muon pT assignment cms-l1t-offline/L1Trigger-L1TMuon#10

silviodonato · 2021-01-14T09:32:22Z

test parameters:

pull_request = cms-l1t-offline/L1Trigger-L1TMuon|Gathering "package" tags and putting the same "package" tag in several releases? #10

cmsbuild · 2021-01-19T15:36:49Z

Pull request #32642 was updated. @cmsbuild, @rekovic, @jmduarte can you please check and sign again.

mrodozov · 2021-01-19T15:49:42Z

the PR is on the fork of cms-data instead of that branch being used for a cms-data PR :)
it's a fork of a fork :D ok let me fix it after the meeting

mrodozov · 2021-01-20T09:04:05Z

test parameters:
pull_request = cms-data/L1Trigger-L1TMuon#19

mrodozov · 2021-01-20T09:04:30Z

looks like this is backport of #32641 on 11_2

mrodozov · 2021-01-20T09:04:47Z

please test

cmsbuild · 2021-01-20T12:08:13Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-53fc5c/12372/summary.html
COMMIT: 6c964d7
CMSSW: CMSSW_11_2_X_2021-01-19-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/32642/12372/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 36
DQMHistoTests: Total histograms compared: 2529619
DQMHistoTests: Total failures: 88
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2529509
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 35 files compared)
Checked 151 log files, 37 edm output root files, 36 DQM output files

eyigitba · 2021-02-04T09:08:10Z

The main PR (#32641 ) had some thread safety issues in the IB. I am now fixing those with a new PR.

Should I update this PR directly or should I wait for it to be merged and submit another backport?

cmsbuild · 2021-02-04T10:06:41Z

Pull request #32642 was updated. @cmsbuild, @rekovic, @jmduarte can you please check and sign again.

mrodozov · 2021-02-04T10:18:00Z

I understand it needs the same data external + code changes from the last commit

mrodozov · 2021-02-04T10:18:10Z

please test

eyigitba · 2021-02-04T10:21:55Z

I understand it needs the same data external + code changes from the last commit

yes that's correct

cmsbuild · 2021-02-04T13:13:43Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-53fc5c/12707/summary.html
COMMIT: 67e5ed4
CMSSW: CMSSW_11_2_X_2021-02-03-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/32642/12707/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 36
DQMHistoTests: Total histograms compared: 2529619
DQMHistoTests: Total failures: 93
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2529503
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 35 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 151 log files, 37 edm output root files, 36 DQM output files

silviodonato · 2021-02-09T13:15:43Z

If you have updated the code to #32813, please update the PR description (ie. please remove #32641)

eyigitba · 2021-02-09T14:33:55Z

If you have updated the code to #32813, please update the PR description (ie. please remove #32641)

Sorry about that. I updated the PR description to point to the correct PR.

rekovic · 2021-02-10T13:53:52Z

+1

cmsbuild · 2021-02-10T13:54:14Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_11_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-02-11T13:04:10Z

+1

christopheralanwest · 2021-02-12T16:17:09Z

There are many failing relvals in the 11_2_X IBs, with errors such as:

----- Begin Fatal Exception 12-Feb-2021 01:54:37 CET-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 1
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Prefetching for module L1TDigiToRaw/'gmtStage2Raw'
   [5] Calling method for module L1TMuonEndCapTrackProducer/'simEmtfDigis'
Exception Message:
edm::FileInPath unable to find file L1Trigger/L1TMuon/data/emtf_luts/model_graph.displ.5.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /data/cmsbld/jenkins/workspace/ib-run-relvals/CMSSW_11_2_X_2021-02-11-2300/poison:/data/cmsbld/jenkins/workspace/ib-run-relvals/CMSSW_11_2_X_2021-02-11-2300/src:/data/cmsbld/jenkins/workspace/ib-run-relvals/CMSSW_11_2_X_2021-02-11-2300/external/cc8_amd64_gcc9/data:/cvmfs/cms-ib.cern.ch/nweek-02667/cc8_amd64_gcc9/cms/cmssw-patch/CMSSW_11_2_X_2021-02-11-2300/poison:/cvmfs/cms-ib.cern.ch/nweek-02667/cc8_amd64_gcc9/cms/cmssw-patch/CMSSW_11_2_X_2021-02-11-2300/src:/cvmfs/cms-ib.cern.ch/nweek-02667/cc8_amd64_gcc9/cms/cmssw-patch/CMSSW_11_2_X_2021-02-11-2300/external/cc8_amd64_gcc9/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-relvals/CMSSW_11_2_X_2021-02-11-2300/pyRelval/135.1_TTbar_13+TTbarFS_13+HARVESTUP15FS+MINIAODMCUP15FS
----- End Fatal Exception -------------------------------------------------

These are apparently a consequence of merging this PR. Is a fix underway?

eyigitba · 2021-02-12T16:44:52Z

This looks like the data file from cms-data/L1Trigger-L1TMuon#19 is missing somehow. I don't know why. @rekovic can we do something about this?

qliphy · 2021-02-13T02:21:34Z

@eyigitba @rekovic @silviodonato @smuzaffar It seems that we need to update 11_2 tag for L1Trigger-L1TMuon to V01-03-00: similarly as done in the PR for master: cms-sw/cmsdist#6616

smuzaffar · 2021-02-13T09:56:11Z

done here cms-sw/cmsdist#6646 and merged for next Ib

eyigitba added 5 commits November 24, 2020 15:46

Adding NNv5 for displaced pT and d0 assignment to EMTF emulator. Upda…

bd9a727

…ting EMTFTrack object to have the relevant setters and getters.

Added the NN data file

ecdce74

Fixed styling. Moved pbFileNameDxy option to config. Removed unused l…

d382216

…ines.

Moved configuration of graphDxy and sessionDxy to configure() rather …

b4f1f95

…than the constructor in PtAssignmentEngineDxy

Removed the old/unused NN protobuf file.

022b8cd

cmsbuild added this to the CMSSW_11_2_X milestone Jan 12, 2021

cmsbuild added l1-pending orp-pending pending-signatures tests-pending labels Jan 12, 2021

Added code checks and code format changes.

66aa0b0

Removed data directory as requested by Vladimir. Changed the path to …

b329ef4

…the new data path.

Changed data file path to match the PR to external data repository

6c964d7

cmsbuild added tests-started and removed tests-pending labels Jan 20, 2021

cmsbuild added tests-approved and removed tests-started labels Jan 20, 2021

Fixing thread safety issues caused by static calls.

67e5ed4

cmsbuild added tests-pending and removed tests-approved labels Feb 4, 2021

cmsbuild added tests-started and removed tests-pending labels Feb 4, 2021

eyigitba mentioned this pull request Feb 4, 2021

EMTF Emulator Update to Add Displaced Muon pT & Dxy Assignment #32813

Merged

cmsbuild added tests-approved and removed tests-started labels Feb 4, 2021

cmsbuild added fully-signed l1-approved and removed l1-pending pending-signatures labels Feb 10, 2021

cmsbuild added orp-approved and removed orp-pending labels Feb 11, 2021

cmsbuild merged commit 367ff03 into cms-sw:CMSSW_11_2_X Feb 11, 2021

qliphy mentioned this pull request Feb 13, 2021

Update tag for L1Trigger-L1TMuon to V01-03-00 cms-sw/cmsdist#6645

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[11_2_X] EMTF Emulator Update to Add Displaced Muon pT & Dxy Assignment #32642

[11_2_X] EMTF Emulator Update to Add Displaced Muon pT & Dxy Assignment #32642

eyigitba commented Jan 12, 2021 •

edited

Loading

cmsbuild commented Jan 12, 2021

cmsbuild commented Jan 12, 2021

rekovic commented Jan 13, 2021

cmsbuild commented Jan 13, 2021

silviodonato commented Jan 14, 2021

silviodonato commented Jan 14, 2021

cmsbuild commented Jan 19, 2021

mrodozov commented Jan 19, 2021 •

edited

Loading

mrodozov commented Jan 20, 2021

mrodozov commented Jan 20, 2021

mrodozov commented Jan 20, 2021

cmsbuild commented Jan 20, 2021

eyigitba commented Feb 4, 2021

cmsbuild commented Feb 4, 2021

mrodozov commented Feb 4, 2021

mrodozov commented Feb 4, 2021

eyigitba commented Feb 4, 2021

cmsbuild commented Feb 4, 2021

silviodonato commented Feb 9, 2021

eyigitba commented Feb 9, 2021

rekovic commented Feb 10, 2021

cmsbuild commented Feb 10, 2021

qliphy commented Feb 11, 2021

christopheralanwest commented Feb 12, 2021

eyigitba commented Feb 12, 2021

qliphy commented Feb 13, 2021

smuzaffar commented Feb 13, 2021

[11_2_X] EMTF Emulator Update to Add Displaced Muon pT & Dxy Assignment #32642

[11_2_X] EMTF Emulator Update to Add Displaced Muon pT & Dxy Assignment #32642

Conversation

eyigitba commented Jan 12, 2021 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jan 12, 2021

cmsbuild commented Jan 12, 2021

rekovic commented Jan 13, 2021

cmsbuild commented Jan 13, 2021

silviodonato commented Jan 14, 2021

silviodonato commented Jan 14, 2021

cmsbuild commented Jan 19, 2021

mrodozov commented Jan 19, 2021 • edited Loading

mrodozov commented Jan 20, 2021

mrodozov commented Jan 20, 2021

mrodozov commented Jan 20, 2021

cmsbuild commented Jan 20, 2021

Comparison Summary

eyigitba commented Feb 4, 2021

cmsbuild commented Feb 4, 2021

mrodozov commented Feb 4, 2021

mrodozov commented Feb 4, 2021

eyigitba commented Feb 4, 2021

cmsbuild commented Feb 4, 2021

Comparison Summary

silviodonato commented Feb 9, 2021

eyigitba commented Feb 9, 2021

rekovic commented Feb 10, 2021

cmsbuild commented Feb 10, 2021

qliphy commented Feb 11, 2021

christopheralanwest commented Feb 12, 2021

eyigitba commented Feb 12, 2021

qliphy commented Feb 13, 2021

smuzaffar commented Feb 13, 2021

eyigitba commented Jan 12, 2021 •

edited

Loading

mrodozov commented Jan 19, 2021 •

edited

Loading