Pr113x L1T Fractional Prescales implemented with Global Trigger #32741

rekovic · 2021-01-26T11:41:35Z

PR description:

Master branch PR of the 11_2_X PR #32377 which is needed for MWGR Feb 2021.

11_3_X: Use of fractional prescales in uGT emulator, by switching from legacy (int based) to new (double based) prescales. The new Conditions format is available in CondDB as L1TGlobalPrescalesVetosFractRcd, in the L1TGlobalPrescalesVetosFract_Stage2_v2_hlt tag.

Supplemented by changes in HLTrigger/HLTcore package

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

Before submitting your pull requests, make sure you followed this checklist:

verify that the PR is really intended for the chosen branch
verify that changes follow CMS Naming, Coding, And Style Rules
verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

cmsbuild · 2021-01-26T11:48:29Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32741/20892

This PR adds an extra 76KB to repository

cmsbuild · 2021-01-26T11:48:54Z

A new Pull Request was created by @rekovic for master.

It involves the following packages:

CommonTools/TriggerUtils
DQM/L1TMonitor
HLTrigger/HLTcore
L1Trigger/L1TGlobal

@cmsbuild, @andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @jmduarte, @Martin-Grunewald, @rekovic, @jfernan2, @fwyzard, @rvenditti can you please review it and eventually sign? Thanks.
@Martin-Grunewald this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

rekovic · 2021-01-26T11:49:07Z

please test

cmsbuild · 2021-01-26T13:59:54Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8640e/12542/summary.html
COMMIT: 2f038e3
CMSSW: CMSSW_11_3_X_2021-01-25-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32741/12542/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsPatAlgos had ERRORS

RelVals

136.731

----- Begin Fatal Exception 26-Jan-2021 14:25:34 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing global begin Run run: 274199
   [1] Calling method for module L1TGlobalSummary/'hltL1TGlobalSummary'
Exception Message:
No "L1TGlobalPrescalesVetosFractRcd" record found in the EventSetup.n
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

136.793

----- Begin Fatal Exception 26-Jan-2021 14:25:34 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing global begin Run run: 301998
   [1] Calling method for module L1TGlobalSummary/'hltL1TGlobalSummary'
Exception Message:
No "L1TGlobalPrescalesVetosFractRcd" record found in the EventSetup.n
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

136.874

----- Begin Fatal Exception 26-Jan-2021 14:25:36 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing global begin Run run: 319450
   [1] Calling method for module L1TGlobalSummary/'hltL1TGlobalSummary'
Exception Message:
No "L1TGlobalPrescalesVetosFractRcd" record found in the EventSetup.n
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

4.764.76_ZMuSkim2012D+ZMuSkim2012D+HLTDSKIM2+RECODR1reHLT2+HARVESTDR1reHLT/step2_ZMuSkim2012D+ZMuSkim2012D+HLTDSKIM2+RECODR1reHLT2+HARVESTDR1reHLT.log
136.722136.722_RunDoubleEG2016B+RunDoubleEG2016B+HLTDR2_2016+RECODR2_2016reHLT_skimDoubleEG_HIPM+HARVESTDR2/step2_RunDoubleEG2016B+RunDoubleEG2016B+HLTDR2_2016+RECODR2_2016reHLT_skimDoubleEG_HIPM+HARVESTDR2.log
136.723136.723_RunDoubleMuon2016B+RunDoubleMuon2016B+HLTDR2_2016+RECODR2_2016reHLT_HIPM+HARVESTDR2/step2_RunDoubleMuon2016B+RunDoubleMuon2016B+HLTDR2_2016+RECODR2_2016reHLT_HIPM+HARVESTDR2.log

Expand to see more relval errors ...

AddOn Tests

hlt_data_Fake2

----- Begin Fatal Exception 26-Jan-2021 14:20:08 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing global begin Run run: 272762
   [1] Calling method for module L1TGlobalSummary/'L1TGlobalSummary'
Exception Message:
No "L1TGlobalPrescalesVetosFractRcd" record found in the EventSetup.n
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

hlt_data_PRef

----- Begin Fatal Exception 26-Jan-2021 14:20:06 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing global begin Run run: 323775
   [1] Calling method for module L1TGlobalSummary/'L1TGlobalSummary'
Exception Message:
No "L1TGlobalPrescalesVetosFractRcd" record found in the EventSetup.n
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

hlt_data_HIon

----- Begin Fatal Exception 26-Jan-2021 14:20:10 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing global begin Run run: 325112
   [1] Calling method for module L1TGlobalSummary/'L1TGlobalSummary'
Exception Message:
No "L1TGlobalPrescalesVetosFractRcd" record found in the EventSetup.n
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

christopheralanwest · 2021-01-26T20:29:18Z

I've encountered a problem in trying to prepare the GTs and I need advice from @cms-sw/hlt-l2 regarding how to proceed. Some of the addOnTests use the run2_hlt_relval and run2_hlt_relval_hi GTs defined at https://github.com/cms-sw/cmssw/blob/master/Configuration/AlCa/python/autoCond.py#L43-L44. These GT are based on the 10_3_X release and so I can't add payloads of type L1TGlobalPrescalesVetosFractRcd both because the 10_3_X CMSSW release does not contain the relevant record and because the boost version used by the CMSSW in which L1TGlobalPrescalesVetosFract_Stage2_v2_hlt was serialized (1.72) is newer than that used by 10_3_X (1.63.0).

Should the run2_hlt_relval and run2_hlt_relval_hi GTs be updated to use a GT from a release series that is currently used by the HLT (11_2_X), or is intended that these GTs are nominally frozen as some sort of means of ensuring compatibility between the HLT actually run at P5 and newer releases? If there is any documentation somewhere that describes how the addOnTests are used in general, that would be very helpful.

Martin-Grunewald · 2021-01-27T07:52:24Z

Can one make compatible GTs by re-serialising the record with the different boost?

Martin-Grunewald · 2021-01-27T07:53:43Z

The addOnTests are to test that everything is in working order at PR test time. They are supposed to reflect any physics/MC production in/for that release.

Martin-Grunewald · 2021-01-27T09:30:44Z

IOW, if the GTs can be updated (without changing the physics content), then this should be OK and also future proof.

rekovic · 2021-02-01T13:40:04Z

+1

silviodonato · 2021-02-01T14:07:04Z

@cms-sw/hlt-l2 do you have further comments?

Martin-Grunewald · 2021-02-01T14:41:42Z

There is still a flag 'tests-rejected'
??

Martin-Grunewald · 2021-02-01T14:42:20Z

please test

Martin-Grunewald · 2021-02-01T14:43:16Z

Rerun tests....

christopheralanwest · 2021-02-01T15:01:41Z

There is still a flag 'tests-rejected'
??

This PR was successfully tested with PR #32758 but apparently the bot was changed recently so that when two PRs are tested together, notifications are sent only to the PR in which the test command was issued, and similarly for the status labels. I don't know if this was a design decision or if it is a bug. @smuzaffar Could you comment?

fwyzard · 2021-02-01T15:45:37Z

please test with #32758

smuzaffar · 2021-02-01T15:55:09Z

@christopheralanwest , it is design decision. Each PR needs to be tested explicitly

silviodonato · 2021-02-01T21:49:34Z

please test with #32758
It seems there was a glitch during the test https://cmssdt.cern.ch/jenkins/job/compare-root-files-short-matrix/41047/console

cmsbuild · 2021-02-02T00:35:58Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8640e/12634/summary.html
COMMIT: 2f038e3
CMSSW: CMSSW_11_3_X_2021-02-01-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/32741/12634/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

There are some workflows for which there are errors in the baseline:
312.0 step 1
1000.0 step 2
1001.0 step 2
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

You potentially added 1481 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2579163
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2579142
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 143 log files, 37 edm output root files, 34 DQM output files

Martin-Grunewald · 2021-02-02T07:13:24Z

+1

cmsbuild · 2021-02-02T07:13:42Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2021-02-02T08:12:17Z

+1

makortel · 2021-02-02T19:10:56Z

This PR likely broke two unit tests in DQM/Integration, see #32796.

panoskatsoulis and others added 6 commits January 26, 2021 12:28

changes in L1TGlobalUtil.h for frac-ps

748b04f

fixed dependencies for l1t GlobalUtils

5f9dd64

Use frac prescoles in GlobUtils and uGT emulator.

f578d06

code format.

624f954

prescale column is an unsigned int - revert developer's change.

2d2109d

changing to new HLT prescale interface as suggested by A. Bocci

2f038e3

cmsbuild added this to the CMSSW_11_3_X milestone Jan 26, 2021

cmsbuild added code-checks-pending dqm-pending hlt-pending l1-pending orp-pending pending-signatures tests-pending labels Jan 26, 2021

rekovic mentioned this pull request Jan 26, 2021

Pr112x L1T Fractional Prescales implemented with Global Trigger #32377

Closed

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 26, 2021

cmsbuild added tests-started and removed tests-pending labels Jan 26, 2021

cmsbuild added tests-rejected and removed tests-started labels Jan 26, 2021

rekovic mentioned this pull request Jan 26, 2021

Pr112x Fractional Prescales L1T + HLT and L1T uGT displaced muons #32744

Merged

cmsbuild added l1-approved and removed l1-pending labels Feb 1, 2021

cmsbuild added tests-started and removed tests-rejected labels Feb 1, 2021

cmsbuild added tests-approved and removed tests-started labels Feb 2, 2021

cmsbuild removed the hlt-pending label Feb 2, 2021

cmsbuild added fully-signed hlt-approved and removed pending-signatures labels Feb 2, 2021

cmsbuild added orp-approved and removed orp-pending labels Feb 2, 2021

cmsbuild merged commit c9e2d2d into cms-sw:master Feb 2, 2021

makortel mentioned this pull request Feb 2, 2021

Failures in two L1 TestDQMOnlineClient unit tests #32796

Closed

Martin-Grunewald mentioned this pull request Aug 17, 2022

[L1T] allow fractional trigger prescales #39094

Closed

missirol mentioned this pull request Aug 26, 2022

deprecate integer types for L1T and HLT prescales in HLTConfigData and its clients #39220

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pr113x L1T Fractional Prescales implemented with Global Trigger #32741

Pr113x L1T Fractional Prescales implemented with Global Trigger #32741

rekovic commented Jan 26, 2021

cmsbuild commented Jan 26, 2021

cmsbuild commented Jan 26, 2021

rekovic commented Jan 26, 2021

cmsbuild commented Jan 26, 2021

christopheralanwest commented Jan 26, 2021

Martin-Grunewald commented Jan 27, 2021

Martin-Grunewald commented Jan 27, 2021

Martin-Grunewald commented Jan 27, 2021

rekovic commented Feb 1, 2021

silviodonato commented Feb 1, 2021

Martin-Grunewald commented Feb 1, 2021

Martin-Grunewald commented Feb 1, 2021

Martin-Grunewald commented Feb 1, 2021

christopheralanwest commented Feb 1, 2021

fwyzard commented Feb 1, 2021

smuzaffar commented Feb 1, 2021

silviodonato commented Feb 1, 2021

cmsbuild commented Feb 2, 2021

Martin-Grunewald commented Feb 2, 2021

cmsbuild commented Feb 2, 2021

silviodonato commented Feb 2, 2021

makortel commented Feb 2, 2021

Pr113x L1T Fractional Prescales implemented with Global Trigger #32741

Pr113x L1T Fractional Prescales implemented with Global Trigger #32741

Conversation

rekovic commented Jan 26, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jan 26, 2021

cmsbuild commented Jan 26, 2021

rekovic commented Jan 26, 2021

cmsbuild commented Jan 26, 2021

Unit Tests

RelVals

RelVals-INPUT

AddOn Tests

christopheralanwest commented Jan 26, 2021

Martin-Grunewald commented Jan 27, 2021

Martin-Grunewald commented Jan 27, 2021

Martin-Grunewald commented Jan 27, 2021

rekovic commented Feb 1, 2021

silviodonato commented Feb 1, 2021

Martin-Grunewald commented Feb 1, 2021

Martin-Grunewald commented Feb 1, 2021

Martin-Grunewald commented Feb 1, 2021

christopheralanwest commented Feb 1, 2021

fwyzard commented Feb 1, 2021

smuzaffar commented Feb 1, 2021

silviodonato commented Feb 1, 2021

cmsbuild commented Feb 2, 2021

Comparison Summary

Martin-Grunewald commented Feb 2, 2021

cmsbuild commented Feb 2, 2021

silviodonato commented Feb 2, 2021

makortel commented Feb 2, 2021