Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Porting HLT Phase 2 simplified menu to 12_4_X #37162

Merged
merged 10 commits into from
Apr 20, 2022
Merged

Porting HLT Phase 2 simplified menu to 12_4_X #37162

merged 10 commits into from
Apr 20, 2022

Conversation

trtomei
Copy link
Contributor

@trtomei trtomei commented Mar 7, 2022 •
edited
Loading

PR description:

This implements the Simplified Menu from the Phase-2 DAQ-HLT-TDR in CMSSW_12_4_X

PR validation:

As of now, this new menu (eventsetup, modules, tasks, paths, etc.) is not yet used anywhere, so it's impossible for this to actually change anything.

There is now a related PR to test this menu in PR tests, RelVals, and IBs (#37324). For the time being, we have to manually test.

This test works:

cmsrel CMSSW_12_4_X_2022-04-06-1100
cd CMSSW_12_4_X_2022-04-06-1100/src
cmsenv
git cms-init
git cms-addpkg HLTrigger/Configuration
git remote add trtomei https://github.com/trtomei/cmssw.git
git fetch trtomei
git merge trtomei/tt4-hltPhase2-12_3_X
scram b -j 8
cmsDriver.py Phase2 -s HLT:75e33 --processName=HLTX \
--conditions auto:phase2_realistic_T21 --geometry Extended2026D77 \
--era Phase2C11I13M9 --eventcontent FEVTDEBUGHLT \
--filein=/store/relval/CMSSW_12_3_0_pre6/RelValTTbar_14TeV/GEN-SIM-RECO/123X_mcRun4_realistic_v8_2026D77noPUrsb-v1/10000/a22358bb-8cd3-4558-ad3f-3ed22ffbe5e9.root \
-n 100 --nThreads 1 --no_exec --customise_commands process.load("HLTrigger.Configuration.HLTPhase2TDR_EventContent_cff");\nprocess.FEVTDEBUGHLToutput.outputCommands.extend(process.HLTPhase2TDR.outputCommands);
cmsRun Phase2_HLT.py

@trtomei
Porting HLT Phase 2 simplified menu to 12_3_X
3266aea 
Porting HLT Phase 2 simplified menu, 12_3_0_pre5 fixes

Porting HLT Phase 2 simplified menu, 12_3_0_pre5 fixes part 2

Adapt HLT Phase 2 menu to 36983
@cmsbuild cmsbuild added this to the CMSSW_12_3_X milestone Mar 7, 2022
@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2022

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37162/28723

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2022

A new Pull Request was created by @trtomei (Thiago Tomei) for master.

It involves the following packages:

  • HLTrigger/Configuration (hlt)

@cmsbuild, @missirol, @Martin-Grunewald can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @silviodonato this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@trtomei
Copy link
Contributor Author

trtomei commented Mar 7, 2022

I think this one supersedes #35342

@trtomei
Update pfTICL_cfi.py
c07b027 
Forgot to explicitly add the new parameter here.
@trtomei
Update pfTICL_cfi.py
fd143da 
Set MTD timing usage to false.
@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2022

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37162/28725

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2022

Pull request #37162 was updated. @cmsbuild, @missirol, @Martin-Grunewald can you please check and sign again.

@missirol
Copy link
Contributor

missirol commented Mar 7, 2022 •
edited
Loading

please test with #37158

Starting the tests even though the new code is not run anywhere. Running "with" the related PR just to include the label requires-external.

@trtomei , in #37158 I read

In view of the imminent closing of the prereleases [..]

Does this mean that the goal is to have this PR (and #37158) integrated before the latest 12_3_X pre-release is opened (i.e. as soon as tomorrow, possibly)? Would there be an issue with having these config files only in 12_4_X?

@trtomei
Copy link
Contributor Author

trtomei commented Mar 7, 2022

Hi Marino,

Ideally, we would like to have this in 12_3_X, for the production! The key is that, since this code doesn't actually run anywhere yet (i.e. is not part of any workflow), we could use the one month until the release of CMSSW_12_3_0 to set up a workflow + RelVal, get it all working, and run this as "the HLT menu" for our production.

At least that is the plan...

@missirol
Copy link
Contributor

missirol commented Mar 7, 2022

Okay, thanks. As soon as the last pre-release of 12_3_X is opened (possibly tomorrow), this PR to master will be redirected to 12_4_X. It sounds to me like this is still okay: this PR can be integrated in 12_4_X, then backported to 12_3_X in time for 12_3_0 (approx. 1 month from now). The same goes for the future PR that should set up the RelVal wf for this (first PR in 12_4_X, then backport in time for 12_3_0). In this scenario, this PR is not urgent for 12_3_0_pre6.

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2022

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-260256/22916/summary.html
COMMIT: fd143da
CMSSW: CMSSW_12_3_X_2022-03-07-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37162/22916/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 5 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3984447
  • DQMHistoTests: Total failures: 8
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3984417
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 204 log files, 45 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

@AdrianoDee AdrianoDee mentioned this pull request Mar 8, 2022
Merged
@cmsbuild
Copy link
Contributor

Pull request #37162 was updated. @cmsbuild, @missirol, @Martin-Grunewald can you please check and sign again.

@Martin-Grunewald
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-260256/23948/summary.html
COMMIT: d93b556
CMSSW: CMSSW_12_4_X_2022-04-14-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37162/23948/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-260256/39434.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 10 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3593039
  • DQMHistoTests: Total failures: 19
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 3592997
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.004 KiB( 47 files compared)
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 200 log files, 45 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

@srimanob
Copy link
Contributor

Look nice, the workflow 39434.75 is no warning message now,
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-260256/23948/runTheMatrix-results/39434.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal/

@Martin-Grunewald
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@missirol
Copy link
Contributor

This PR is ready to be integrated, but it should be merged together (i.e. same IB) with #37324, which is not fully-signed yet.

If I understand correctly, the goal is to (1) merge both in 12_4_X, (2) backport them both to 12_3_X, and (3) eventually request a new 12_3_X release to be used for Phase-2 MC production.

This PR and #37324 are not really urgent for 12_4_0_pre3, but they might be a nice-to-have.

@srimanob , what do you think?

@srimanob
Copy link
Contributor

Hi @missirol

Thanks. I just sign the #37324 and ping PdmV to check and to sign again.
It will be ideal if they go together, but this PR can go first without waiting for #37324.

For production, as discussion happens last week, we will not need a backport. We will make sample with 12_3 and rerun any combination, HLT-only or L1(emulation)+HLT as preferred in 12_4. If run locally, pre-release is fine to use. This is to avoid a big backport to 12_3.

@missirol
Copy link
Contributor

urgent

It will be ideal if they go together, but this PR can go first without waiting for #37324.

"urgent" is just to signal that this PR now targets integration in 12_4_0_pre3 (see comment below).

For production, as discussion happens last week, we will not need a backport. We will make sample with 12_3 and rerun any combination, HLT-only or L1(emulation)+HLT as preferred in 12_4. If run locally, pre-release is fine to use. This is to avoid a big backport to 12_3.

Ah, okay, I didn't know. So, no need for backports. Thanks, @srimanob.

@qliphy
Copy link
Contributor

qliphy commented Apr 20, 2022

+1

@cmsbuild cmsbuild merged commit f6d6afd into cms-sw:master Apr 20, 2022
This was referenced Apr 20, 2022
Closed
Merged
Merged
Merged
@srimanob
Copy link
Contributor

The Phase-2 HLT workflow is running in IB since CMSSW_12_4_X_2022-04-20-1100
So far, so good.

@trtomei trtomei deleted the tt4-hltPhase2-12_3_X branch August 18, 2022 19:36
@missirol missirol mentioned this pull request Sep 7, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Martin-Grunewald Martin-Grunewald Martin-Grunewald left review comments

Assignees
No one assigned
Labels
code-checks-approved fully-signed hlt-approved orp-approved requires-external tests-approved urgent
Projects
None yet
Milestone
CMSSW_12_4_X
Development

Successfully merging this pull request may close these issues.
9 participants
@trtomei @cmsbuild @missirol @srimanob @Martin-Grunewald @silviodonato @mmusich @qliphy @smuzaffar