Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add TICL from Reco customise #37343

Merged
merged 2 commits into from
Mar 30, 2022
Merged

Add TICL from Reco customise #37343

merged 2 commits into from
Mar 30, 2022

Conversation

rovere
Copy link
Contributor

@rovere rovere commented Mar 25, 2022

PR description:

Add a customise function to be able to quickly run HGCAL TICL-reconstruction from the RECO data-tier. This will cut down the development cycle by a considerable amount. Also, save the output using a dedicated, slimmed, HGCAL-only event content to save space.

PR validation:

Local testing works just fine.

@cmsbuild cmsbuild added this to the CMSSW_12_4_X milestone Mar 25, 2022
@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37343/28991

  • This PR adds an extra 12KB to repository

@rovere
Copy link
Contributor Author

rovere commented Mar 25, 2022

@cmsbuild please test

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @rovere (Marco Rovere) for master.

It involves the following packages:

  • RecoHGCal/Configuration (upgrade, reconstruction)
  • RecoHGCal/TICL (upgrade, reconstruction)

@jpata, @AdrianoDee, @srimanob, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@felicepantaleo, @apsallid, @sobhatta, @lecriste, @hatakeyamak, @ebrondol this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cc4568/23383/summary.html
COMMIT: 960912a
CMSSW: CMSSW_12_4_X_2022-03-24-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37343/23383/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3415 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3695650
  • DQMHistoTests: Total failures: 48073
  • DQMHistoTests: Total nulls: 45
  • DQMHistoTests: Total successes: 3647510
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 204 log files, 45 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

@jpata
Copy link
Contributor

jpata commented Mar 25, 2022

@rovere just to check - this does not run in any standard workflow, it's just for "private" tests by the TICL team?

@rovere
Add layerCluster producer
1462ddc 
If TICLFromReco is run on top of the optimized HGCAL-only event content,
the producer is not needed. The local density map computed while
creating the layer clusters is part of the special HGCAL-only event
content. Unfortunately, that product is not persisted in any ordinary
data-tiers but is consumed by the HGCAL validation. Adding the
layercluster producer fixes the bug and makes the customize even more
generic.
@rovere
Copy link
Contributor Author

rovere commented Mar 25, 2022 •
edited
Loading

@rovere just to check - this does not run in any standard workflow, it's just for "private" tests by the TICL team?

@jpata that's exactly correct. I'd prefer to have it in release so that it can be used by all the developers. Re-running just TICL -related reconstruction from RECO, instead of the full workflow is just much faster. And the event content is up to ~10 times smaller. Quite useful, I think.

@rovere
Copy link
Contributor Author

rovere commented Mar 25, 2022

@cmsbuild please test

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37343/28997

  • This PR adds an extra 12KB to repository

@cmsbuild
Copy link
Contributor

Pull request #37343 was updated. @jpata, @AdrianoDee, @srimanob, @clacaputo, @slava77 can you please check and sign again.

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cc4568/23396/summary.html
COMMIT: 1462ddc
CMSSW: CMSSW_12_4_X_2022-03-25-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37343/23396/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3585 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3695650
  • DQMHistoTests: Total failures: 52135
  • DQMHistoTests: Total nulls: 57
  • DQMHistoTests: Total successes: 3643436
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 204 log files, 45 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

@jpata
Copy link
Contributor

jpata commented Mar 28, 2022

@cmsbuild please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cc4568/23445/summary.html
COMMIT: 1462ddc
CMSSW: CMSSW_12_4_X_2022-03-27-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37343/23445/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3589460
  • DQMHistoTests: Total failures: 8
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3589430
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 200 log files, 45 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

@civanch civanch mentioned this pull request Mar 28, 2022
Merged
@srimanob
Copy link
Contributor

+Upgrade

The introduced customization does not effect the main workflow.

@jpata
Copy link
Contributor

jpata commented Mar 29, 2022

+reconstruction

  • private customization not used in standard reco

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@qliphy
Copy link
Contributor

qliphy commented Mar 30, 2022

+1

@cmsbuild cmsbuild merged commit 2ab5c60 into cms-sw:master Mar 30, 2022
@cmsbuild cmsbuild mentioned this pull request Mar 31, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
code-checks-approved fully-signed orp-approved reconstruction-approved tests-approved upgrade-approved
Projects
None yet
Milestone
CMSSW_12_4_X
Development

Successfully merging this pull request may close these issues.
5 participants
@rovere @cmsbuild @jpata @srimanob @qliphy