Rebase of #19222: JME python configuration cleanup for 92x #19406

rappoccio · 2017-06-22T15:21:48Z

Had a git merge problem that I could not fix in #19222, so re-adding a new PR. Here is the original message:

The JME python configurations had many vestiges left of previous migrations, as well as considerable "top-level" customization that is no longer relevant. I have consolidated the RecoPFJets_cff customizations into separate cfi python files in RecoJets/JetProducers.

Removed iterative cone, SIScone, and AK5 PF and Calo jets in all configurations.
Consolidated the various grooming schemes into a single file per jet radius (so, AK4 puppi is now in ak4PFJets_cfi, for instance).
Left AK4, AK8, KT4, KT6, and only AK5/7 Castor collections since those are the only ones used anywhere in the repository.
Added an example AK8 GenJet configuration with grooming applied, since this is a common question. It is not presently run anywhere, though.
EDIT: No changes should occur to any output, this is a reorganization.

cmsbuild · 2017-06-22T15:23:02Z

A new Pull Request was created by @rappoccio for master.

It involves the following packages:

CommonTools/ParticleFlow
PhysicsTools/PatAlgos
PhysicsTools/PatUtils
PhysicsTools/TagAndProbe
RecoJets/Configuration
RecoJets/JetProducers

@perrotta, @cmsbuild, @slava77, @monttj, @davidlange6 can you please review it and eventually sign? Thanks.
@TaiSakuma, @gouskos, @imarches, @ahinzmann, @acaudron, @gpetruc, @mmarionncern, @jdolen, @nhanvtran, @JyothsnaKomaragiri, @gkasieczka, @yslai, @schoef, @ferencek, @mverzett, @mariadalfonso, @pvmulder, @seemasharmafnal, @cbernet this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

perrotta · 2017-06-22T15:27:25Z

please test

cmsbuild · 2017-06-22T15:27:45Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/20850/console Started: 2017/06/22 17:28

cmsbuild · 2017-06-22T18:32:24Z

+1
Tested at: a7f4cda
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19406/20850/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
4bfcf97
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19406/20850/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19406/20850/git-merge-result

cmsbuild · 2017-06-22T18:32:27Z

Comparison job queued.

cmsbuild · 2017-06-22T21:53:02Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19406/20850/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 25 differences found in the comparisons
DQMHistoTests: Total files compared: 22
DQMHistoTests: Total histograms compared: 1802916
DQMHistoTests: Total failures: 14601
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 1788149
DQMHistoTests: Total skipped: 166
DQMHistoTests: Total Missing objects: 0
Checked 90 log files, 14 edm output root files, 22 DQM output files

rappoccio · 2017-06-23T13:11:20Z

Looking at this, there seem to be some O(10%) differences in soft drop. I will follow up.

ahinzmann · 2017-06-23T13:18:05Z

RecoJets/JetProducers/python/ak4PFJets_cfi.py

@@ -11,3 +11,25 @@
    rParam       = cms.double(0.4)


How about turning on "doAreaFastjet=True" for ak4PFJets (and thus all other derived ak jet algorithms) during this cleanup. This is currently done only in the PAT/MiniAOD configuration and lead to many errors in the past.

OK, will do.

rappoccio · 2017-06-23T15:15:25Z

OK I figured out what the remaining problem it. It's actually technically a bug in the previous implementation. The AK8 CHS soft dropped jets were previously only requiring pt > 100 GeV, whereas it was intended to also have a pre-clustering with AK8 that has an |eta| < 2.4 cut.

@ahinzmann @jdolen do you two agree with the addition of the |eta| < 2.4 cut and the AK8 preclustering?

Our new implementation will slightly speed things up, and fix this "unintended feature". @slava77 is that acceptable?

slava77 · 2017-06-23T16:11:51Z

On 6/23/17 8:15 AM, rappoccio wrote: OK I figured out what the remaining problem it. It's actually technically a bug in the previous implementation. The AK8 CHS soft dropped jets were previously only requiring pt > 100 GeV, whereas it was intended to also have a pre-clustering with AK8 that has an |eta| < 2.4 cut. @ahinzmann <https://github.com/ahinzmann> @jdolen <https://github.com/jdolen> do you two agree with the addition of the |eta| < 2.4 cut and the AK8 preclustering? Our new implementation will slightly speed things up, and fix this "unintended feature". @slava77 <https://github.com/slava77> is that acceptable?

with so many changes in code lines, I'd rather decouple refactoring from physics change.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19406 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbq6i4FZ9-YrqdPq8FD0mpjuM36wfks5sG9aOgaJpZM4OCdDc>.

rappoccio · 2017-06-23T16:41:49Z

@slava77 understood. I will retain the old functionality.

slava77 · 2017-06-26T18:26:07Z

the errors appear to be unrelated to this PR

#5  std::string::_M_data (this=0x14e) at /mnt/build/davidlt/gcc530_final/b/BUILD/slc6_amd64_gcc530/external/gcc/5.3.0/gcc-tags_gcc_5_3_0_release-231255/obj/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:2691
#6  std::string::_M_rep (this=0x14e) at /mnt/build/davidlt/gcc530_final/b/BUILD/slc6_amd64_gcc530/external/gcc/5.3.0/gcc-tags_gcc_5_3_0_release-231255/obj/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:2699
#7  std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string (this=0x7fe98f765030, __str=...) at /mnt/build/davidlt/gcc530_final/b/BUILD/slc6_amd64_gcc530/external/gcc/5.3.0/gcc-tags_gcc_5_3_0_release-231255/obj/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:617
#8  0x00007fe991872fd5 in XrdCl::URL::GetHostId (this=<optimized out>) at /build/cmsbld/auto-builds/CMSSW_9_1_0_pre3-slc6_amd64_gcc530/build/CMSSW_9_1_0_pre3-build/BUILD/slc6_amd64_gcc530/external/xrootd/4.6.1.RC2/xrootd-4.6.1.RC2/src/./XrdCl/XrdClURL.hh:72

@Dr15Jones is this a known xrootd failure?

Dr15Jones · 2017-06-26T18:39:10Z

I can't say that the failure looks familiar. @bbockelm ?

cmsbuild · 2017-06-26T19:49:49Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19406/20920/summary.html

Comparison Summary:

You potentially added 137 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 22
DQMHistoTests: Total histograms compared: 1813798
DQMHistoTests: Total failures: 14580
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 1799052
DQMHistoTests: Total skipped: 166
DQMHistoTests: Total Missing objects: 0
Checked 90 log files, 14 edm output root files, 22 DQM output files

perrotta · 2017-06-27T08:47:19Z

please test
(let see what happens with the new 93X IB)

cmsbuild · 2017-06-27T08:47:36Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/20932/console Started: 2017/06/27 10:48

cmsbuild · 2017-06-27T10:03:43Z

+1
Tested at: 1829abf
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19406/20932/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
3b01ccc
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19406/20932/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19406/20932/git-merge-result

cmsbuild · 2017-06-27T10:03:45Z

Comparison job queued.

cmsbuild · 2017-06-27T12:51:01Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19406/20932/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 22
DQMHistoTests: Total histograms compared: 1754263
DQMHistoTests: Total failures: 14560
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 1739537
DQMHistoTests: Total skipped: 166
DQMHistoTests: Total Missing objects: 0
Checked 90 log files, 14 edm output root files, 22 DQM output files

perrotta · 2017-06-28T10:11:34Z

+1
Reorganization and clean up of jet configurations
After the latest fixes, there is no effect any more on the tested workflows

davidlange6 · 2017-06-30T07:27:57Z

merge

cmsbuild added this to the CMSSW_9_2_X milestone Jun 22, 2017

cmsbuild added analysis-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jun 22, 2017

cmsbuild added tests-started and removed tests-pending labels Jun 22, 2017

cmsbuild added tests-approved and removed tests-started labels Jun 22, 2017

cmsbuild added comparison-available and removed comparison-pending labels Jun 22, 2017

ahinzmann reviewed Jun 23, 2017

View reviewed changes

cmsbuild added alca-pending comparison-pending core-pending geometry-pending and removed comparison-available tests-approved labels Jun 23, 2017

cmsbuild added comparison-available and removed comparison-pending labels Jun 26, 2017

cmsbuild added comparison-pending tests-pending tests-started and removed comparison-available tests-rejected tests-pending labels Jun 27, 2017

cmsbuild added tests-approved and removed tests-started labels Jun 27, 2017

cmsbuild added comparison-available and removed comparison-pending labels Jun 27, 2017

smuzaffar modified the milestone: CMSSW_9_3_X Jun 27, 2017

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Jun 28, 2017

cmsbuild merged commit 97467bb into cms-sw:master Jun 30, 2017

rappoccio deleted the JMECleanup92X_take2 branch February 6, 2018 15:18

raggleton mentioned this pull request Oct 19, 2018

[80Xv6] Fix subjet area - don't merge yet! - questions for users UHH2/UHH2#1032

Closed

laurenhay mentioned this pull request Aug 24, 2022

Replace Geometry_cff with GeometryDB_cff in RecoJets #38784

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebase of #19222: JME python configuration cleanup for 92x #19406

Rebase of #19222: JME python configuration cleanup for 92x #19406

rappoccio commented Jun 22, 2017 •

edited

Loading

cmsbuild commented Jun 22, 2017

perrotta commented Jun 22, 2017

cmsbuild commented Jun 22, 2017 •

edited

Loading

cmsbuild commented Jun 22, 2017

cmsbuild commented Jun 22, 2017

cmsbuild commented Jun 22, 2017

rappoccio commented Jun 23, 2017

ahinzmann Jun 23, 2017

rappoccio Jun 23, 2017

rappoccio commented Jun 23, 2017

slava77 commented Jun 23, 2017 via email

rappoccio commented Jun 23, 2017

slava77 commented Jun 26, 2017

Dr15Jones commented Jun 26, 2017

cmsbuild commented Jun 26, 2017

perrotta commented Jun 27, 2017

cmsbuild commented Jun 27, 2017 •

edited

Loading

cmsbuild commented Jun 27, 2017

cmsbuild commented Jun 27, 2017

cmsbuild commented Jun 27, 2017

perrotta commented Jun 28, 2017

davidlange6 commented Jun 30, 2017

Rebase of #19222: JME python configuration cleanup for 92x #19406

Rebase of #19222: JME python configuration cleanup for 92x #19406

Conversation

rappoccio commented Jun 22, 2017 • edited Loading

cmsbuild commented Jun 22, 2017

perrotta commented Jun 22, 2017

cmsbuild commented Jun 22, 2017 • edited Loading

cmsbuild commented Jun 22, 2017

cmsbuild commented Jun 22, 2017

cmsbuild commented Jun 22, 2017

rappoccio commented Jun 23, 2017

ahinzmann Jun 23, 2017

Choose a reason for hiding this comment

rappoccio Jun 23, 2017

Choose a reason for hiding this comment

rappoccio commented Jun 23, 2017

slava77 commented Jun 23, 2017 via email

rappoccio commented Jun 23, 2017

slava77 commented Jun 26, 2017

Dr15Jones commented Jun 26, 2017

cmsbuild commented Jun 26, 2017

perrotta commented Jun 27, 2017

cmsbuild commented Jun 27, 2017 • edited Loading

cmsbuild commented Jun 27, 2017

cmsbuild commented Jun 27, 2017

cmsbuild commented Jun 27, 2017

perrotta commented Jun 28, 2017

davidlange6 commented Jun 30, 2017

rappoccio commented Jun 22, 2017 •

edited

Loading

cmsbuild commented Jun 22, 2017 •

edited

Loading

cmsbuild commented Jun 27, 2017 •

edited

Loading