[10_6_X] DDXv2 and BTV changes in Nano #31618, #31786

[andrzejnovak]

Backport of #31618, #31786

• DDX "v1" is kept alongside of v2
• NanoAOD era modifiers are synced to master

[cmsbuild]

A new Pull Request was created by @andrzejnovak (Andrzej Novak) for CMSSW_10_6_X.

It involves the following packages:

PhysicsTools/NanoAOD

@cmsbuild, @santocch, @mariadalfonso, @gouskos, @fgolf can you please review it and eventually sign? Thanks.
@gpetruc, @peruzzim, @swertz this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[mariadalfonso]

@andrzejnovak

this backport PR seems take care only of the previously made nano. but not of the remini-UL and nano that come from there.

In master #31618 you have modifications in PhysicsTools/PatAlgos/python/slimming/applyDeepBtagging_cff.py
and this V2 should be the UL-reminiAOD default.
Then in nano from the UL-remini we do not re-run the V2 and we just get from the mini.

[andrzejnovak]

@mariadalfonso Ok so that was not clear to me. The 106 mini should by default use DDX v2? I thought this was not desirable since there already exists 106 mini which does not contain the V2.

[mariadalfonso]

@andrzejnovak
it's correct that the 106 mini out of the box should be unmodified.

That's why we need to use modifiers.
The run2_miniAOD_UL will be the new type of mini when we have the reminiAOD and any modification should be beyond this modifier.
The V2 can be the default once the run2_miniAOD_UL is activated.

[cmsbuild]

Pull request #31847 was updated. @perrotta, @gouskos, @cmsbuild, @fgolf, @slava77, @jpata, @mariadalfonso, @santocch can you please check and sign again.

[mariadalfonso]

please test

[cmsbuild]

The tests are being triggered in jenkins.

• CMSSW_10_6_X_2020-10-21-1100/slc7_amd64_gcc700: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10185/console Started: 2020/10/21 20:57

[cmsbuild]

-1

Tested at: d1100f0

CMSSW: CMSSW_10_6_X_2020-10-21-1100
SCRAM_ARCH: slc7_amd64_gcc700
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd48d5/10185/summary.html

I found follow errors while testing this PR

Failed tests: Build

• Build:

I found compilation warning when building: See details on the summary page.

[cmsbuild]

Comparison not run due to Build errors (RelVals and Igprof tests were also skipped)

[mariadalfonso]

Compiling src/PhysicsTools/NanoAOD/python/nano_cff.py ...
Sorry: IndentationError: unexpected indent (nano_cff.py, line 346)
gmake[1]: *** [config/SCRAM/GMake/Makefile.rules:1889: CompilePython] Error 1

[cmsbuild]

The tests are being triggered in jenkins.

• CMSSW_10_6_X_2020-11-02-2300/slc7_amd64_gcc700: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10457/console Started: 2020/11/03 02:45

[slava77]

+1

for #31847 39aa5df

• concerning reco package changes only: the backport of feat: switch to DDXv2 as baseline #31618 is appropriate, even though it's not direct; DDXv2 taggers were added for run2_miniAOD_UL rather than replacing the v1 as was done in the master (this is now provided in the updated PR description from a few days ago).
  • it looks like the PhysicsTools/NanoAOD part is still under discussion
• jenkins tests pass and comparisons with the baseline show differences in
  • the UL re-miniAOD workflow 136.88811: 6 new tags were added in slimmedJetsAK8, as expected
  • nanoAOD plots have changes both in the "from scratch" case in wf 10824.0 and in nanoAOD wf made from 2017 MC inputs in wf 1325.7. It sounds like these are still under review by xpog.

[cmsbuild]

+1
Tested at: 39aa5df
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd48d5/10457/summary.html
CMSSW: CMSSW_10_6_X_2020-11-02-2300
SCRAM_ARCH: slc7_amd64_gcc700

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd48d5/10457/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 312 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 3214664
• DQMHistoTests: Total failures: 7
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3214323
• DQMHistoTests: Total skipped: 334
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 5.474 KiB( 33 files compared)
• DQMHistoSizes: changed ( 1325.7 ): 5.474 KiB Physics/NanoAODDQM
• Checked 140 log files, 14 edm output root files, 34 DQM output files

[mariadalfonso]

+xpog

Changes inline with master:

• Fat Jet have the new DeepTagV2 for all eras
• AK4Jets have btagDeepFlav redefined as in master

Additionally:

• for the 2016MC of UL type we have included a fix for the jetID that missed to be backported as discussed here [10_6_X] DDXv2 and BTV changes in Nano #31618, #31786 #31847 (comment)

• there are two spurious lines that have no effect related to a cleanup that happened in master but not backported
  https://github.com/cms-sw/cmssw/pull/31847/files#diff-377770d6a7d4705da48aa96709bdedefcb42d5b5497c85ba41a3f11074c148a0R480-R481
  cleanup can be done after

[santocch]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[silviodonato]

+1