Validation plots for APE and bug fixes: Backport of #34828

[mteroerd]

PR description:

Changes to the APE tool:

-Bug fixes: Remove lines that were previously added to prevent memory leaks, but which created errors. Also tweak some lines in test/plottingTools.
-Added a new plotting tool to allow for creation of plots with residual distributions plus some other validation plots.
-Added an option to autoSubmitter, so that APE can be more easily measured on cosmics data sets.

PR validation:

Ran some basic tests of the code on the most recent APE campaign to confirm that the changes work as intended.

if this PR is a backport please specify the original PR and why you need to backport that PR:

Backport of #34828

[cmsbuild]

A new Pull Request was created by @mteroerd (Marius Teroerde) for CMSSW_12_0_X.

It involves the following packages:

• Alignment/APEEstimation (alca)

@malbouis, @yuanchao, @cmsbuild, @tlampen, @pohsun, @francescobrivio, @tvami can you please review it and eventually sign? Thanks.
@mmusich, @tocheng, @adewit this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

• Backported from Validation plots for APE and bug fixes #34828

[mmusich]

type bug-fix

[mmusich]

@cmsbuild please test

[mmusich]

backport of #34828

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-36b0e0/18018/summary.html
COMMIT: bda431d
CMSSW: CMSSW_12_0_X_2021-08-24-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35007/18018/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 39
• DQMHistoTests: Total histograms compared: 2998564
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2998542
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
• Checked 165 log files, 37 edm output root files, 39 DQM output files
• TriggerResults: no differences found

[tvami]

+alca

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[qliphy]

+1