[DD4hep] Tests Cleanup and 2021 Production Scenario Update

[ianna]

PR description:

• 2021 production scenario updated
• Use it in the tests
• Simplify comparison for identifying a regular expression
• Anchor a regular expression
• Tests moved to appropriate subsystems
• Integration tests added to run them

PR validation:

unit tests and integration tests run by default

if this PR is a backport please specify the original PR and why you need to backport that PR:

Before submitting your pull requests, make sure you followed this checklist:

• verify that the PR is really intended for the chosen branch
• verify that changes follow CMS Naming, Coding, And Style Rules
• verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

[ianna]

@slomeo - FYI

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30093/15833

• This PR adds an extra 84KB to repository

• Found files with invalid states:

  • DetectorDescription/DDCMS/data/cms-test-muon-geometry-2015.xml:
    • Added: dc29cb9
    • Deleted: 6a7ba31
  • DetectorDescription/DDCMS/data/cms-test-geometry-2021.xml:
    • Added: dc29cb9
    • Deleted: 6a7ba31
  • Geometry/CMSCommonData/data/cmsDD4hepGeometry2021.xml:
    • Added: dc29cb9
    • Deleted: 6a7ba31

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/Geometry/python/GeometryDD4hepExtended2021_cff.py modified in PR(s): Update of the RPC migration to DD4hep  #30047
  • File DetectorDescription/DDCMS/interface/Filter.h modified in PR(s): [RFC] Test clang-tidy --checks performance-unnecessary-value-param #29933
  • File DetectorDescription/DDCMS/src/DDFilteredView.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-use-auto #29936
  • File DetectorDescription/DDCMS/src/Filter.cc modified in PR(s): [RFC] Test clang-tidy --checks performance-unnecessary-value-param #29933
  • File Geometry/CMSCommonData/data/dd4hep/cmsExtendedGeometry2021.xml modified in PR(s): Update of the RPC migration to DD4hep  #30047

[cmsbuild]

A new Pull Request was created by @ianna (Ianna Osborne) for master.

It involves the following packages:

Configuration/Geometry
DetectorDescription/DDCMS
Geometry/CMSCommonData
Geometry/DTGeometryBuilder
Geometry/MuonCommonData
Geometry/TrackerCommonData
MagneticField/GeomBuilder

@perrotta, @civanch, @Dr15Jones, @makortel, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @kpedro88, @slava77 can you please review it and eventually sign? Thanks.
@namapane, @battibass, @makortel, @vargasa, @VinInn, @Martin-Grunewald, @ptcox, @ebrondol, @ghugo83, @fabiocos, @slomeo, @venturia this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

[ianna]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6768/console Started: 2020/06/03 14:39

[cmsbuild]

-1

Tested at: cd144a0

CMSSW: CMSSW_11_2_X_2020-06-02-2300
SCRAM_ARCH: slc7_amd64_gcc820
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-82b7d4/6768/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests

• Unit Tests:

I found errors in the following unit tests:

---> test DD4hepGeometryTestDriver had ERRORS
---> test GeometryDTGeometryBuilderTestDriver had ERRORS
---> test GeometryTrackerCommonDataTestDriver had ERRORS

[cmsbuild]

Comparison job queued.

[cvuosalo]

@ianna A lot of different changes are made in this PR. I think you have fixed some bugs or enhanced functionality in DDFilteredView, but I'm not sure. Does this PR address any of the problems some DD4hep migraters have been encountering? If so, please mention them in the PR description so we know what has been fixed.
Also, I see you have started changing double to float in various algorithms. Could you please mention these changes in the PR description?

[cvuosalo]

+1

[civanch]

+1

[ianna]

@perrotta, @makortel, @kpedro88, @slava77 can you please check

[slava77]

+1

for #30093 e1a5aee

• reco-category files in MagneticField are mainly refactored after a move of some files from DetectorDescription/DDCMS
• jenkins tests pass and comparisons with the baseline show no (relevant) differences

[kpedro88]

+upgrade

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

[silviodonato]

+1

[cvuosalo]

For the record, additional details about this PR:

The PR is done in preparation for DDCMS code migration to DD4hep repository. All DPG related tests were moved under the DPG subsystems along with their automated execution. Many of the subsystems, for which the DPG are responsible, did not have it implemented before.

The tests used a private 2021 scenario that was more up to date then the production one. I’ve updated the 2021 production scenario and modified all test configurations to use it. The first was removed.

This fixed Sergio’s problem with an offset in the comparison: the modified YE thickness was not propagated to the scenario he was using.