use auto-generated configuration fragments in a few `RecoTracker` classes #37023

mmusich · 2022-02-22T09:59:42Z

PR description:

This is a technical follow-up to PR #36983 and in particular to #36983 (comment).
As the title says the purpose is to use auto-generated configuration fragments in a few RecoTracker classes instead of direct implementations in the python code.
I profit of this PR to clean-up the getFilledConfigurationDescription4CCC method introduced in commit 10e227e to be versatile (e.g. serve all possible values of the CCC).

PR validation:

cmssw compiles after checking dependencies.

if this PR is a backport please specify the original PR and why you need to backport that PR:

N/A

cmsbuild · 2022-02-22T10:07:25Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37023/28451

This PR adds an extra 24KB to repository

cmsbuild · 2022-02-22T10:07:44Z

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

RecoLocalTracker/SiStripClusterizer (reconstruction)
RecoPixelVertexing/PixelLowPtUtilities (reconstruction)
RecoTracker/MeasurementDet (reconstruction)
RecoTracker/TransientTrackingRecHit (reconstruction)
TrackingTools/TrajectoryFiltering (reconstruction)

@jpata, @cmsbuild, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@mtosi, @echabert, @pieterdavid, @swertz, @robervalwalsh, @yduhm, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @bellan, @ebrondol, @lecriste, @felicepantaleo, @gpetruc, @gbenelli, @mmusich, @threus, @dgulhan, @alesaggio this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

mmusich · 2022-02-22T10:22:14Z

please test

cmsbuild · 2022-02-22T14:24:55Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45f544/22572/summary.html
COMMIT: b069ad1
CMSSW: CMSSW_12_3_X_2022-02-21-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37023/22572/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test testMuonTrackRefitting had ERRORS

RelVals

4.22

----- Begin Fatal Exception 22-Feb-2022 14:30:51 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_RAW2DIGI_L1Reco_RECO_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: Required parameter ComponentName was not set

At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Types.py(98): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(512): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1145): _insertManyInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1413): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

5.1

----- Begin Fatal Exception 22-Feb-2022 14:31:15 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named TTbar_8TeV_TuneCUETP8M1_cfi_GEN_SIM_RECOBEFMIX_DIGI_L1_DIGI2RAW_L1Reco_RECO_VALIDATION_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: Required parameter ComponentName was not set

At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Types.py(98): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(512): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1145): _insertManyInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1413): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

135.4

----- Begin Fatal Exception 22-Feb-2022 14:31:16 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named ZEE_13TeV_TuneCUETP8M1_cfi_GEN_SIM_RECOBEFMIX_DIGI_L1_DIGI2RAW_L1Reco_RECO_VALIDATION_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: Required parameter ComponentName was not set

At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Types.py(98): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(512): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1145): _insertManyInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1413): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

4.224.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log
4.64.6_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1/step2_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1.log
134.813134.813_RunCosmics2015C+RunCosmics2015C+RECOCOSDRUN2+ALCACOSDRUN2+HARVESTDCRUN2/step2_RunCosmics2015C+RunCosmics2015C+RECOCOSDRUN2+ALCACOSDRUN2+HARVESTDCRUN2.log

Expand to see more relval errors ...

AddOn Tests

fastsim

----- Begin Fatal Exception 22-Feb-2022 14:30:51 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named TTbar_8TeV_TuneCUETP8M1_cfi_GEN_SIM_RECOBEFMIX_DIGI_L1_DIGI2RAW_L1Reco_RECO_VALIDATION.py
Exception Message:
 unknown python problem occurred.
RuntimeError: Required parameter ComponentName was not set

At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Types.py(98): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(512): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1145): _insertManyInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1413): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

fastsim1

----- Begin Fatal Exception 22-Feb-2022 14:30:51 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named TTbar_13TeV_TuneCUETP8M1_cfi_GEN_SIM_RECOBEFMIX_DIGI_L1_DIGI2RAW_L1Reco_RECO_VALIDATION.py
Exception Message:
 unknown python problem occurred.
RuntimeError: Required parameter ComponentName was not set

At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Types.py(98): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(512): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1145): _insertManyInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1413): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

fastsim2

----- Begin Fatal Exception 22-Feb-2022 14:30:52 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named TTbar_13TeV_TuneCUETP8M1_cfi_GEN_SIM_RECOBEFMIX_DIGI_L1_DIGI2RAW_L1Reco_RECO_VALIDATION.py
Exception Message:
 unknown python problem occurred.
RuntimeError: Required parameter ComponentName was not set

At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Types.py(98): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Mixins.py(512): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1145): _insertManyInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-21-2300/python/FWCore/ParameterSet/Config.py(1413): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

cmsbuild · 2022-02-22T16:15:31Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37023/28459

This PR adds an extra 16KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37023/28459/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37023/28459/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

…e CCC fillDescriptions

cmsbuild · 2022-02-22T16:24:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37023/28460

This PR adds an extra 16KB to repository

cmsbuild · 2022-02-22T16:25:17Z

Pull request #37023 was updated. @jpata, @cmsbuild, @clacaputo, @slava77 can you please check and sign again.

mmusich · 2022-02-22T18:03:49Z

@cmsbuild, please test

cmsbuild · 2022-02-22T21:13:37Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45f544/22586/summary.html
COMMIT: 3cb2459
CMSSW: CMSSW_12_3_X_2022-02-22-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37023/22586/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3965143
DQMHistoTests: Total failures: 12
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3965109
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

jpata · 2022-02-23T09:23:59Z

+reconstruction

technical, no reco differences

cmsbuild · 2022-02-23T09:24:22Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-02-23T13:40:33Z

+1

cmsbuild added this to the CMSSW_12_3_X milestone Feb 22, 2022

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Feb 22, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 22, 2022

cmsbuild added tests-started and removed tests-pending labels Feb 22, 2022

cmsbuild added tests-rejected and removed tests-started labels Feb 22, 2022

mmusich force-pushed the moreDescriptionsRecoTracker_12_3_X branch from b069ad1 to 27b04ac Compare February 22, 2022 16:07

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Feb 22, 2022

cmsbuild added code-checks-rejected and removed code-checks-pending labels Feb 22, 2022

use auto-generated fillDescription methods where possible and finaliz…

3cb2459

…e CCC fillDescriptions

mmusich force-pushed the moreDescriptionsRecoTracker_12_3_X branch from 27b04ac to 3cb2459 Compare February 22, 2022 16:17

cmsbuild added code-checks-pending and removed code-checks-rejected labels Feb 22, 2022

cmsbuild removed the code-checks-pending label Feb 22, 2022

cmsbuild added the code-checks-approved label Feb 22, 2022

cmsbuild added tests-started and removed tests-pending labels Feb 22, 2022

cmsbuild added tests-approved and removed tests-started labels Feb 22, 2022

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Feb 23, 2022

cmsbuild added orp-approved and removed orp-pending labels Feb 23, 2022

cmsbuild merged commit 084abba into cms-sw:master Feb 23, 2022

mmusich deleted the moreDescriptionsRecoTracker_12_3_X branch February 23, 2022 13:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use auto-generated configuration fragments in a few `RecoTracker` classes #37023

use auto-generated configuration fragments in a few `RecoTracker` classes #37023

mmusich commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

mmusich commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

mmusich commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

jpata commented Feb 23, 2022

cmsbuild commented Feb 23, 2022

perrotta commented Feb 23, 2022

use auto-generated configuration fragments in a few RecoTracker classes #37023

use auto-generated configuration fragments in a few RecoTracker classes #37023

Conversation

mmusich commented Feb 22, 2022

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

mmusich commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

Unit Tests

RelVals

RelVals-INPUT

AddOn Tests

cmsbuild commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

mmusich commented Feb 22, 2022

cmsbuild commented Feb 22, 2022

Comparison Summary

jpata commented Feb 23, 2022

cmsbuild commented Feb 23, 2022

perrotta commented Feb 23, 2022

use auto-generated configuration fragments in a few `RecoTracker` classes #37023

use auto-generated configuration fragments in a few `RecoTracker` classes #37023