Ecal DPG - Backporting new software[PR 27175] to recover isolated new channels #28831

rchatter · 2020-01-30T21:08:35Z

Backporting PR#27175 [@nancymarinelli @argiro ]

In order to validate and backport PR28040 we need to backport PR#27175 and PR#28514 to make a validation for which PPD will have to advice us on the way how to do it.

This is the first step where we are backporting PR#27175 [ https://github.com//pull/27175], which has already been merged.

cmsbuild · 2020-01-30T21:09:02Z

A new Pull Request was created by @rchatter (Rajdeep Mohan Chatterjee) for CMSSW_10_6_X.

It involves the following packages:

RecoLocalCalo/EcalDeadChannelRecoveryAlgos
RecoLocalCalo/EcalRecProducers

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@apsallid, @argiro this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

slava77 · 2020-01-31T17:14:08Z

@cmsbuild please test

cmsbuild · 2020-01-31T17:14:31Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4455/console Started: 2020/01/31 18:15

cmsbuild · 2020-01-31T17:50:57Z

-1

Tested at: 606742e

CMSSW: CMSSW_10_6_X_2020-01-31-1100
SCRAM_ARCH: slc7_amd64_gcc700
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-af7653/4455/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following workflows:
4.22 step2

runTheMatrix-results/4.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log

136.85 step2

runTheMatrix-results/136.85_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM/step2_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM.log

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

140.56 step2

runTheMatrix-results/140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log

101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

8.0 step3

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step3_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log

7.3 step2

runTheMatrix-results/7.3_CosmicsSPLoose_UP18+CosmicsSPLoose_UP18+DIGICOS_UP18+RECOCOS_UP18+ALCACOS_UP18+HARVESTCOS_UP18/step2_CosmicsSPLoose_UP18+CosmicsSPLoose_UP18+DIGICOS_UP18+RECOCOS_UP18+ALCACOS_UP18+HARVESTCOS_UP18.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

136.788 step3

runTheMatrix-results/136.788_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017/step3_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

158.0 step2

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step2_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

10824.0 step2

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step2_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

11624.0 step3

runTheMatrix-results/11624.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

10224.0 step3

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU.log

250202.181 step3

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step3_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21.log

27434.0 step3

runTheMatrix-results/27434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D35+RecoFullGlobal_2023D35+HARVESTFullGlobal_2023D35/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D35+RecoFullGlobal_2023D35+HARVESTFullGlobal_2023D35.log

29034.0 step3

runTheMatrix-results/29034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D41+RecoFullGlobal_2023D41+HARVESTFullGlobal_2023D41/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D41+RecoFullGlobal_2023D41+HARVESTFullGlobal_2023D41.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Fri Jan 31 18:44:20 2020-date Fri Jan 31 18:43:51 2020 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_DATA.root --fileout file:RelVal_Raw_Fake2_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:45:18 2020-date Fri Jan 31 18:43:57 2020 s - exit: 23040
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw-patch/CMSSW_10_6_X_2020-01-31-1100/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Fri Jan 31 18:45:00 2020-date Fri Jan 31 18:44:00 2020 s - exit: 21248
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:45:00 2020-date Fri Jan 31 18:44:00 2020 s - exit: 23040
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw-patch/CMSSW_10_6_X_2020-01-31-1100/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Fri Jan 31 18:45:19 2020-date Fri Jan 31 18:44:02 2020 s - exit: 21248
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018_pp_on_AA --processName=HLTRECO --filein file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:45:19 2020-date Fri Jan 31 18:44:02 2020 s - exit: 23040
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw-patch/CMSSW_10_6_X_2020-01-31-1100/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Fri Jan 31 18:48:53 2020-date Fri Jan 31 18:44:07 2020 s - exit: 21248
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:53 2020-date Fri Jan 31 18:44:07 2020 s - exit: 23040
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - time: date Fri Jan 31 18:44:45 2020-date Fri Jan 31 18:44:11 2020 s - exit: 23040
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw-patch/CMSSW_10_6_X_2020-01-31-1100/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Fri Jan 31 18:48:25 2020-date Fri Jan 31 18:44:12 2020 s - exit: 21248
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:25 2020-date Fri Jan 31 18:44:12 2020 s - exit: 23040
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - time: date Fri Jan 31 18:44:51 2020-date Fri Jan 31 18:44:15 2020 s - exit: 23040
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:06 2020-date Fri Jan 31 18:44:20 2020 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:37 2020-date Fri Jan 31 18:44:21 2020 s - exit: 23040
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw-patch/CMSSW_10_6_X_2020-01-31-1100/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Fri Jan 31 18:48:43 2020-date Fri Jan 31 18:44:26 2020 s - exit: 21248
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018_pp_on_AA --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:43 2020-date Fri Jan 31 18:44:26 2020 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_MC.root --fileout file:RelVal_Raw_Fake2_MC_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:36 2020-date Fri Jan 31 18:44:28 2020 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_MC.root --fileout file:RelVal_Raw_Fake1_MC_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:55 2020-date Fri Jan 31 18:44:33 2020 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run1_data_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_DATA.root --fileout file:RelVal_Raw_Fake_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:06 2020-date Fri Jan 31 18:44:41 2020 s - exit: 23040
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw-patch/CMSSW_10_6_X_2020-01-31-1100/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Fri Jan 31 18:48:38 2020-date Fri Jan 31 18:44:43 2020 s - exit: 21248
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:38 2020-date Fri Jan 31 18:44:43 2020 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_DATA.root --fileout file:RelVal_Raw_Fake1_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:48:06 2020-date Fri Jan 31 18:44:46 2020 s - exit: 23040
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Fri Jan 31 18:49:48 2020-date Fri Jan 31 18:44:47 2020 s - exit: 23040

cmsbuild · 2020-01-31T17:51:00Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

rchatter · 2020-01-31T18:00:35Z

Sorry for this. Will investigate and re-commit.

slava77 · 2020-02-03T09:25:06Z

@cmsbuild please test with cms-sw/cmsdist#5516

cmsbuild · 2020-02-03T09:25:30Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#5516
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4464/console Started: 2020/02/03 10:34

cmsbuild · 2020-02-04T22:50:59Z

+1
Tested at: d9ae2a2
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ea2591/4498/summary.html
CMSSW: CMSSW_10_6_X_2020-02-04-1100
SCRAM_ARCH: slc7_amd64_gcc700

cmsbuild · 2020-02-04T22:51:05Z

Comparison job queued.

cmsbuild · 2020-02-05T00:21:13Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ea2591/4498/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3212324
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3211989
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
Checked 137 log files, 14 edm output root files, 33 DQM output files

perrotta · 2020-02-05T08:57:46Z

RecoLocalCalo/EcalDeadChannelRecoveryAlgos/interface/EcalDeadChannelRecoveryAlgos.h


- private:
-  EcalDeadChannelRecoveryNN<DetIdT> nn;


This is just to be sure: backports into a production release must satisfy the "no change" rule. Here you remove the NN method for the deadchannel recovery, and substitute it with the BDTG one.
The default setting for recoverEBIsolatedChannels is false, and I don't see in CMSSW any place or official workflow in which it is turned into true. Therefore, I assume that the backport cannot modify the result in any of those official workflows.
Can you confirm that there are no use cases in 10_6_X that require such NN method for the ecal deadchannel recovery, and it id therefore safe to fully replace it with the BDTG one?

nancymarinelli · 2020-02-05T09:09:34Z

On 05/02/20 09:57, perrotta wrote: perrotta commented on this pull request. > - private: - EcalDeadChannelRecoveryNN<DetIdT> nn; This is just to be sure: backports into a production release must satisfy the "no change" rule. Here you remove the NN method for the deadchannel recovery, and substitute it with the BDTG one. The default setting for ```recoverEBIsolatedChannels``` is ```false```, and I don't see in CMSSW any place or official workflow in which it is turned into ```true```. Therefore, I assume that the backport cannot modify the result in any of those official workflows.

No modification of the official workflows. A special configuration will be made in due time, according to plans with PPD for the large validation scale

Can you confirm that there are no use cases in 10_6_X that require such NN method for the ecal deadchannel recovery, and it id therefore safe to fully replace it with the BDTG one?

Hi, no use cases for NN.

…

--

________________________________________ Nancy Marinelli Research Associate Professor University of Notre Dame, IN, US CERN, Bdg 40/3-A01, 1211 Geneva SWITZERLAND Phone +41-22-76-70809 fax +41-22-76-78940

perrotta · 2020-02-05T09:43:46Z

+1

Backport of Ecal DPG - Deploy new software (based on BDT) to recover isolated dead channels… #27175, and the subsequent code optimizations implemented in master for the same
It requires [10.6.X] update RecoLocalCalo-EcalDeadChannelRecoveryAlgos to V01-01-00 cmsdist#5516
It replaces the previous Ecal dead channel recovery method based on NN with a new one based on BDT
- Ecal dead channel recovery is not enabled in any standard or used workflow, therefore removing the NN based recovery method does not affect the output of any such workflow in 10_6
Several updated files have been directly copied from the 11_X version: this brings together all clang formatting changes that were applied in the meanwhile, unrelated to this PR. This is not harming and it does not modify the flow of the code, as I checked explicitely
Jenkins tests pass and show no differences ,as expected

cmsbuild · 2020-02-05T09:44:14Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @davidlange6, @silviodonato, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-02-05T21:48:42Z

code-checks

cmsbuild · 2020-02-05T21:49:06Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-02-05T21:56:04Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28831/13646

This PR adds an extra 8KB to repository

silviodonato

I found three small differences between #28831 and #27175.
Could you confirm they are harmless?

RecoLocalCalo/EcalDeadChannelRecoveryAlgos/interface/EcalDeadChannelRecoveryBDTG.h

RecoLocalCalo/EcalRecProducers/plugins/EcalRecHitProducer.cc

silviodonato · 2020-02-07T14:47:23Z

+1
all differences were simply cleaning and code-checks changes.

Martin-Grunewald · 2020-02-10T09:38:00Z

Seems this PR messes up many 10_6 IB relvals.

Martin-Grunewald · 2020-02-10T09:38:22Z

----- Begin Fatal Exception 10-Feb-2020 10:35:04 CET-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
Exception Message:
edm::FileInPath unable to find file RecoLocalCalo/EcalDeadChannelRecoveryAlgos/\
data/BDTWeights/bdtgAllRH_8GT700MeV_noCracks_ZskimData2017_v1.xml anywhere in t\
he search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /data/CMS6/CMSSW_10_6_X_2020-02-09-0000/poison:/data/C\
MS6/CMSSW_10_6_X_2020-02-09-0000/src:/data/CMS6/CMSSW_10_6_X_2020-02-09-0000/ex\
ternal/slc7_amd64_gcc700/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms\
/cmssw/CMSSW_10_6_X_2020-02-09-0000/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_\
gcc700/cms/cmssw/CMSSW_10_6_X_2020-02-09-0000/external/slc7_amd64_gcc700/data
Current directory is: /data/CMS6/CMSSW_10_6_X_2020-02-09-0000/src/HLTrigger/Con\
figuration/test
----- End Fatal Exception -------------------------------------------------

perrotta · 2020-02-10T09:47:41Z

As I wrote in the approval notes, this PR requires an external: cms-sw/cmsdist#5516
Please @silviodonato also merge that one

silviodonato · 2020-02-10T09:59:28Z

Right, I have merged them right now. Thanks @Martin-Grunewald @perrotta

nancymarinelli and others added 2 commits January 30, 2020 16:40

Cherry pick files from PR27175 for Backport

09ae771

Fix unique-ptr related issue in EcalRecHitProducer.cc

606742e

cmsbuild added this to the CMSSW_10_6_X milestone Jan 30, 2020

cmsbuild added comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jan 30, 2020

cmsbuild added tests-started and removed tests-pending labels Jan 31, 2020

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Jan 31, 2020

cmsbuild added comparison-pending tests-pending and removed comparison-notrun tests-rejected labels Feb 3, 2020

cmsbuild mentioned this pull request Feb 3, 2020

[10.6.X] update RecoLocalCalo-EcalDeadChannelRecoveryAlgos to V01-01-00 cms-sw/cmsdist#5516

Merged

cmsbuild added requires-external tests-started and removed tests-pending labels Feb 3, 2020

cmsbuild added requires-external tests-started labels Feb 4, 2020

cmsbuild added tests-approved and removed tests-started labels Feb 4, 2020

cmsbuild added comparison-available and removed comparison-pending labels Feb 5, 2020

perrotta reviewed Feb 5, 2020

View reviewed changes

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Feb 5, 2020

silviodonato reviewed Feb 7, 2020

View reviewed changes

RecoLocalCalo/EcalDeadChannelRecoveryAlgos/interface/EcalDeadChannelRecoveryBDTG.h Show resolved Hide resolved

RecoLocalCalo/EcalRecProducers/plugins/EcalRecHitProducer.cc Show resolved Hide resolved

RecoLocalCalo/EcalRecProducers/plugins/EcalRecHitProducer.cc Show resolved Hide resolved

cmsbuild added orp-approved and removed orp-pending labels Feb 7, 2020

cmsbuild merged commit 256a5fd into cms-sw:CMSSW_10_6_X Feb 7, 2020

rchatter mentioned this pull request Feb 20, 2020

Backport: ECAL cluster seed flags and ability to apply updated seeding thresholds from GT #29010

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ecal DPG - Backporting new software[PR 27175] to recover isolated new channels #28831

Ecal DPG - Backporting new software[PR 27175] to recover isolated new channels #28831

rchatter commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

slava77 commented Jan 31, 2020

cmsbuild commented Jan 31, 2020 •

edited

Loading

cmsbuild commented Jan 31, 2020

cmsbuild commented Jan 31, 2020

rchatter commented Jan 31, 2020

slava77 commented Feb 3, 2020

cmsbuild commented Feb 3, 2020 •

edited

Loading

cmsbuild commented Feb 4, 2020

cmsbuild commented Feb 4, 2020

cmsbuild commented Feb 5, 2020

perrotta Feb 5, 2020

nancymarinelli commented Feb 5, 2020 via email

perrotta commented Feb 5, 2020 •

edited

Loading

cmsbuild commented Feb 5, 2020

silviodonato commented Feb 5, 2020

cmsbuild commented Feb 5, 2020

cmsbuild commented Feb 5, 2020

silviodonato left a comment

silviodonato commented Feb 7, 2020

Martin-Grunewald commented Feb 10, 2020

Martin-Grunewald commented Feb 10, 2020

perrotta commented Feb 10, 2020

silviodonato commented Feb 10, 2020

Ecal DPG - Backporting new software[PR 27175] to recover isolated new channels #28831

Ecal DPG - Backporting new software[PR 27175] to recover isolated new channels #28831

Conversation

rchatter commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

slava77 commented Jan 31, 2020

cmsbuild commented Jan 31, 2020 • edited Loading

cmsbuild commented Jan 31, 2020

cmsbuild commented Jan 31, 2020

rchatter commented Jan 31, 2020

slava77 commented Feb 3, 2020

cmsbuild commented Feb 3, 2020 • edited Loading

cmsbuild commented Feb 4, 2020

cmsbuild commented Feb 4, 2020

cmsbuild commented Feb 5, 2020

perrotta Feb 5, 2020

Choose a reason for hiding this comment

nancymarinelli commented Feb 5, 2020 via email

perrotta commented Feb 5, 2020 • edited Loading

cmsbuild commented Feb 5, 2020

silviodonato commented Feb 5, 2020

cmsbuild commented Feb 5, 2020

cmsbuild commented Feb 5, 2020

silviodonato left a comment

Choose a reason for hiding this comment

silviodonato commented Feb 7, 2020

Martin-Grunewald commented Feb 10, 2020

Martin-Grunewald commented Feb 10, 2020

perrotta commented Feb 10, 2020

silviodonato commented Feb 10, 2020

cmsbuild commented Jan 31, 2020 •

edited

Loading

cmsbuild commented Feb 3, 2020 •

edited

Loading

perrotta commented Feb 5, 2020 •

edited

Loading