Drop of D49-D85 Phase-2 geometry #38594

srimanob · 2022-07-04T19:51:43Z

Following the HGCAL proposal in
https://docs.google.com/document/d/1fCIpJJQLw0vGCKyR9LRmFmWVcAQZOlHk5jK-ESQLJnQ/edit?usp=sharing,
they propose to preserve only V16 and V17 of HGCAL (== C17-C19).

So the affected geometries include

D49 = T15+C9+M4+I10+O4+F2 (HLT TDR baseline)
D60 = T15+C10+M4+I10+O4+F3 (With HFNose)
D68 = T21+C11+M6+I11+O5+F4 (For HGCAL study on evolution of detector)
D70 = T21+C13+M7+I11+O6+F6 (For HGCAL study on evolution of detector)
D76 = T21+C14+M9+I13+O7+F6
D77 = T24+C14+M9+I13+O7+F6 (Current default scenario)
D80 = T25+C14+M9+I13+O7+F6
D81 = T26+C14+M9+I13+O7+F6
D82 = T21+C15+M9+I13+O7+F7
D83 = T24+C16+M9+I13+O7+F6
D84 = T24+C13+M7+I11+O6+F6 (For HGCAL study on evolution of HGCal replacing D70)
D85 = T24+C14+M9+I14+O7+F6

This issue is to collect feedback, or need if other version of geometry should be preserved.

The plan is to implement this proposal when the next release series start, i.e. 12_6_X.

cmsbuild · 2022-07-04T19:52:04Z

A new Issue was created by @srimanob Phat Srimanobhas.

@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

srimanob · 2022-07-04T19:53:27Z

FYI @kpedro88 @bsunanda @AdrianoDee

@trtomei @fwyzard
Do you still need to keep D49 for HLT?

srimanob · 2022-07-04T19:54:53Z

assign upgrade

cmsbuild · 2022-07-04T19:55:14Z

New categories assigned: upgrade

@AdrianoDee,@srimanob you have been requested to review this Pull request/Issue and eventually sign? Thanks

srimanob · 2022-07-04T19:55:44Z

FYI @cms-sw/simulation-l2 @cms-sw/hgcal-dpg-l2

kpedro88 · 2022-07-04T19:57:02Z

will need to add a new scenario with HF nose (compatible with one of the latest HGCal geometries)
@srimanob @AdrianoDee we may want to take this opportunity to reset the workflow numbers... it's possibly been three years since we last did this in Clean up Phase2 workflows #27449

fabiocos · 2022-07-05T07:43:39Z

from the MTD point of view keeping just scenario I15 is ok, this will allow us to drop support in the code for obsolete scenarios (like single disc ring-shaped ETL in D49), simplifying the structure. Some initial cleaning is already proposed for BTL in #38589 (not needing this proposal), in preparation of a new BTL geometry (and MTD scenario) to be added in the coming weeks/months.

srimanob · 2022-07-06T06:31:42Z

will need to add a new scenario with HF nose (compatible with one of the latest HGCal geometries)

This is handled in #38596 (Thanks @bsunanda)

srimanob · 2022-07-12T09:51:45Z

I've made a draft PR, so ppl can check:
#38702

kpedro88 · 2022-07-12T16:44:04Z

@srimanob what do you think about resetting the workflow numbers?

srimanob · 2022-07-12T17:06:02Z

Hi @kpedro88
Yeah, that is what I would like to ask you. If D86 will be the first workflow, should it just be 20000.0 after resetting?

Currently, the first upgrade workflow after cleaning will be
38600.0 2026D86+FourMuPt_1_200_pythia8_GenSimHLBeamSpot+DigiTrigger+RecoGlobal+HARVESTGlobal

kpedro88 · 2022-07-12T18:42:08Z

Yes; in particular, you should change this line as follows: numWFConflict = [[25000,26000],[50000,51000]]

srimanob · 2022-07-12T19:18:16Z

OK. That line I updated already in the PR, but just to reflect the drop of workflows. I will update again.

cbotta · 2022-07-13T09:57:35Z

Hello. The L1T team would like to ask if the D49 could be kept. It can be useful for validation studies on the HLT-TDR samples (that we are using for L1T developments since a year). And I think HLT Upgrade would echo this.

srimanob · 2022-07-13T13:21:20Z

@cbotta Do you have an idea on how long will you need it? Could you please provide more information on the schedule of development?

The current plan is to drop by the next release series, i.e. 12_6 at the end of August.

cbotta · 2022-07-14T09:52:05Z

@srimanob After discussing with the team: we will very likely have our new MC production in 12_5. We think it is useful to keep the D49 up to this release, so that we have one release that can run on both MC productions. But after that we will switch to the new samples, and so it is fine to remove it from 12_6. Thanks!

srimanob · 2022-07-14T17:16:21Z

@cbotta
This is great, thanks. I will add this bullet to tomorrow SIM meeting.

srimanob · 2022-09-19T14:04:15Z

Hi @cbotta @cms-sw/l1-l2
May I come back to the question on dropping geometries. It is unclear to me as we are not done yet the L1T dev, could you please comment.

FYI @bsunanda @cms-sw/simulation-l2 @cms-sw/hcal-dpg-l2

cbotta · 2022-09-19T14:24:36Z

Hi. My understanding is that L1T is still targeting 12_5_0 for the MC production, therefore my previous comment still holds. @cecilecaillol

srimanob · 2022-09-19T14:26:50Z

Hi @cbotta
Thanks. The question is if we can handle the cleaning in master (12_6) now, or we should wait a bit until all L1T devs converge.

cbotta · 2022-09-19T14:30:45Z

If possible I would wait for @cecilecaillol to confirm that all what we need can be ported to 12_5, and we can start the production there. Should be happening soon

srimanob · 2022-09-23T13:57:17Z

@cbotta @cecilecaillol @cms-sw/l1-l2
After the simulation meeting today, there are requests to merge the clean up PR #38702 asap. D49 will not be supported anymore in 12_6.

From your plan, it should be OK as all L1T code will be backported to 12_5, so 12_5 will be the release which support both D49 and D88. If there is an objection, please raise it here before the next release meeting (27 Sep). Thanks for understanding.

FYI @bsunanda @fabiocos

cecilecaillol · 2022-09-23T18:55:29Z

We should have a 12_5_1 release for our MC production so this is fine from our side

srimanob · 2022-09-29T07:30:07Z

Unit test failures after merging cleaning PR:

Validation/Geometry/test/runP_HGCal_cfg.py
Geometry/TrackerGeometryBuilder/test/python/testPixelTopologyMapTest_cfg.py
SimTracker/TrackerMaterialAnalysis/test/trackingMaterialProducer10GeVNeutrino_ForPhaseII.py
SLHCUpgradeSimulations/Geometry/test/phase2_digi_reco_pixelntuple_cfg.py
SLHCUpgradeSimulations/Geometry/test/writeFile_phase2_cfg.py
CondTools/SiPixel/test/SiPixelTemplateDBObjectUploader_Phase2_cfg.py

They call D49 or D76. This should be updated to D88 or later.

FYI @cms-sw/geometry-l2 @cms-sw/trk-dpg-l2
Should I update all to D88? If OK, I can provide the PR. Thanks.

bsunanda · 2022-09-29T07:39:32Z

Please use D92 (or the later version with new Tracker and MTD) and not D88.

…

________________________________ From: Phat Srimanobhas ***@***.*** Sent: 29 September 2022 09:30 To: cms-sw/cmssw Cc: Sunanda Banerjee; Mention Subject: Re: [cms-sw/cmssw] Drop of D49-D85 Phase-2 geometry (Issue #38594) Unit test failures after merging cleaning PR: * Validation/Geometry/test/runP_HGCal_cfg.py * Geometry/TrackerGeometryBuilder/test/python/testPixelTopologyMapTest_cfg.py * SimTracker/TrackerMaterialAnalysis/test/trackingMaterialProducer10GeVNeutrino_ForPhaseII.py * SLHCUpgradeSimulations/Geometry/test/writeFile_phase2_cfg.py * CondTools/SiPixel/test/SiPixelTemplateDBObjectUploader_Phase2_cfg.py They call D49 or D76. This should be updated to D88 or later. FYI @cms-sw/geometry-l2<https://github.com/orgs/cms-sw/teams/geometry-l2> @cms-sw/trk-dpg-l2<https://github.com/orgs/cms-sw/teams/trk-dpg-l2> — Reply to this email directly, view it on GitHub<#38594 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABGMZOQQKLZA5KOVNKDQVNTWAVAQZANCNFSM52UCJQSQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

mmusich · 2022-09-29T07:51:13Z

@srimanob I suppose you can also clean up the T* versioned autoCond Global Tags for the Tracker geometries that have been removed?

cmssw/Configuration/AlCa/python/autoCondPhase2.py

Lines 110 to 119 in 4e89b5b

    
           activeDets = ["T15","T21","T25","T26","T30"] 
        
           phase2GTs = {} 
        
           for det in activeDets: 
        
               appendedTags = () 
        
               for key in activeKeys: 
        
                   if (det in allTags[key]): 
        
                       appendedTags += allTags[key][det] 
        
                   else : 
        
                       pass 
        
               phase2GTs["phase2_realistic_"+det] = ('phase2_realistic', appendedTags)

@cms-sw/alca-l2

srimanob · 2022-09-29T07:55:42Z

@srimanob I suppose you can also clean up the T* versioned autoCond Global Tags for the Tracker geometries that have been removed?

cmssw/Configuration/AlCa/python/autoCondPhase2.py

Lines 110 to 119 in 4e89b5b

activeDets = ["T15","T21","T25","T26","T30"]

phase2GTs = {}

for det in activeDets:

appendedTags = ()

for key in activeKeys:

if (det in allTags[key]):

appendedTags += allTags[key][det]

else :

pass

phase2GTs["phase2_realistic_"+det] = ('phase2_realistic', appendedTags)

@cms-sw/alca-l2

Hi @mmusich @cms-sw/alca-l2
Sure, I can include in my PR. Thanks for information.

mmusich · 2022-09-29T08:51:28Z

Please use D92 (or the later version with new Tracker and MTD) and not D88.

I see a couple of problems:

in:

cmssw/SLHCUpgradeSimulations/Geometry/test/phase2_digi_reco_pixelntuple_cfg.py

Line 32 in 4e89b5b

'/store/relval/CMSSW_11_2_0_pre1/RelValSingleMuPt10/GEN-SIM/110X_mcRun4_realistic_v3_2026D49noPU-v1/10000/743B02CC-F5B9-5642-A7EF-EE222E18C54F.root'

as far as I can see there is no suitable GEN-SIM dataset for D92¹

in:

cmssw/CondTools/SiPixel/test/createTestDBObjects.sh

Line 20 in 4e89b5b

    
           cmsRun  ${LOCAL_TEST_DIR}/SiPixel2DTemplateDBObjectUploader_Phase2_cfg.py MagField=3.8 Version=7 Append=mc_25x100 Map=${LOCAL_TEST_DIR}/../data/phase2_T15_mapping.csv TemplateFilePath=CalibTracker/SiPixelESProducers/data/Phase2_25by100_templates_2020October denominator=True || die "Failure running SiPixel2DTemplateDBObjectUploader_Phase2_cfg.py" $?

we don't have the suitable mapping files in https://github.com/cms-sw/cmssw/tree/master/CondTools/SiPixel/data nor in https://github.com/cms-data/CalibTracker-SiPixelESProducers (@tvami FYI)

from DAS $ dasgoclient -query='dataset dataset=/RelValSingleMuPt10/*2026D92*/GEN-SIM' | wc -l is 0 ↩

srimanob · 2022-09-29T11:09:59Z

Let's go for D88 in unit tests until we validate D92 and move it to baseline, and have everything ready.

mmusich · 2022-09-29T11:47:26Z

@srimanob

PR #39535 should take care of these:

Geometry/TrackerGeometryBuilder/test/python/testPixelTopologyMapTest_cfg.py
SimTracker/TrackerMaterialAnalysis/test/trackingMaterialProducer10GeVNeutrino_ForPhaseII.py
SLHCUpgradeSimulations/Geometry/test/phase2_digi_reco_pixelntuple_cfg.py
SLHCUpgradeSimulations/Geometry/test/writeFile_phase2_cfg.py

as for the others:

Validation/Geometry/test/runP_HGCal_cfg.py

I would let it to HGCAL experts (FYI @cms-sw/hgcal-dpg-l2 )

while

CondTools/SiPixel/test/SiPixelTemplateDBObjectUploader_Phase2_cfg.py

requires more involved changes, including an update of the cms-data externals, that I would leave to the Pixel CPE experts (@tvami @SanjanaSekhar)

tvami · 2022-09-29T12:08:54Z

CondTools/SiPixel/test/SiPixelTemplateDBObjectUploader_Phase2_cfg.py
requires more involved changes, including an update of the cms-data externals, that I would leave to the Pixel CPE experts (@tvami @SanjanaSekhar)

Sure I'll have a look

mmusich · 2022-09-29T13:29:17Z

changing topic, shouldn't be https://github.com/cms-sw/cmssw/blob/master/Configuration/Geometry/README.md also updated to remove the unsupported geometries?

srimanob · 2022-09-29T14:06:30Z

changing topic, shouldn't be https://github.com/cms-sw/cmssw/blob/master/Configuration/Geometry/README.md also updated to remove the unsupported geometries?

Thx. I think i forgot it whilt waiting the PR to converge. I will update it together with workflow id resetting.

srimanob · 2022-10-04T08:41:27Z

Validation/Geometry/test/runP_HGCal_cfg.py
Proposed PR is Migrate Validation/Geometry to D92 #39590
FYI @cms-sw/hgcal-dpg-l2 @cms-sw/geometry-l2

srimanob · 2022-10-06T06:34:52Z

Staring from CMSSW_12_6_X_2022-10-05-1100, unit tests seem to be OK now.

srimanob · 2022-11-03T14:10:29Z

Done.

cmsbuild added the pending-assignment label Jul 4, 2022

cmsbuild added pending-signatures upgrade-pending and removed pending-assignment labels Jul 4, 2022

srimanob changed the title ~~Drop of C9-C16 Phase-2 geometry~~ Drop of D49-D85 Phase-2 geometry Jul 11, 2022

srimanob mentioned this issue Jul 12, 2022

Clean up Phase-2 geometry (Up to D85) #38702

Merged

mmusich mentioned this issue Sep 29, 2022

fix failing unit tests after #38702 merge (Drop of geometry D49) #39535

Merged

srimanob mentioned this issue Oct 2, 2022

Reset Phase-2 workflow ID #39571

Merged

tvami mentioned this issue Oct 2, 2022

Change Phase-2 CPE unit test to use T21 geom #39572

Merged

srimanob mentioned this issue Oct 4, 2022

Migrate Validation/Geometry to D92 #39590

Merged

srimanob closed this as completed Nov 3, 2022

epalencia mentioned this issue May 9, 2023

L1track jet emulator update (backport of PR40563) cms-l1t-offline/cmssw#1116

Closed

Drop of D49-D85 Phase-2 geometry #38594

Drop of D49-D85 Phase-2 geometry #38594

Comments

srimanob commented Jul 4, 2022 • edited Loading

cmsbuild commented Jul 4, 2022

srimanob commented Jul 4, 2022

srimanob commented Jul 4, 2022

cmsbuild commented Jul 4, 2022

srimanob commented Jul 4, 2022

kpedro88 commented Jul 4, 2022

fabiocos commented Jul 5, 2022

srimanob commented Jul 6, 2022

srimanob commented Jul 12, 2022

kpedro88 commented Jul 12, 2022

srimanob commented Jul 12, 2022

kpedro88 commented Jul 12, 2022

srimanob commented Jul 12, 2022

cbotta commented Jul 13, 2022

srimanob commented Jul 13, 2022

cbotta commented Jul 14, 2022

srimanob commented Jul 14, 2022

srimanob commented Sep 19, 2022

cbotta commented Sep 19, 2022

srimanob commented Sep 19, 2022

cbotta commented Sep 19, 2022

srimanob commented Sep 23, 2022

cecilecaillol commented Sep 23, 2022

srimanob commented Sep 29, 2022 • edited Loading

bsunanda commented Sep 29, 2022 via email

mmusich commented Sep 29, 2022

srimanob commented Sep 29, 2022

mmusich commented Sep 29, 2022

Footnotes

srimanob commented Sep 29, 2022 • edited Loading

mmusich commented Sep 29, 2022

tvami commented Sep 29, 2022

mmusich commented Sep 29, 2022

srimanob commented Sep 29, 2022

srimanob commented Oct 4, 2022

srimanob commented Oct 6, 2022

srimanob commented Nov 3, 2022

srimanob commented Jul 4, 2022 •

edited

Loading

srimanob commented Sep 29, 2022 •

edited

Loading

srimanob commented Sep 29, 2022 •

edited

Loading