HGCal raw data handling: Unpacker, Geometry, Local Reco

[IzaakWN]

PR description:

This adds work from the Raw Data Handling Group of the HGCal DPG.

This PR includes:

1. Updates to the unpacker (for the test beam last week)
   • Follow up on PR HGCal emulator and unpacker #41461.
   • We moved the emulators to more appropriate location (EventFilter/HGCalRawToDigi → SimCalorimetry/HGCalSimAlgos), they will be subject of a future PR.
2. Updates to the logical mapping & (dense) indexing in Geometry/HGCalMapping.
   • Follow up PR HGCAL Electronics mapping #43751.
3. The introduction of electronics configuration & calibrations, read from JSONs in CondFormats.
   • No need to create special tag for these records yet as this is work in progress.
4. New Alpaka SoA formats for DIGI and RecHits in DataFormats/HGCalDigi/interface/HGCalDigiSoA.h.
5. Alpaka calibration kernels for the DIGI → RecHit step.
6. Adding TH2Poly for DQM. Removed from this PR (see below).

We kindly request this to be merged into CMSSW_14_1_0_pre7 in time for the coming September test beams where validation will continue.

PR validation:

No CMSSW workflow is impacted by this PR.

We validated this with test beam data last week. You can find latest DQM plots here: http://hgc-vm-2024.cern.ch:8070/dqm/online-dev/session

Further documentation

• https://indico.cern.ch/event/1417813/#7-raw-data-handling-towards-be
• We are working on documentation for the HGCal website: https://hgcaldocs.web.cern.ch (Gitlab)

[cmsbuild]

cms-bot internal usage

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45665/41204

• Found files with invalid states:

  • SimCalorimetry/HGCalSimProducers/plugins/HGCalDigiSoaFiller.cc:
    • Added: f4f5e5a
    • Deleted: 82e90fb
  • RecoLocalCalo/HGCalRecAlgos/test/testHGCalRecHitESProducer_cfg.py:
    • Added: 5bdf753
    • Modified: 394aade, 2ebc1fb, e109222
    • Deleted: 72e2507
  • RecoLocalCalo/HGCalRecAlgos/src/HGCalCalibrationParameterHostCollection.cc:
    • Added: b110814
    • Modified: 394aade
    • Deleted: 2ebc1fb
  • CondFormats/HGCalObjects/src/HGCalCalibrationParameterHost.cc:
    • Added: 2ebc1fb
    • Deleted: 5d6cfa2
  • DataFormats/HGCalDigi/interface/HGCalDigiHostCollection.h:
    • Added: f4f5e5a
    • Deleted: f381b45
  • CondFormats/DataRecord/src/HGCalMappingModuleIndexerRcd.cc:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • Geometry/HGCalMapping/data/modulelocator.txt:
    • Added: f02b407
    • Deleted: 5c5ff69
  • RecoLocalCalo/HGCalRecAlgos/src/alpaka/HGCalCalibrationParameterDeviceCollection.cc:
    • Added: b59f356
    • Deleted: 2ebc1fb
  • CondFormats/HGCalObjects/src/headers.h:
    • Deleted: 8b0f706
    • Added: 404be7d
    • Modified: e73baa9
  • RecoLocalCalo/HGCalRecAlgos/src/ES_HGCalCalibrationParameter.cc:
    • Added: 5bdf753
    • Deleted: b110814
  • CondFormats/HGCalObjects/src/alpaka/HGCalMappingParameterDeviceCollection.cc:
    • Added: b59f356
    • Deleted: f381b45
  • DataFormats/HGCalDigi/interface/alpaka/HGCalDigiDeviceCollection.h:
    • Added: f4f5e5a
    • Deleted: f381b45
  • RecoLocalCalo/HGCalRecAlgos/interface/HGCalCalibrationParameterHostCollection.h:
    • Added: 5bdf753
    • Modified: 394aade
    • Deleted: 2ebc1fb
  • RecoLocalCalo/HGCalRecAlgos/plugins/alpaka/TestHGCalRecHitESProducers.cc:
    • Added: 5bdf753
    • Modified: 394aade
    • Deleted: 2ebc1fb
  • CondFormats/DataRecord/interface/HGCalMappingModuleIndexerRcd.h:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • CondFormats/DataRecord/interface/HGCalMappingCellRcd.h:
    • Added: 0baf56d
    • Deleted: 7975bbd
  • CondFormats/DataRecord/src/HGCalElectronicsMappingRcd.cc:
    • Deleted: 0baf56d
    • Added: 7975bbd
  • CondFormats/HGCalObjects/src/HGCalMappingParameterHostCollection.cc:
    • Added: b110814
    • Deleted: f381b45
  • Geometry/HGCalMapping/data/modulelocator_test.txt:
    • Added: f02b407
    • Deleted: 5c5ff69
  • RecoLocalCalo/HGCalRecAlgos/interface/HGCalCalibrationParameterSoA.h:
    • Added: 5bdf753
    • Modified: 394aade
    • Deleted: 2ebc1fb
  • DataFormats/HGCalDigi/interface/HGCalECONDInfoSoA.h:
    • Added: 9756f10
    • Modified: 415e9ed, 49f9cfb
    • Deleted: 762c834
  • Geometry/HGCalMapping/data/channels_sipmontile.hgcal.txt:
    • Added: f02b407
    • Deleted: 5c5ff69
  • CondFormats/DataRecord/src/HGCalMappingCellRcd.cc:
    • Added: 0baf56d
    • Deleted: 7975bbd
  • CondFormats/DataRecord/src/HGCalMappingCellIndexerRcd.cc:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • CondFormats/DataRecord/src/HGCalMappingModuleRcd.cc:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • Geometry/HGCalMapping/data/WaferCellMapTraces.txt:
    • Added: f02b407
    • Deleted: 5c5ff69
  • DataFormats/HGCalDigi/interface/alpaka/HGCalECONDInfoDevice.h:
    • Added: 9756f10
    • Deleted: 762c834
  • SimCalorimetry/HGCalSimAlgos/interface/HGCalRawDataDefinitions.h:
    • Added: 4fae6f8
    • Deleted: 2a083a9
  • RecoLocalCalo/HGCalRecAlgos/src/alpaka/HGCalCalibrationParameterHostCollection.cc:
    • Added: 4fad636
    • Deleted: b59f356
  • CondFormats/DataRecord/interface/HGCalMappingCellIndexerRcd.h:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • RecoLocalCalo/HGCalRecAlgos/plugins/alpaka/HGCalRecHitProducer.cc:
    • Added: e109222
    • Modified: 7975bbd, 02155c7
    • Deleted: 2247d87
  • DataFormats/HGCalDigi/interface/HGCalECONDInfoHost.h:
    • Added: 9756f10
    • Modified: 415e9ed
    • Deleted: 762c834
  • Geometry/HGCalMapping/plugins/HGCalMappingIndexESSource.cc:
    • Added: f02b407
    • Modified: 5c5ff69, a21eec7, 0baf56d, f73ce1d, 3de5708, 3907071
    • Deleted: 577e6d6
  • RecoLocalCalo/HGCalRecAlgos/src/alpaka/ES_HGCalCalibrationParameter.cc:
    • Added: 5bdf753
    • Deleted: 4fad636
  • CondFormats/HGCalObjects/src/alpaka/HGCalCalibrationParameterDevice.cc:
    • Added: 2ebc1fb
    • Deleted: 5d6cfa2
  • RecoLocalCalo/HGCalRecAlgos/interface/alpaka/HGCalCalibrationParameterDeviceCollection.h:
    • Added: 5bdf753
    • Deleted: 2ebc1fb
  • CondFormats/DataRecord/interface/HGCalMappingModuleRcd.h:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • CondFormats/HGCalObjects/src/alpaka/HGCalMappingParameterHostCollection.cc:
    • Added: 4fad636
    • Deleted: b59f356

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DQMServices/Core/interface/DQMStore.h modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932
  • File DQMServices/Core/interface/MonitorElement.h modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932
  • File DQMServices/Core/src/MonitorElement.cc modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932
  • File DataFormats/Histograms/interface/MEtoEDMFormat.h modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932
  • File DataFormats/Histograms/interface/MonitorElementCollection.h modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45665/41204/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45665/41204/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[pfs]

assign hgcal-dpg

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45665/41205

• Found files with invalid states:

  • SimCalorimetry/HGCalSimProducers/plugins/HGCalDigiSoaFiller.cc:
    • Added: f4f5e5a
    • Deleted: 82e90fb
  • RecoLocalCalo/HGCalRecAlgos/test/testHGCalRecHitESProducer_cfg.py:
    • Added: 5bdf753
    • Modified: 394aade, 2ebc1fb, e109222
    • Deleted: 72e2507
  • RecoLocalCalo/HGCalRecAlgos/src/HGCalCalibrationParameterHostCollection.cc:
    • Added: b110814
    • Modified: 394aade
    • Deleted: 2ebc1fb
  • CondFormats/HGCalObjects/src/HGCalCalibrationParameterHost.cc:
    • Added: 2ebc1fb
    • Deleted: 5d6cfa2
  • DataFormats/HGCalDigi/interface/HGCalDigiHostCollection.h:
    • Added: f4f5e5a
    • Deleted: f381b45
  • CondFormats/DataRecord/src/HGCalMappingModuleIndexerRcd.cc:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • Geometry/HGCalMapping/data/modulelocator.txt:
    • Added: f02b407
    • Deleted: 5c5ff69
  • RecoLocalCalo/HGCalRecAlgos/src/alpaka/HGCalCalibrationParameterDeviceCollection.cc:
    • Added: b59f356
    • Deleted: 2ebc1fb
  • CondFormats/HGCalObjects/src/headers.h:
    • Deleted: 8b0f706
    • Added: 404be7d
    • Modified: e73baa9
  • RecoLocalCalo/HGCalRecAlgos/src/ES_HGCalCalibrationParameter.cc:
    • Added: 5bdf753
    • Deleted: b110814
  • CondFormats/HGCalObjects/src/alpaka/HGCalMappingParameterDeviceCollection.cc:
    • Added: b59f356
    • Deleted: f381b45
  • DataFormats/HGCalDigi/interface/alpaka/HGCalDigiDeviceCollection.h:
    • Added: f4f5e5a
    • Deleted: f381b45
  • RecoLocalCalo/HGCalRecAlgos/interface/HGCalCalibrationParameterHostCollection.h:
    • Added: 5bdf753
    • Modified: 394aade
    • Deleted: 2ebc1fb
  • RecoLocalCalo/HGCalRecAlgos/plugins/alpaka/TestHGCalRecHitESProducers.cc:
    • Added: 5bdf753
    • Modified: 394aade
    • Deleted: 2ebc1fb
  • CondFormats/DataRecord/interface/HGCalMappingModuleIndexerRcd.h:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • CondFormats/DataRecord/interface/HGCalMappingCellRcd.h:
    • Added: 0baf56d
    • Deleted: 7975bbd
  • CondFormats/DataRecord/src/HGCalElectronicsMappingRcd.cc:
    • Deleted: 0baf56d
    • Added: 7975bbd
  • CondFormats/HGCalObjects/src/HGCalMappingParameterHostCollection.cc:
    • Added: b110814
    • Deleted: f381b45
  • Geometry/HGCalMapping/data/modulelocator_test.txt:
    • Added: f02b407
    • Deleted: 5c5ff69
  • RecoLocalCalo/HGCalRecAlgos/interface/HGCalCalibrationParameterSoA.h:
    • Added: 5bdf753
    • Modified: 394aade
    • Deleted: 2ebc1fb
  • DataFormats/HGCalDigi/interface/HGCalECONDInfoSoA.h:
    • Added: 9756f10
    • Modified: 415e9ed, 49f9cfb
    • Deleted: 762c834
  • Geometry/HGCalMapping/data/channels_sipmontile.hgcal.txt:
    • Added: f02b407
    • Deleted: 5c5ff69
  • CondFormats/DataRecord/src/HGCalMappingCellRcd.cc:
    • Added: 0baf56d
    • Deleted: 7975bbd
  • CondFormats/DataRecord/src/HGCalMappingCellIndexerRcd.cc:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • CondFormats/DataRecord/src/HGCalMappingModuleRcd.cc:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • Geometry/HGCalMapping/data/WaferCellMapTraces.txt:
    • Added: f02b407
    • Deleted: 5c5ff69
  • DataFormats/HGCalDigi/interface/alpaka/HGCalECONDInfoDevice.h:
    • Added: 9756f10
    • Deleted: 762c834
  • SimCalorimetry/HGCalSimAlgos/interface/HGCalRawDataDefinitions.h:
    • Added: 4fae6f8
    • Deleted: 2a083a9
  • RecoLocalCalo/HGCalRecAlgos/src/alpaka/HGCalCalibrationParameterHostCollection.cc:
    • Added: 4fad636
    • Deleted: b59f356
  • CondFormats/DataRecord/interface/HGCalMappingCellIndexerRcd.h:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • RecoLocalCalo/HGCalRecAlgos/plugins/alpaka/HGCalRecHitProducer.cc:
    • Added: e109222
    • Modified: 7975bbd, 02155c7
    • Deleted: 2247d87
  • DataFormats/HGCalDigi/interface/HGCalECONDInfoHost.h:
    • Added: 9756f10
    • Modified: 415e9ed
    • Deleted: 762c834
  • Geometry/HGCalMapping/plugins/HGCalMappingIndexESSource.cc:
    • Added: f02b407
    • Modified: 5c5ff69, a21eec7, 0baf56d, f73ce1d, 3de5708, 3907071
    • Deleted: 577e6d6
  • RecoLocalCalo/HGCalRecAlgos/src/alpaka/ES_HGCalCalibrationParameter.cc:
    • Added: 5bdf753
    • Deleted: 4fad636
  • CondFormats/HGCalObjects/src/alpaka/HGCalCalibrationParameterDevice.cc:
    • Added: 2ebc1fb
    • Deleted: 5d6cfa2
  • RecoLocalCalo/HGCalRecAlgos/interface/alpaka/HGCalCalibrationParameterDeviceCollection.h:
    • Added: 5bdf753
    • Deleted: 2ebc1fb
  • CondFormats/DataRecord/interface/HGCalMappingModuleRcd.h:
    • Added: f02b407, 0baf56d
    • Modified: f326885
    • Deleted: a21eec7, 7975bbd
  • CondFormats/HGCalObjects/src/alpaka/HGCalMappingParameterHostCollection.cc:
    • Added: 4fad636
    • Deleted: b59f356

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DQMServices/Core/interface/DQMStore.h modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932
  • File DQMServices/Core/interface/MonitorElement.h modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932
  • File DQMServices/Core/src/MonitorElement.cc modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932
  • File DataFormats/Histograms/interface/MEtoEDMFormat.h modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932
  • File DataFormats/Histograms/interface/MonitorElementCollection.h modified in PR(s): Implement TH2Poly in DQM Services for HGCal DQM #41932

[cmsbuild]

New categories assigned: hgcal-dpg

@rovere,@cseez,@pfs,@felicepantaleo you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

A new Pull Request was created by @IzaakWN for master.

It involves the following packages:

• CondFormats/DataRecord (alca, db)
• CondFormats/HGCalObjects (alca, upgrade, db)
• DQMServices/Core (dqm)
• DataFormats/HGCalDigi (upgrade, simulation)
• DataFormats/HGCalRecHit (****)
• DataFormats/Histograms (core, dqm)
• EventFilter/HGCalRawToDigi (reconstruction, upgrade)
• Geometry/HGCalMapping (geometry, upgrade)
• RecoLocalCalo/HGCalRecAlgos (reconstruction, upgrade)
• SimCalorimetry/HGCalSimAlgos (upgrade, simulation)
• SimCalorimetry/HGCalSimProducers (upgrade, simulation)

The following packages do not have a category, yet:

DataFormats/HGCalRecHit
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@Dr15Jones, @antoniovagnerini, @bsunanda, @civanch, @cmsbuild, @consuegs, @cseez, @felicepantaleo, @francescobrivio, @jfernan2, @makortel, @mandrenguyen, @mdhildreth, @nothingface0, @perrotta, @pfs, @rovere, @rvenditti, @saumyaphor4252, @smuzaffar, @srimanob, @subirsarkar, @syuvivida, @tjavaid can you please review it and eventually sign? Thanks.
@JanChyczynski, @Martin-Grunewald, @PonIlya, @ReyerBand, @apsallid, @argiro, @barvic, @bsunanda, @cseez, @edjtscott, @fabiocos, @felicepantaleo, @hatakeyamak, @lgray, @martinamalberti, @missirol, @mmusich, @pfs, @rchatter, @rovere, @rsreds, @sameasy, @seemasharmafnal, @sethzenz, @thomreis, @tocheng, @vandreev11, @wang0jin, @youyingli, @yuanchao this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[perrotta]

+1

• No relevant updates in the AlCa/DB sections since previous signature in HGCal raw data handling: Unpacker, Geometry, Local Reco #45665 (comment)
• Same comment can be repeated here: AlCa/DB part of this PR looks fine, taking into account that configuration parameters are not (yet!) taken from the DB

[Moanwar]

+Upgrade

[pfs]

hi can we get the remaining approvals from @cms-sw/geometry-l2 and @cms-sw/simulation-l2 and @cms-sw/orp-l2 please?

[bsunanda]

+geometry

[pfs]

@civanch could we get your approval please? (you had already signed early on I think)

[civanch]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @sextonkennedy, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

[mandrenguyen]

+1

[pfs]

Many thanks for all the help to get this in release!