Improved method for EB digi simulation #40855

jhakala · 2023-02-23T00:40:50Z

PR description:

This PR implements a new method for simulating EB digis which should be more accurate and thus will improve the quality of the MC for timing studies. The method is based on simulating the response of the APD+readout electronics in a way that takes into account the depth of PCaloHit energy deposits, so pulse shape fluctuations (sometimes referred to as the "Landau term") are modeled. In addition to improving the EB digis, a new collection is added in the EcalTimeDigis which stores a finely-sampled analog waveform, meant to emulate the output of the preamplifier prior to its sampling by the ADC.

In addition to implementing the method, I added a new CondFormat object, such that the "component shapes" used in the method can be accessed via the EventSetup from the CondDB.

The output will change in the following ways:

depending on a cfg parameter, a new EBUnsuppressedDigi collection will be added in the DIGI step. (default is that it is omitted.)
depending on a cfg parameter, a new array is added to the EcalTimeDigis which has the finely-sampled simulated waveform (it has 10 BX with 25 double values per BX [1 ns sampling] for each EB crystal.) (default is that this is omitted too)

This doesn't depend on any other PRs or externals. Here are some slides on the method: https://indico.cern.ch/event/1187720/#5-status-of-simulation-of-timi
Those slides are slightly out of date but I can provide more documentation if desired.

PR validation:

I've tested this with my own configuration files based on the ecal_devel workflows, in particular runTheMatrix workflow 24234.61, but with customizations added to enable the new functionality. I've looked at the new digis and waveforms from the method and they look as expected (slightly sharper waveforms with more pronounced fluctuations in pulse shapes.) I ran some of the recommended scram b runtests but it looked like they were failing due to unrelated reasons.

cmsbuild · 2023-02-23T00:50:05Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40855/34312

This PR adds an extra 100KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/StandardSequences/python/Eras.py modified in PR(s): Add run3 tracking low pu era #33532

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40855/34312/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40855/34312/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2023-02-23T15:43:52Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40855/34327

This PR adds an extra 100KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/StandardSequences/python/Eras.py modified in PR(s): Add run3 tracking low pu era #33532

cmsbuild · 2023-02-23T15:44:21Z

A new Pull Request was created by @jhakala (John Hakala) for master.

It involves the following packages:

CondCore/EcalPlugins (db)
CondFormats/DataRecord (db, alca)
CondFormats/EcalObjects (db, alca)
Configuration/Eras (operations)
Configuration/StandardSequences (operations)
DataFormats/EcalDigi (simulation)
SimCalorimetry/Configuration (simulation)
SimCalorimetry/EcalSimAlgos (simulation)
SimCalorimetry/EcalSimProducers (simulation)
SimGeneral/MixingModule (simulation)

@perrotta, @malbouis, @rappoccio, @yuanchao, @civanch, @mdhildreth, @tvami, @cmsbuild, @saumyaphor4252, @ggovi, @francescobrivio, @ChrisMisan, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@VourMa, @felicepantaleo, @argiro, @Martin-Grunewald, @thomreis, @slomeo, @makortel, @JanFSchulte, @dgulhan, @missirol, @seemasharmafnal, @simonepigazzini, @GiacomoSguazzoni, @tocheng, @VinInn, @rovere, @mmusich, @mtosi, @fabiocos, @rchatter, @AnnikaStein, @sameasy this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

civanch · 2023-02-23T18:09:25Z

please test

cmsbuild · 2023-02-23T18:36:44Z

-1

Failed Tests: HeaderConsistency ClangBuild
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-40d5f3/30850/summary.html
COMMIT: e0b676c
CMSSW: CMSSW_13_1_X_2023-02-23-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40855/30850/install.sh to create a dev area with all the needed externals and cmssw changes.

Clang Build

I found compilation warning while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

See details on the summary page.

cmsbuild · 2023-02-23T19:57:57Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40855/34332

This PR adds an extra 36KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/StandardSequences/python/Eras.py modified in PR(s): Add run3 tracking low pu era #33532

perrotta · 2023-04-01T08:58:31Z

+1

perrotta · 2023-04-01T09:00:00Z

merge

The last update was just an innocuous rebase: keep all previously given signatures for good

srimanob · 2023-04-12T11:24:26Z

Hi @jhakala
I am wondering

if you will provide the follow up PR on the workflow?
What is the plan for 13_1 HLT production? Will this method becomes default, or it will be alternative (i.e. requests specific samples with it)

FYI @AdrianoDee @SohamBhattacharya

srimanob · 2023-06-06T10:30:08Z

+Upgrade

VinInn · 2023-06-06T11:20:08Z

DataFormats/EcalDigi/interface/EcalTimeDigi.h

  void setSample(unsigned int i, const float sam) { data_[i] = sam; }
  void setSampleOfInterest(int i) { sampleOfInterest_ = i; }

  /// Gets the BX==0 sample. If =-1 then it means that only OOT hits are present
  int sampleOfInterest() const { return sampleOfInterest_; }
+  std::vector<float> waveform() const { return waveform_; }
+
+  static const unsigned int WAVEFORMSAMPLES = 250;


why all capital?
constexpr is preferred.

VinInn · 2023-06-06T11:21:01Z

DataFormats/EcalDigi/src/EcalTimeDigi.cc

@@ -2,11 +2,12 @@

 namespace {
  constexpr unsigned int MAXSAMPLES = 10;


VinInn · 2023-06-06T11:22:06Z

SimCalorimetry/EcalSimAlgos/interface/EcalTimeMapDigitizer.h


 class CaloSubdetectorGeometry;

 class EcalTimeMapDigitizer {
 public:
  struct time_average {
-    static const unsigned short time_average_capacity = 10;
+    static const unsigned short time_average_capacity = 10;  // this corresponds to the number of BX


constexpr

VinInn · 2023-06-06T11:23:12Z

SimCalorimetry/EcalSimAlgos/src/ComponentShape.cc

+  else {  // fill with dummy values, since this code is only reached before the actual digi simulation
+    m_thresh = 0.00013;
+    time_interval = 1.0;
+    aVec.reserve(500);


aVec.resize(500,0.);

VinInn · 2023-06-06T11:25:46Z

SimCalorimetry/EcalSimAlgos/src/EcalShapeBase.cc

-  for (unsigned int i(0); i != shapeArray.size(); ++i) {
-    shapeArray[i] = shapeArray[i] / maxelt;
+  if (normalize) {
+    for (unsigned int i(0); i != shapeArray.size(); ++i) {


for (auto & s: shapeArray) s/=maxelt;

VinInn · 2023-06-06T11:30:34Z

SimCalorimetry/EcalSimAlgos/src/EcalTimeMapDigitizer.cc

+        for (unsigned int bin(0); bin != result.waveform_capacity; ++bin) {
+          if (ComponentShapeCollection::toDepthBin((*it).depth()) <= ComponentShapeCollection::maxDepthBin()) {
+            result.waveform[bin] +=
+                (*(shapes()->at((*it).depth())))(binTime - jitter - 25 * (bunchCrossing - m_minBunch)) * (*it).energy();


use operator[] not at for production

VinInn · 2023-06-06T11:33:51Z

SimCalorimetry/EcalSimAlgos/interface/ComponentSimParameterMap.h

+
+private:
+  bool m_addToBarrel;
+  bool m_separateDigi;


VinInn · 2023-06-06T11:34:31Z

SimCalorimetry/EcalSimAlgos/interface/ComponentShapeCollection.h

+  void fillCollection(bool useDBShape);
+  void fillCollection(edm::ConsumesCollector iC);
+
+  bool m_useDBShape;


VinInn · 2023-06-06T11:35:25Z

SimCalorimetry/EcalSimAlgos/interface/ComponentShape.h

+
+private:
+  edm::ESGetToken<EcalSimComponentShape, EcalSimComponentShapeRcd> espsToken_;
+  int shapeIndex_;


VinInn · 2023-06-06T11:36:59Z

CondCore/EcalPlugins/plugins/EcalSimComponentShape_PayloadInspector.cc

+        double time = (*payload).time_interval;
+        std::vector<std::vector<float> > EBshapes = (*payload).barrel_shapes;
+        char nameBuffer[50];
+        for (auto EBshape : EBshapes) {


for (auto const &EBshape : EBshapes)

VinInn · 2023-06-06T11:39:46Z

CalibCalorimetry/EcalTrivialCondModules/src/EcalTrivialConditionRetriever.cc

+  // --- get the shapes from the user provided txt files
+  if (!EBSimComponentShapeFiles_.empty()) {
+    int iShape(0);
+    for (auto filename : EBSimComponentShapeFiles_) {


auto const & filename

VinInn · 2023-06-06T11:40:13Z

CalibCalorimetry/EcalTrivialCondModules/src/EcalTrivialConditionRetriever.cc

+    for (auto filename : EBSimComponentShapeFiles_) {
+      std::ifstream shapeEBFile;
+      EBshapes.emplace_back();
+      shapeEBFile.open(EBSimComponentShapeFiles_[iShape].c_str());


shapeEBFile.open(filename.c_str());

VinInn · 2023-06-06T11:43:43Z

CalibCalorimetry/EcalTrivialCondModules/src/EcalTrivialConditionRetriever.cc

+      shapeEBFile.open(EBSimComponentShapeFiles_[iShape].c_str());
+      float ww;
+      while (shapeEBFile >> ww)
+        EBshapes[iShape].push_back(ww);


EBshapes.back().push_back(ww);

or auto & EBshape = EBshapes.back() outside the while-loop and then
EBshape.push_back(ww); here

VinInn · 2023-06-06T12:04:41Z

SimCalorimetry/EcalSimAlgos/src/EcalTimeMapDigitizer.cc

-  GlobalPoint layerPos =
-      (cellGeometry)->getPosition(double(layer) + 0.5);  //depth in mm in the middle of the layer position
+  GlobalPoint layerPos = (cellGeometry)->getPosition();
+  //(cellGeometry)->getPosition(double(layer) + 0.5);  //depth in mm in the middle of the layer position // JCH : I am not sure this is doing what it's supposed to, probably unimplemented since CaloCellGeometry returns the same value regardless of this double
  return layerPos.mag() * cm / c_light;


this looks like a constant: maybe precompute and store it in cellGeometry?

VinInn · 2023-06-06T12:09:04Z

SimCalorimetry/EcalSimAlgos/interface/EcalTimeMapDigitizer.h

@@ -13,25 +13,32 @@
 #include "DataFormats/EcalDetId/interface/EcalSubdetector.h"
 #include "CalibFormats/CaloObjects/interface/CaloTSamplesBase.h"
 #include "DataFormats/EcalDigi/interface/EcalDigiCollections.h"
+#include "SimCalorimetry/EcalSimAlgos/interface/ComponentShapeCollection.h"

 class CaloSubdetectorGeometry;

 class EcalTimeMapDigitizer {
 public:
  struct time_average {


TimeAverage according to CMS naming rules

VinInn · 2023-06-06T12:11:38Z

SimCalorimetry/EcalSimAlgos/interface/EcalTimeMapDigitizer.h

@@ -66,14 +76,16 @@ class EcalTimeMapDigitizer {

  typedef std::vector<unsigned int> VecInd;

-  explicit EcalTimeMapDigitizer(EcalSubdetector myDet);
+  explicit EcalTimeMapDigitizer(EcalSubdetector myDet, ComponentShapeCollection* componentShapes);

  virtual ~EcalTimeMapDigitizer();


why virtual?

VinInn · 2023-06-06T12:13:33Z

SimCalorimetry/EcalSimAlgos/src/ComponentShape.cc

+  }
+}
+
+double ComponentShape::timeToRise() const {


static constexpr
possibly inline

cmsbuild added this to the CMSSW_13_1_X milestone Feb 23, 2023

cmsbuild added simulation-pending alca-pending db-pending operations-pending pending-signatures tests-pending orp-pending code-checks-pending labels Feb 23, 2023

cmsbuild added code-checks-rejected code-checks-pending and removed code-checks-pending code-checks-rejected labels Feb 23, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 23, 2023

cmsbuild added tests-started and removed tests-pending labels Feb 23, 2023

cmsbuild added tests-rejected tests-pending code-checks-pending and removed tests-started tests-rejected code-checks-approved labels Feb 23, 2023

cmsbuild removed the code-checks-pending label Feb 23, 2023

cmsbuild added the tests-approved label Mar 31, 2023

cmsbuild added operations-approved orp-approved and removed operations-pending orp-pending labels Apr 1, 2023

cmsbuild merged commit dd08066 into cms-sw:master Apr 1, 2023

smuzaffar mentioned this pull request Apr 7, 2023

[DBG] fix compilation error for dbg IBs #41302

Merged

cmsbuild added upgrade-approved and removed upgrade-pending labels Jun 6, 2023

VinInn reviewed Jun 6, 2023

View reviewed changes

DataFormats/EcalDigi/src/EcalTimeDigi.cc

@@ -2,11 +2,12 @@

namespace {

constexpr unsigned int MAXSAMPLES = 10;

Copy link

Contributor

VinInn Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

VinInn reviewed Jun 6, 2023

View reviewed changes

SimCalorimetry/EcalSimAlgos/interface/ComponentSimParameterMap.h

private:

bool m_addToBarrel;

bool m_separateDigi;

Copy link

Contributor

VinInn Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const ?

VinInn reviewed Jun 6, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improved method for EB digi simulation #40855

Improved method for EB digi simulation #40855

jhakala commented Feb 23, 2023 •

edited

Loading

cmsbuild commented Feb 23, 2023

cmsbuild commented Feb 23, 2023

cmsbuild commented Feb 23, 2023

civanch commented Feb 23, 2023

cmsbuild commented Feb 23, 2023

cmsbuild commented Feb 23, 2023

perrotta commented Apr 1, 2023

perrotta commented Apr 1, 2023

srimanob commented Apr 12, 2023

srimanob commented Jun 6, 2023

VinInn Jun 6, 2023 •

edited

Loading

VinInn Jun 6, 2023

VinInn Jun 6, 2023

VinInn Jun 6, 2023

VinInn Jun 6, 2023 •

edited

Loading

VinInn Jun 6, 2023

VinInn Jun 6, 2023

VinInn Jun 6, 2023

VinInn Jun 6, 2023

VinInn Jun 6, 2023

VinInn Jun 6, 2023

VinInn Jun 6, 2023

VinInn Jun 6, 2023

VinInn Jun 6, 2023 •

edited

Loading

VinInn Jun 6, 2023

VinInn Jun 6, 2023 •

edited

Loading

VinInn Jun 6, 2023

		@@ -2,11 +2,12 @@

		namespace {
		constexpr unsigned int MAXSAMPLES = 10;

Improved method for EB digi simulation #40855

Improved method for EB digi simulation #40855

Conversation

jhakala commented Feb 23, 2023 • edited Loading

PR description:

PR validation:

cmsbuild commented Feb 23, 2023

cmsbuild commented Feb 23, 2023

cmsbuild commented Feb 23, 2023

civanch commented Feb 23, 2023

cmsbuild commented Feb 23, 2023

Clang Build

cmsbuild commented Feb 23, 2023

perrotta commented Apr 1, 2023

perrotta commented Apr 1, 2023

srimanob commented Apr 12, 2023

srimanob commented Jun 6, 2023

VinInn Jun 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VinInn Jun 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VinInn Jun 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VinInn Jun 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jhakala commented Feb 23, 2023 •

edited

Loading

VinInn Jun 6, 2023 •

edited

Loading

VinInn Jun 6, 2023 •

edited

Loading

VinInn Jun 6, 2023 •

edited

Loading

VinInn Jun 6, 2023 •

edited

Loading