Access edm::event from GeneratorInterface #28636

wouf · 2019-12-17T06:45:56Z

Three HI generators has an issue with edm::event access in embedding mode. It trying to read empty edm::event using BaseHadroniser::getEDMEvent function:
Pyquen:
https://github.com/cms-sw/cmssw/blob/master/GeneratorInterface/PyquenInterface/src/PyquenHadronizer.cc#L125
Hydjet2:
https://github.com/cms-sw/cmssw/blob/master/GeneratorInterface/Hydjet2Interface/src/Hydjet2Hadronizer.cc#L491
Hydjet:
https://github.com/cms-sw/cmssw/blob/master/GeneratorInterface/HydjetInterface/src/HydjetHadronizer.cc#L195

To correct it it is needed call BaseHadroniser::setEDMEvent somewhere, but for now I can't to figure out how to access edm::event (from HiMixGEN) in GeneratorFilter before event generation.

cmsbuild · 2019-12-17T06:46:19Z

A new Issue was created by @wouf .

@davidlange6, @Dr15Jones, @smuzaffar, @fabiocos, @kpedro88 can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

Dr15Jones · 2019-12-17T14:39:17Z

assign generator

Dr15Jones · 2019-12-17T19:27:58Z

assign generators

cmsbuild · 2019-12-17T19:28:20Z

New categories assigned: generators

@alberto-sanchez,@SiewYan,@qliphy,@efeyazgan,@mkirsano,@agrohsje you have been requested to review this Pull request/Issue and eventually sign? Thanks

wouf · 2020-01-06T17:57:56Z

Any ideas on this?

qliphy · 2020-01-14T02:11:19Z

@wouf Would you please give more details about this issue? What is embedding mode and why this was not used before? and do you have a recipe to reproduce the error?

wouf · 2020-01-14T08:45:48Z

@qliphy The embedding mode on generator level, is needed to mix heavy ion events: to do it we have to care about impact parameter and event plane angle - it have to be the same for both events. To mix two MinBias samples we have to generate the second one using the first as underlying. For HYDJET, on each event (before generating) we have to read underlying sample event like
https://github.com/cms-sw/cmssw/blob/master/GeneratorInterface/HydjetInterface/src/HydjetHadronizer.cc#L195
to get impact parameter and event plane angle L201, L202 used as input in HYDJET.
Before it was not used with HYDJET, because the only last version (from 28 November 2019) support this feature.

To reproduce it with HYDJET one have to use something like
cmsDriver.py Hydjet_Quenched_MinBias_5020GeV_cfi --conditions auto:phase1_2018_realistic --scenario HeavyIons -n 1 --era Run2_2018 --eventcontent RAWSIM --relval 2000,1 -s GEN,SIM --datatier GEN-SIM --beamspot Realistic25ns13TeVEarly2018Collision --geometry DB:Extended --pileup HiMixGEN --pileup_input file:Hydjet_Quenched_B12_5020GeV_cfi_GEN_SIM.root --fileout test_GEN_SIM_PU.root --no_exec
And set embeddingMode = cms.bool(True). Then just print the size of event е just after L195

Or one can just look through BaseHadroniser code and realize that we have just empty edm::event, before we care about setEDMEvent.

qliphy · 2020-01-15T09:58:51Z

Thanks! Can you point us to the input file?

wouf · 2020-01-15T12:04:50Z

@qliphy It may be any generated in HepMC format with hi section. I created it with:
cmsDriver.py Hydjet_Quenched_B12_5020GeV_cfi --conditions auto:phase1_2018_realistic --scenario HeavyIons -n 1 --era Run2_2018 --eventcontent RAWSIM --relval 2000,1 -s GEN,SIM --datatier GEN-SIM --beamspot Realistic25ns13TeVEarly2018Collision --geometry DB:Extended

qliphy · 2020-01-20T09:53:07Z

Thanks @wouf I tried to reproduce your error:

After producing an input file, and then proceed to mix the events, I got
(1) cmsRun Hydjet_Quenched_MinBias_5020GeV_cfi_GEN_SIM_PU.py
----- Begin Fatal Exception 20-Jan-2020 09:08:41 CET-----------------------
An exception of category 'Configuration' occurred while
[0] Constructing the EventProcessor
[1] Constructing module: class=HydjetGeneratorFilter label='generator'
Exception Message:
MissingParameter: Parameter 'backgroundLabel' not found.

Then I added in the python file as below:
backgroundLabel = cms.InputTag("generator","unsmeared")

(2) then I got
....
Impact parameter larger than three nuclear radius!
Begin processing the 1st record. Run 1, Event 1, LumiSection 1 on stream 0 at 20-Jan-2020 09:13:34.079 CET
----- Begin Fatal Exception 20-Jan-2020 09:13:34 CET-----------------------
An exception of category 'GetByLabelWithoutRegistration' occurred while
[0] Processing Event run: 1 lumi: 1 event: 1 stream: 0
[1] Running path 'simulation_step'
[2] Calling method for module HydjetGeneratorFilter/'generator'
Exception Message:
::getByLabel without corresponding call to consumes or mayConsumes for this module.
type: edm::HepMCProduct
module label: generator
product instance name: 'unsmeared'
process name: ''

Have you already added consumes syntax to replace getByLabel by getByToken?
https://github.com/cms-sw/cmssw/blob/master/GeneratorInterface/HydjetInterface/src/HydjetHadronizer.cc#L197

wouf · 2020-01-20T16:33:44Z

I added one to #28764

qliphy · 2020-01-21T07:09:50Z

Thanks @wouf
However, when I tried as below, it doesn't compile:

cmsrel CMSSW_11_1_0_pre1
git cms-addpkg GeneratorInterface/HydjetInterface
git cms-merge-topic wouf:from-CMSSW_11_0_0_pre7

/tmp/qili/XXX/CMSSW_11_1_0_pre1/src/GeneratorInterface/HydjetInterface/src/HydjetHadronizer.cc: In constructor 'gen::HydjetHadronizer::HydjetHadronizer(const edm::ParameterSet&)':
/tmp/qili/XXX/CMSSW_11_1_0_pre1/src/GeneratorInterface/HydjetInterface/src/HydjetHadronizer.cc:106:12: error: 'mayConsume' was not declared in this scope
src_ = mayConsume(iConfig.getUntrackedParameteredm::InputTag( "backgroundLabel", edm::InputTag("generator","unsmeared") ));
^~~~~~~~~~
/tmp/qili/XXX/CMSSW_11_1_0_pre1/src/GeneratorInterface/HydjetInterface/src/HydjetHadronizer.cc:106:35: error: expected primary-expression before '>' token
src_ = mayConsume(iConfig.getUntrackedParameteredm::InputTag( "backgroundLabel", edm::InputTag("generator","unsmeared") ));
^
/tmp/qili/XXX/CMSSW_11_1_0_pre1/src/GeneratorInterface/HydjetInterface/src/HydjetHadronizer.cc:106:37: error: 'iConfig' was not declared in this scope
src_ = mayConsume(iConfig.getUntrackedParameteredm::InputTag( "backgroundLabel", edm::InputTag("generator","unsmeared") ));

wouf · 2020-01-21T08:06:47Z

@qliphy I'm sorry, I committed wrong project. Now fixed.

qliphy · 2020-01-21T08:20:34Z

Thanks, now I get other error messages as below:

/cvmfs/cms.cern.ch/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: tmp/slc7_amd64_gcc820/src/GeneratorInterface/HydjetInterface/src/GeneratorInterfaceHydjetInterface/HydjetHadronizer.cc.o: in function gen::HydjetHadronizer::add_heavy_ion_rec(HepMC::GenEvent*)': HydjetHadronizer.cc:(.text+0x95): undefined reference to hyfpar_'
/cvmfs/cms.cern.ch/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: HydjetHadronizer.cc:(.text+0x15f): undefined reference to hyjpar_' /cvmfs/cms.cern.ch/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: tmp/slc7_amd64_gcc820/src/GeneratorInterface/HydjetInterface/src/GeneratorInterfaceHydjetInterface/HydjetHadronizer.cc.o: in function gen::HydjetHadronizer::call_hyinit(double, double, int, double, double, double, int)':
HydjetHadronizer.cc:(.text+0x231): undefined reference to hyinit_' /cvmfs/cms.cern.ch/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: tmp/slc7_amd64_gcc820/src/GeneratorInterface/HydjetInterface/src/GeneratorInterfaceHydjetInterface/HydjetHadronizer.cc.o: in function gen::HydjetHadronizer::build_hyjet_vertex(int, int)':
HydjetHadronizer.cc:(.text+0x373): undefined reference to `hyjets_'

wouf · 2020-01-21T08:22:19Z

@qliphy Are You tested it with external hydjet?

wouf · 2020-01-21T09:11:50Z

The version below 1.9.1 does not support "embedding mode"

wouf · 2020-01-23T07:53:11Z

@qliphy were you able to reproduce the issue?

qliphy · 2020-01-23T11:58:07Z

Not yet, Still trying.

wouf · 2020-01-23T12:30:54Z

@qliphy I can provide my sequence for local testing if You need

qliphy · 2020-01-23T12:49:05Z

@wouf that would be useful indeed.

wouf · 2020-01-23T19:57:00Z

@qliphy sorry for the late replay

On lxplus7 (in the test folder) please do:
export SCRAM_ARCH=slc7_amd64_gcc700
source /cvmfs/sft.cern.ch/lcg/contrib/gcc/7.2.0/x86_64-centos7-gcc7-opt/setup.sh

Then run the script:
git clone https://github.com/wouf/scripts.git
sh scripts/TestHydjet.sh

Then get hydjet 1.9.1 interface:
scram project CMSSW CMSSW_11_0_0_pre7
cd CMSSW_11_0_0_pre7/src
cmsenv
git cms-addpkg GeneratorInterface/HydjetInterface
git cms-merge-topic wouf:from-CMSSW_11_0_0_pre7

Then setup hydjet libs:
scram setup ../../testFolder/slc7_amd64_gcc700/external/hydjet-toolfile/1.0/etc/scram.d/hydjet.xml
scram b

To check if it is really version 1.9.1, please run the test:
cmsRun GeneratorInterface/HydjetInterface/test/testHydjet.py

qliphy · 2020-01-27T08:18:29Z

@wouf I confirm I can reproduce the error.

(1) In Hydjet_Quenched_MinBias_5020GeV_cfi_GEN_SIM_PU.py, there is
# filter all path with the production filter sequence
for path in process.paths:
getattr(process,path).insert(0, process.generator)
where process.generator runs first:
process.generator = cms.EDFilter("HydjetGeneratorFilter", ....

(2) The relevant pgen path is defined as here:
process.generation_step = cms.Path(process.pgen)
pgen = cms.Sequence(cms.SequencePlaceholder("randomEngineStateProducer")+cms.SequencePlaceholder("mix")+VertexSmearing+GenSmeared+genParticles+hiGenJets)

(3) and thus the mixing module runs after generator path
process.mix.input.fileNames = cms.untracked.vstring(['file:Hydjet_Quenched_B12_5020GeV_cfi_GEN_SIM.root'])

(4) All these should explain why currently getEDMEvent gets empty, and probably setEDMEvent should be called after/in the mixing module, e.g.:
https://github.com/cms-sw/cmssw/blob/master/SimGeneral/MixingModule/plugins/MixingModule.cc#L169-L175
and one further adjusts a bit above sequence order.

Hope software experts @davidlange6, @Dr15Jones, @smuzaffar, @fabiocos, @kpedro88 @mkirsano @alberto-sanchez @SiewYan can have a look and give more suggestion or help.

Dr15Jones · 2020-01-27T14:00:04Z

@makortel FYI

makortel · 2020-01-27T16:13:24Z

I'm pretty sure I'm missing something so please correct me

The relevant module is HydjetGeneratorFilter defined here

cmssw/GeneratorInterface/HydjetInterface/interface/HydjetGeneratorFilter.h

Line 9 in 65737ec

typedef edm::GeneratorFilter<gen::HydjetHadronizer, gen::ExternalDecayDriver> HydjetGeneratorFilter;

The edm::GeneratorFilter<...> calls the setEDMEvent() here

cmssw/GeneratorInterface/Core/interface/GeneratorFilter.h

Lines 140 to 142 in 65737ec

    
           hadronizer_.setEDMEvent(ev); 
        
           if (!hadronizer_.generatePartonsAndHadronize())

It follows that setEDMEvent() should not be called from the MixingModule (in fact can not be called)

From the discussion above it is not clear to me what exactly is the problem. Is it that getEDMEvent() returns "garbage" (that would imply setEDMEvent() call being missing)? Or that the edm::Event itself gets propagated fine, but is missing some product (that would imply ordering problem in the Sequence)? Or something else?

wouf · 2020-04-22T20:47:00Z

Thank You for the explanation! Тhat doesn't sound comforting...

makortel · 2020-04-23T01:16:39Z

Тhat doesn't sound comforting...

I agree. I don't have good ideas for which way to proceed.

Is there any hope for getting Hydjet thread-friendly (like Pythia8 alone is, assuming Hydjet is not yet)?

How much time does the GEN step take compared to SIM step?

wouf · 2020-04-23T08:27:23Z

Тhat doesn't sound comforting...

I agree. I don't have good ideas for which way to proceed.

Is there any hope for getting Hydjet thread-friendly (like Pythia8 alone is, assuming Hydjet is not yet)?

The current Hydjet family 1.x and 2.x uses Pyquen 1.x as a part. Pyquen family 1.x is based on Pythia 6 (PYQUEN mean PYthia QUENched). As far as I know no way to make Pythia 6 thread-friendly (at least due to fortran 77 specification with SAVE variables, right?).

How much time does the GEN step take compared to SIM step?

On 10 events GEN workflow spend 0m53s, GEN-SIM - 60m50s.
But at this moment we have a lot of warnings from GEANT (which increases the simulation time):

%MSG-w SimG4CoreGenerator: in event :  OscarMTProducer:g4SimHits  23-Apr-2020 09:05:14 CEST Run: 1 Event: 10
10 a particle  pdgid= 2101 has status=2 but has no decay vertex, so will be fully tracked by Geant4
%MSG

I guess it's related to HEPEVT COMMON-block size, which now is 4000, what may be not enough for HI. For now I'm trying to manage it, but with no success yet: changing HEPEVT_Wrapper::s_max_number_entries=10000 only for Hydjet does no helps. Possible we need also to change NMXHEP in Pythia6 and SimDataFormats.

wouf · 2020-04-30T08:32:49Z

Here is another thought based on a discussion with @Dr15Jones.

The current workflow looks like (a bit simplified)
                          MinBias                           MinBias
                             |                                 |
                             v                                 v
EmptySource -> generator -> mix -> genParticles etc -> SIM -> mix -> Digi2Raw
with first mix mixing the HepMCProduct and the second mix mixing the digis (and SIM simulates only the signal part). Reading the same MinBias files/events twice does have a cost on the computing infrastructure.

The second file read could be avoided by mixing both the HepMCProduct and the digis in the premixing way
             MinBias
                |
                v
EmptySource -> mix -> generator -> SIM -> mix -> mixOverlay -> genParticles etc -> Digi2Raw
i.e. mix mix the HepMCProduct and digis by ignoring the signal, and for digis producing similar output as premixing does for the pileup library. The generator reads the CrossingFrame<HepMCProduct> as discussed before, and SIM would continue to simulate the signal part. The second mix would then produce the digis of the signal event, and mixOverlay would overlay the MinBias HepMCProduct+digis over the signal HepMCProduct+digis (similar to the pileup overlay in premixing). After that the genParticles etc and Digi2Raw would be run.

Dear @makortel and @Dr15Jones , for HepMCProduct it works perfect with skipSignal parameter, but for HepMCProduct+digis I have to use theDigitizers in HiMixEmbGEN_cff.py
It looks like we need to skip signal for digitizer as well:

Exception Message:
Unrunnable schedule
Module run order problem found: 
generatorSmeared after VtxSmeared [path generation_step], VtxSmeared after randomEngineStateProducer [path generation_step], randomEngineStateProducer after generator [path generation_step], generator after mix [path generation_step], mix consumes generatorSmeared

Could You please to help with it?

makortel · 2020-04-30T14:10:55Z

@wouf Is your question about the second occurrence of mix in

             MinBias
                |
                v
EmptySource -> mix -> generator -> SIM -> mix -> mixOverlay -> genParticles etc -> Digi2Raw

?

makortel · 2020-04-30T14:20:21Z

How much time does the GEN step take compared to SIM step?

On 10 events GEN workflow spend 0m53s, GEN-SIM - 60m50s.

With 60-to-1 ratio naively we could be able to feed 60-wise concurrency in SIM with 1 instance of GEN (ignoring edge effects etc). At least for short-to-mid term I would imagine that to be acceptable (assuming there is no filtering of events between GEN and SIM).

wouf · 2020-04-30T14:47:29Z

@wouf Is your question about the second occurrence of mix in

             MinBias
                |
                v
EmptySource -> mix -> generator -> SIM -> mix -> mixOverlay -> genParticles etc -> Digi2Raw

?

No, about the first one. As far as I understand this strategy, first mix is for background, second - for signal without background (to avoid file reading). Then we just overlay it.

wouf · 2020-04-30T14:59:13Z

How much time does the GEN step take compared to SIM step?

On 10 events GEN workflow spend 0m53s, GEN-SIM - 60m50s.

With 60-to-1 ratio naively we could be able to feed 60-wise concurrency in SIM with 1 instance of GEN (ignoring edge effects etc). At least for short-to-mid term I would imagine that to be acceptable (assuming there is no filtering of events between GEN and SIM).

Yes, more important is to understand why we have warnings from OscarMTProducer, and to avoid situation when GEANT have to manage this. It looks like it happens with HYDJET, EPOS, HIJING, PYQUEN.

makortel · 2020-04-30T16:53:43Z

@wouf Thanks for the clarification.

for HepMCProduct it works perfect with skipSignal parameter, but for HepMCProduct+digis I have to use theDigitizers in HiMixEmbGEN_cff.py
It looks like we need to skip signal for digitizer as well:
Exception Message:
Unrunnable schedule
Module run order problem found: 
generatorSmeared after VtxSmeared [path generation_step], VtxSmeared after randomEngineStateProducer [path generation_step], randomEngineStateProducer after generator [path generation_step], generator after mix [path generation_step], mix consumes generatorSmeared

I took a look of the edmConfigDump' of step2 configuration of 158.0, and see that the theDigitizerscontain two dependencies ongeneratorSmeared: mergedtruthfor tracking MC truth, andpuVtxfor position information of the "pileup" vertices. By quick look both of them genuinely need thegeneratorSmeared` product.

One option would be to remove these two if they are not needed (I can't tell).

Another option would be to continue to mix the digis in the DIGI step, i.e. not along HepMCProduct.

Another option would be to mix the digis along HepMCProduct, and store the MB SimTracks in a PCrossingFrame to be passed for the second mix in the DIGI step to be merged with the signal SimTracks and converted to TrackingParticles. This option likely needs some code development.

Perhaps the best strategy would be to get the full chain working first before trying to optimize? (i.e. second option above)

wouf · 2020-04-30T20:00:26Z

@wouf Thanks for the clarification.
for HepMCProduct it works perfect with skipSignal parameter, but for HepMCProduct+digis I have to use theDigitizers in HiMixEmbGEN_cff.py
It looks like we need to skip signal for digitizer as well:
Exception Message:
Unrunnable schedule
Module run order problem found: 
generatorSmeared after VtxSmeared [path generation_step], VtxSmeared after randomEngineStateProducer [path generation_step], randomEngineStateProducer after generator [path generation_step], generator after mix [path generation_step], mix consumes generatorSmeared
I took a look of the edmConfigDump' of step2 configuration of 158.0, and see that the theDigitizerscontain two dependencies ongeneratorSmeared: mergedtruthfor tracking MC truth, andpuVtxfor position information of the "pileup" vertices. By quick look both of them genuinely need thegeneratorSmeared` product.

One option would be to remove these two if they are not needed (I can't tell).

Another option would be to continue to mix the digis in the DIGI step, i.e. not along HepMCProduct.

Another option would be to mix the digis along HepMCProduct, and store the MB SimTracks in a PCrossingFrame to be passed for the second mix in the DIGI step to be merged with the signal SimTracks and converted to TrackingParticles. This option likely needs some code development.

Perhaps the best strategy would be to get the full chain working first before trying to optimize? (i.e. second option above)

If I understand correctly, this point suppose to use two synchronized GEN-SIM collections (background and signal) as input for DIGI step? In this case the events should be mixed head to head, samples have to have the same number of events, what would be useless, I guess.

makortel · 2020-04-30T22:18:35Z

One option would be to remove these two if they are not needed (I can't tell).

Another option would be to continue to mix the digis in the DIGI step, i.e. not along HepMCProduct.

Another option would be to mix the digis along HepMCProduct, and store the MB SimTracks in a PCrossingFrame to be passed for the second mix in the DIGI step to be merged with the signal SimTracks and converted to TrackingParticles. This option likely needs some code development.
Perhaps the best strategy would be to get the full chain working first before trying to optimize? (i.e. second option above)

If I understand correctly, this point suppose to use two synchronized GEN-SIM collections (background and signal) as input for DIGI step? In this case the events should be mixed head to head, samples have to have the same number of events, what would be useless, I guess.

(assuming we are discussing about the third option above, I wasn't fully sure) Two collections of SimTracks yes, but in the same Event.

wouf · 2020-05-01T05:38:54Z

@makortel Sorry for this misunderstanding, I mean second point above:

One option would be to remove these two if they are not needed (I can't tell).

I'm not sure about it.

Another option would be to continue to mix the digis in the DIGI step, i.e. not along HepMCProduct.

If I understand correctly, this point suppose to use two synchronized GEN-SIM collections (background and signal) as input for DIGI step? In this case the events should be mixed head to head, samples have to have the same number of events, what would be useless, I guess.

Another option would be to mix the digis along HepMCProduct, and store the MB SimTracks in a PCrossingFrame to be passed for the second mix in the DIGI step to be merged with the signal SimTracks and converted to TrackingParticles. This option likely needs some code development.
(assuming we are discussing about the third option above, I wasn't fully sure) Two collections of SimTracks yes, but in the same Event.

Do You mean something like this?
If so, this lead to the segmentation violation:

==8734== Callgrind, a call-graph generating cache profiler
==8734== Copyright (C) 2002-2017, and GNU GPL'd, by Josef Weidendorfer et al.
==8734== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==8734== Command: cmsRun Hydjet_Quenched_B12_5020GeV_cfi_GEN_SIM_PU.py
==8734== 
==8734== For interactive control, run 'callgrind_control -h'.
01-May-2020 07:18:55 CEST  Initiating request to open file file:Hydjet_Quenched_MinBias_5020GeV_cfi_GEN_SIM.root
01-May-2020 07:19:00 CEST  Successfully opened file file:Hydjet_Quenched_MinBias_5020GeV_cfi_GEN_SIM.root


A fatal system signal has occurred: segmentation violation
The following is the call stack containing the origin of the signal.

==8734== Unsupported clone() flags: 0x311
==8734== 
==8734== The only supported clone() uses are:
==8734==  - via a threads library (LinuxThreads or NPTL)
==8734==  - via the implementation of fork or vfork
==8734== 
==8734== Valgrind detected that your program requires
==8734== the following unimplemented functionality:
==8734==    Valgrind does not support general clone().
==8734== This may be because the functionality is hard to implement,
==8734== or because no reasonable program would behave this way,
==8734== or because nobody has yet needed it.  In any case, let us know at
==8734== www.valgrind.org and/or try to work around the problem, if you can.
==8734== 
==8734== Valgrind has to exit now.  Sorry.  Bye!
==8734== 

sched status:
  running_tid=2

Thread 1: status = VgTs_WaitSys syscall 7 (lwpid 8734)
==8734==    at 0x74B5BED: ??? (in /usr/lib64/libc-2.17.so)
==8734==    by 0xCB5BF8E: full_read.constprop.406 (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/pluginFWCoreServicesPlugins.so)
==8734==    by 0xCB5C6CB: edm::service::InitRootHandlers::stacktraceFromThread() (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/pluginFWCoreServicesPlugins.so)
==8734==    by 0xCB5D5A8: sig_dostack_then_abort (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/pluginFWCoreServicesPlugins.so)
==8734==    by 0x71B55EF: ??? (in /usr/lib64/libpthread-2.17.so)
==8734==    by 0x75182E0: __memcpy_ssse3_back (in /usr/lib64/libc-2.17.so)
==8734==    by 0x4179A1: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/bin/slc7_amd64_gcc820/cmsRun)
==8734==    by 0x50203A5: edm::InputTag::operator=(edm::InputTag const&) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreUtilities.so)
==8734==    by 0x25B2FF9C: edm::MixingModule::MixingModule(edm::ParameterSet const&, edm::MixingCache::Config const*) (MixingModule.cc:222)
==8734==    by 0x25B6E6D8: edm::stream::ProducingModuleAdaptor<edm::MixingModule, edm::stream::EDProducerBase, edm::stream::EDProducerAdaptorBase>::setupStreamModules() (makeGlobal.h:42)
==8734==    by 0x4CBAE8C: edm::stream::ProducingModuleAdaptorBase<edm::stream::EDProducerBase>::doPreallocate(edm::PreallocationConfiguration const&) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4BE7985: edm::Maker::makeModule(edm::MakeModuleParams const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4C091E4: edm::Factory::makeModule(edm::MakeModuleParams const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4C0F6A4: edm::ModuleRegistry::getModule(edm::MakeModuleParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4B79198: edm::WorkerRegistry::getWorker(edm::WorkerParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4C0561C: edm::WorkerManager::getWorker(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4BD3525: edm::StreamSchedule::fillWorkers(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::vector<edm::WorkerInPath, std::allocator<edm::WorkerInPath> >&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4BD3C82: edm::StreamSchedule::fillTrigPath(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<edm::HLTGlobalStatus>, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4BD5DB3: edm::StreamSchedule::StreamSchedule(std::shared_ptr<edm::TriggerResultInserter>, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProductRegistry&, edm::BranchIDListHelper&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration>, bool, edm::StreamID, edm::ProcessContext const*) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4C428FE: edm::Schedule::Schedule(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::ProductRegistry&, edm::BranchIDListHelper&, edm::ThinnedAssociationsHelper&, edm::SubProcessParentageHelper const*, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration>, bool, edm::PreallocationConfiguration const&, edm::ProcessContext const*) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4C88E41: edm::ScheduleItems::initSchedule(edm::ParameterSet&, bool, edm::PreallocationConfiguration const&, edm::ProcessContext const*) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4C1A297: edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x4C1C204: edm::EventProcessor::EventProcessor(std::shared_ptr<edm::ProcessDesc>, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/libFWCoreFramework.so)
==8734==    by 0x412D08: main::{lambda()#1}::operator()() const (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/bin/slc7_amd64_gcc820/cmsRun)
==8734==    by 0x411371: main (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/bin/slc7_amd64_gcc820/cmsRun)

Thread 2: status = VgTs_Runnable syscall 56 (lwpid 8858)
==8734==    at 0x74C0851: clone (in /usr/lib64/libc-2.17.so)
==8734==    by 0xCB5BB13: edm::service::cmssw_stacktrace_fork() (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/pluginFWCoreServicesPlugins.so)
==8734==    by 0xCB5C5E9: edm::service::InitRootHandlers::stacktraceHelperThread() (in /cvmfs/cms.cern.ch/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_0_pre5/lib/slc7_amd64_gcc820/pluginFWCoreServicesPlugins.so)
==8734==    by 0x6DC1CCE: execute_native_thread_routine (thread.cc:80)
==8734==    by 0x71ADE64: start_thread (in /usr/lib64/libpthread-2.17.so)
==8734==    by 0x74C088C: clone (in /usr/lib64/libc-2.17.so)

wouf · 2020-05-01T05:59:06Z

I took a look of the edmConfigDump' of step2 configuration of 158.0, and see that the theDigitizerscontain two dependencies ongeneratorSmeared: mergedtruthfor tracking MC truth, andpuVtxfor position information of the "pileup" vertices. By quick look both of them genuinely need thegeneratorSmeared` product.

Can't they use generatorSmeared from the background file (for background only)?

makortel · 2020-05-01T18:58:22Z

Another option would be to continue to mix the digis in the DIGI step, i.e. not along HepMCProduct.

If I understand correctly, this point suppose to use two synchronized GEN-SIM collections (background and signal) as input for DIGI step? In this case the events should be mixed head to head, samples have to have the same number of events, what would be useless, I guess.

Isn't that what the "embedding mode" workflow is doing now, i.e. run MixingModule first in GEN (to overlay gen products), and then again (in playback mode) in DIGI to overlay the SIM products. Or at least that was my understanding based on
#28636 (comment)
#28636 (comment)
#28636 (comment)

Another option would be to mix the digis along HepMCProduct, and store the MB SimTracks in a PCrossingFrame to be passed for the second mix in the DIGI step to be merged with the signal SimTracks and converted to TrackingParticles. This option likely needs some code development.
(assuming we are discussing about the third option above, I wasn't fully sure) Two collections of SimTracks yes, but in the same Event.

Do You mean something like this?

Along that way, but I would not be surprised if MixingModule would need something additional to work properly to skip the signal products (I didn't look), and certainly more work would be needed to properly create TrackingParticles from the overlaid signal+background SimTrack collections.

makortel · 2020-05-01T19:00:47Z

I took a look of the edmConfigDump' of step2 configuration of 158.0, and see that the theDigitizerscontain two dependencies ongeneratorSmeared: mergedtruthfor tracking MC truth, andpuVtxfor position information of the "pileup" vertices. By quick look both of them genuinely need thegeneratorSmeared` product.

Can't they use generatorSmeared from the background file (for background only)?

Possibly in theory, if the background event is the one who drives the vertex position (I can't remember if that was the case). In practice some (possibly heavy) code development would be needed to achieve that.

wouf · 2020-05-01T20:58:47Z

Isn't that what the "embedding mode" workflow is doing now, i.e. run MixingModule first in GEN (to overlay gen products), and then again (in playback mode) in DIGI to overlay the SIM products. Or at least that was my understanding based on
#28636 (comment)
#28636 (comment)
#28636 (comment)

Well, if we trying to use standard workflow, pgen for GEN-SIM should looks like:

pgen = cms.Sequence(cms.SequencePlaceholder("mix")+cms.SequencePlaceholder("generator")+cms.SequencePlaceholder("randomEngineStateProducer")+cms.SequencePlaceholder("mix")+VertexSmearing+GenSmeared+genParticles+hiGenJets)

here the first mix is only to deliver proper generator parameters, so it would run with skipSignal parameter. BUT the second mix should be run without skipSignal, this brings us back to the #28636 (comment)
Could You please to advise how to turn skipSignal off for the second mix?

Along that way, but I would not be surprised if MixingModule would need something additional to work properly to skip the signal products (I didn't look), and certainly more work would be needed to properly create TrackingParticles from the overlaid signal+background SimTrack collections.

So, the game is not worth the candle?

wouf · 2020-05-01T21:16:07Z

Can't they use generatorSmeared from the background file (for background only)?

Possibly in theory, if the background event is the one who drives the vertex position (I can't remember if that was the case). In practice some (possibly heavy) code development would be needed to achieve that.

I assumed this is what You meant in the #28636 (comment) Isn't it? Is it long-term plan?

makortel · 2020-05-01T23:47:58Z

Isn't that what the "embedding mode" workflow is doing now, i.e. run MixingModule first in GEN (to overlay gen products), and then again (in playback mode) in DIGI to overlay the SIM products. Or at least that was my understanding based on
#28636 (comment)
#28636 (comment)
#28636 (comment)

Well, if we trying to use standard workflow, pgen for GEN-SIM should looks like:
pgen = cms.Sequence(cms.SequencePlaceholder("mix")+cms.SequencePlaceholder("generator")+cms.SequencePlaceholder("randomEngineStateProducer")+cms.SequencePlaceholder("mix")+VertexSmearing+GenSmeared+genParticles+hiGenJets)
here the first mix is only to deliver proper generator parameters, so it would run with skipSignal parameter. BUT the second mix should be run without skipSignal, this brings us back to the #28636 (comment)
Could You please to advise how to turn skipSignal off for the second mix?

The second mix would run as in the current workflow, i.e. in the DIGI step and overlay the SIM information from the signal and from the background, and run the final digitization on top of that. Or am I missing something why that would not work?

makortel · 2020-05-01T23:57:01Z

Can't they use generatorSmeared from the background file (for background only)?

Possibly in theory, if the background event is the one who drives the vertex position (I can't remember if that was the case). In practice some (possibly heavy) code development would be needed to achieve that.

I assumed this is what You meant in the #28636 (comment) Isn't it?

Yes, I meant that. But at that time I was not aware of the circular dependence of mergedtruth and puVtx eventually depending on the output of the generator, which makes it a lot more complicated.

Is it long-term plan?

I am not really in a position to tell. I suppose it is up to the simulation L2's and likely eventually on the manpower to do it.

wouf · 2020-05-02T08:09:02Z

Well, if we trying to use standard workflow, pgen for GEN-SIM should looks like:
pgen = cms.Sequence(cms.SequencePlaceholder("mix")+cms.SequencePlaceholder("generator")+cms.SequencePlaceholder("randomEngineStateProducer")+cms.SequencePlaceholder("mix")+VertexSmearing+GenSmeared+genParticles+hiGenJets)
here the first mix is only to deliver proper generator parameters, so it would run with skipSignal parameter. BUT the second mix should be run without skipSignal, this brings us back to the #28636 (comment)
Could You please to advise how to turn skipSignal off for the second mix?
The second mix would run as in the current workflow, i.e. in the DIGI step and overlay the SIM information from the signal and from the background, and run the final digitization on top of that. Or am I missing something why that would not work?

I guess it needs signal in CF as well? Or maybe I'm doing something wrong, but with

pgen = cms.Sequence(cms.SequencePlaceholder("mix")+cms.SequencePlaceholder("generator")+cms.SequencePlaceholder("randomEngineStateProducer")+VertexSmearing+GenSmeared+genParticles+hiGenJets)

I get:

----- Begin Fatal Exception 02-May-2020 09:51:52 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLTAnalyzerEndpath'
   [2] Prefetching for module L1TRawToDigi/'hltGtStage2Digis'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Prefetching for module GEMDigiToRawModule/'gemPacker'
   [5] Calling method for module GEMDigiProducer/'simMuonGEMDigis'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: CrossingFrame<PSimHit>
Looking for module label: mix
Looking for productInstanceName: g4SimHitsMuonGEMHits

P.S. - worflows for GEN-SIM:

cmsDriver.py Hydjet_Quenched_B12_5020GeV_cfi --conditions auto:phase1_2018_realistic --scenario HeavyIons -n 2 --era Run2_2018 --eventcontent RAWSIM --relval 2000,1 -s GEN,SIM --datatier GEN-SIM --beamspot Realistic25ns13TeVEarly2018Collision --geometry DB:Extended --pileup HiMixEmbGEN --pileup_input file:Hydjet_Quenched_MinBias_5020GeV_cfi_GEN_SIM.root --fileout step1_GEN_SIM_PU.root

for DIGIRAW:

cmsDriver.py step2 -s DIGI,L1,DIGI2RAW,HLT --conditions auto:phase1_2018_realistic --scenario HeavyIons -n 2 --era Run2_2018 --eventcontent RAWSIM --relval 2000,1 --datatier GEN-SIM-RAW --filein=file:step1_GEN_SIM_PU.root --beamspot Realistic25ns13TeVEarly2018Collision --geometry DB:Extended --pileup HiMixGEN --pileup_input file:Hydjet_Quenched_MinBias_5020GeV_cfi_GEN_SIM.root

wouf · 2020-05-02T08:15:54Z

I assumed this is what You meant in the #28636 (comment) Isn't it?

Yes, I meant that. But at that time I was not aware of the circular dependence of mergedtruth and puVtx eventually depending on the output of the generator, which makes it a lot more complicated.

I see, thank You for explanation.

Is it long-term plan?

I am not really in a position to tell. I suppose it is up to the simulation L2's and likely eventually on the manpower to do it.

Do You know, who may answer more exact?

wouf · 2020-05-02T13:08:41Z

cmsDriver.py step2 -s DIGI,L1,DIGI2RAW,HLT --conditions auto:phase1_2018_realistic --scenario HeavyIons -n 2 --era Run2_2018 --eventcontent RAWSIM --relval 2000,1 --datatier GEN-SIM-RAW --filein=file:step1_GEN_SIM_PU.root --beamspot Realistic25ns13TeVEarly2018Collision --geometry DB:Extended --pileup HiMixGEN --pileup_input file:Hydjet_Quenched_MinBias_5020GeV_cfi_GEN_SIM.root

Sorry, my mistake! With --pileup HiMix it works. Correct workflow for step2 is:

cmsDriver.py step2 -s DIGI:pdigi_hi,L1,DIGI2RAW,HLT:@fake2 --conditions auto:phase1_2018_realistic --scenario HeavyIons -n 2 --era Run2_2018 --eventcontent FEVTDEBUGHLT --datatier GEN-SIM-DIGI-RAW-HLTDEBUG --filein=file:step1_GEN_SIM_PU.root --beamspot Realistic25ns13TeVEarly2018Collision --geometry DB:Extended --pileup HiMix --pileup_input file:Hydjet_Quenched_MinBias_5020GeV_cfi_GEN_SIM.root

makortel · 2020-05-06T18:35:51Z

I guess it needs signal in CF as well?

No, MixingModule reads all of its input from "regular" containers (for both signal and pileup events). CrossingFrame is an output product of the MixingModule (mainly used by muon digitizers). Do you have all the era customizations from

cmssw/SimGeneral/MixingModule/python/mixObjects_cfi.py

Lines 226 to 251 in 0dd520d

    
           from Configuration.Eras.Modifier_run2_GEM_2017_cff import run2_GEM_2017 
        
           run2_GEM_2017.toModify( theMixObjects, 
        
               mixSH = dict( 
        
                   input = theMixObjects.mixSH.input + [ cms.InputTag("g4SimHits","MuonGEMHits") ], 
        
                   subdets = theMixObjects.mixSH.subdets + [ 'MuonGEMHits' ], 
        
                   crossingFrames = theMixObjects.mixSH.crossingFrames + [ 'MuonGEMHits' ] 
        
               ) 
        
           ) 
        
           (premix_stage1 & run2_GEM_2017).toModify(theMixObjects, 
        
               mixSH = dict( 
        
                   pcrossingFrames = theMixObjects.mixSH.pcrossingFrames + [ 'MuonGEMHits' ] 
        
               ) 
        
           ) 
        
           from Configuration.Eras.Modifier_run3_GEM_cff import run3_GEM 
        
           run3_GEM.toModify( theMixObjects, 
        
               mixSH = dict( 
        
                   input = theMixObjects.mixSH.input + [ cms.InputTag("g4SimHits","MuonGEMHits") ], 
        
                   subdets = theMixObjects.mixSH.subdets + [ 'MuonGEMHits' ], 
        
                   crossingFrames = theMixObjects.mixSH.crossingFrames + [ 'MuonGEMHits' ] 
        
               ) 
        
           ) 
        
           (premix_stage1 & run3_GEM).toModify(theMixObjects, 
        
               mixSH = dict( 
        
                   pcrossingFrames = theMixObjects.mixSH.pcrossingFrames + [ 'MuonGEMHits' ] 
        
               ) 
        
           )

?

Is it long-term plan?

I am not really in a position to tell. I suppose it is up to the simulation L2's and likely eventually on the manpower to do it.

Do You know, who may answer more exact?

Technically it would be simulation L2s @civanch and @sbein. Depending on the expected improvements in the workflow execution in grid I might be able to help, but given the extent of development that would require planning.

wouf · 2020-05-08T19:59:19Z

I guess it needs signal in CF as well?

No, MixingModule reads all of its input from "regular" containers

Could You please to clarify, what is the "regular" containers? Thanks!

(for both signal and pileup events). CrossingFrame is an output product of the MixingModule mainly used by muon digitizers). Do you have all the era customizations from

cmssw/SimGeneral/MixingModule/python/mixObjects_cfi.py

Lines 226 to 251 in 0dd520d

from Configuration.Eras.Modifier_run2_GEM_2017_cff import run2_GEM_2017

run2_GEM_2017.toModify( theMixObjects,

mixSH = dict(

input = theMixObjects.mixSH.input + [ cms.InputTag("g4SimHits","MuonGEMHits") ],

subdets = theMixObjects.mixSH.subdets + [ 'MuonGEMHits' ],

crossingFrames = theMixObjects.mixSH.crossingFrames + [ 'MuonGEMHits' ]

)

)

(premix_stage1 & run2_GEM_2017).toModify(theMixObjects,

mixSH = dict(

pcrossingFrames = theMixObjects.mixSH.pcrossingFrames + [ 'MuonGEMHits' ]

)

)

from Configuration.Eras.Modifier_run3_GEM_cff import run3_GEM

run3_GEM.toModify( theMixObjects,

mixSH = dict(

input = theMixObjects.mixSH.input + [ cms.InputTag("g4SimHits","MuonGEMHits") ],

subdets = theMixObjects.mixSH.subdets + [ 'MuonGEMHits' ],

crossingFrames = theMixObjects.mixSH.crossingFrames + [ 'MuonGEMHits' ]

)

)

(premix_stage1 & run3_GEM).toModify(theMixObjects,

mixSH = dict(

pcrossingFrames = theMixObjects.mixSH.pcrossingFrames + [ 'MuonGEMHits' ]

)

)

?

I guess none of them. We are using --era Run2_2018_pp_on_AA Why do You asking?

Technically it would be simulation L2s @civanch and @sbein. Depending on the expected improvements in the workflow execution in grid I might be able to help, but given the extent of development that would require planning.

I see, thanks.

makortel · 2020-05-11T14:08:18Z

I guess it needs signal in CF as well?

No, MixingModule reads all of its input from "regular" containers

Could You please to clarify, what is the "regular" containers? Thanks!

The containers produced by OscarMTProducer, .e.g. for PSimHits that is std::vector<PSimHit>.

Do you have all the era customizations from SimGeneral/MixingModule/python/mixObjects_cfi.py?

I guess none of them. We are using --era Run2_2018_pp_on_AA Why do You asking?

The missing product error you showed in #28636 (comment) was about hits in GEM, which in mixObjects_cfi are added with run2_GEM_2017 and run3_GEM modifiers, which means that if you override the objects like here
https://github.com/wouf/cmssw/blob/from-CMSSW_11_1_0_pre5/SimGeneral/MixingModule/python/HiMixEmbGEN_cff.py
you need to also add the GEM objects with the modifiers.

wouf · 2020-05-13T21:44:58Z

I guess it needs signal in CF as well?

No, MixingModule reads all of its input from "regular" containers

Could You please to clarify, what is the "regular" containers? Thanks!

The containers produced by OscarMTProducer, .e.g. for PSimHits that is std::vector<PSimHit>.

Do you have all the era customizations from SimGeneral/MixingModule/python/mixObjects_cfi.py?

I guess none of them. We are using --era Run2_2018_pp_on_AA Why do You asking?

The missing product error you showed in #28636 (comment) was about hits in GEM, which in mixObjects_cfi are added with run2_GEM_2017 and run3_GEM modifiers, which means that if you override the objects like here
https://github.com/wouf/cmssw/blob/from-CMSSW_11_1_0_pre5/SimGeneral/MixingModule/python/HiMixEmbGEN_cff.py
you need to also add the GEM objects with the modifiers.

Thank You!

wouf · 2020-06-10T21:53:10Z

Thank You @makortel , @qliphy , @Dr15Jones , @gkrintir , @mandrenguyen , @stepobr , all!
The result of this discussion in #30185

cmsbuild added the pending-assignment label Dec 17, 2019

cmsbuild added generators-pending pending-signatures and removed pending-assignment labels Dec 17, 2019

wouf mentioned this issue Jan 20, 2020

Hydjet Interface External #28764

Closed

wouf mentioned this issue Jan 20, 2020

Hydjet integration as external cms-sw/cmsdist#5412

Merged

wouf mentioned this issue Jun 10, 2020

HYDJET, PYQUEN and the new embedding #30185

Merged

wouf closed this as completed Jun 10, 2020

wouf mentioned this issue Oct 26, 2022

Deal with late parents in HydjetHadronizer #39784

Merged

Access edm::event from GeneratorInterface #28636

Access edm::event from GeneratorInterface #28636

Comments

wouf commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

Dr15Jones commented Dec 17, 2019

Dr15Jones commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

wouf commented Jan 6, 2020

qliphy commented Jan 14, 2020

wouf commented Jan 14, 2020 • edited Loading

qliphy commented Jan 15, 2020

wouf commented Jan 15, 2020

qliphy commented Jan 20, 2020

wouf commented Jan 20, 2020

qliphy commented Jan 21, 2020 • edited Loading

wouf commented Jan 21, 2020

qliphy commented Jan 21, 2020

wouf commented Jan 21, 2020 • edited Loading

wouf commented Jan 21, 2020

wouf commented Jan 23, 2020

qliphy commented Jan 23, 2020

wouf commented Jan 23, 2020

qliphy commented Jan 23, 2020

wouf commented Jan 23, 2020

qliphy commented Jan 27, 2020 • edited Loading

Dr15Jones commented Jan 27, 2020

makortel commented Jan 27, 2020

wouf commented Apr 22, 2020

makortel commented Apr 23, 2020

wouf commented Apr 23, 2020 • edited Loading

wouf commented Apr 30, 2020

makortel commented Apr 30, 2020

makortel commented Apr 30, 2020

wouf commented Apr 30, 2020

wouf commented Apr 30, 2020

makortel commented Apr 30, 2020

wouf commented Apr 30, 2020

makortel commented Apr 30, 2020

wouf commented May 1, 2020

wouf commented May 1, 2020 • edited Loading

makortel commented May 1, 2020

makortel commented May 1, 2020

wouf commented May 1, 2020 • edited Loading

wouf commented May 1, 2020

makortel commented May 1, 2020

makortel commented May 1, 2020

wouf commented May 2, 2020

wouf commented May 2, 2020

wouf commented May 2, 2020

makortel commented May 6, 2020

wouf commented May 8, 2020

makortel commented May 11, 2020

wouf commented May 13, 2020

wouf commented Jun 10, 2020

wouf commented Jan 14, 2020 •

edited

Loading

qliphy commented Jan 21, 2020 •

edited

Loading

wouf commented Jan 21, 2020 •

edited

Loading

qliphy commented Jan 27, 2020 •

edited

Loading

wouf commented Apr 23, 2020 •

edited

Loading

wouf commented May 1, 2020 •

edited

Loading

wouf commented May 1, 2020 •

edited

Loading