Integrating PPS full sim in Run3 (only) #30575

mundim · 2020-07-07T14:10:06Z

PR description:

Another step towards the ntegration of PPS full simulation into cmssw (still only for Run3).

This PR propagates the changes made by @civanch in the central simulation. Now, PPS simulation is triggered by the flag LHCTransport (False by default), which needs to be set to TRUE. Since the PPS geometry is not yet loaded into the DB, the digi was removed from the chain in order to avoid failure in the standard tests. In summary, this PR changes SimPPS/PPSSimTrackProducer and PPSProtonTransport to save inthe LHCTransport data container only the propagate protons (only secondary vertices in from of PPS region).

Another PR will be issued when PPS geometry gets loaded into the DB, pluging in the digi part.

PR validation:

scram b runtests ran with no issue
scram b code-checks and scram b code-formats also ran ok

runTheMatrix.py -l limited -i all --ibeos was ran again with the results:
35 33 27 20 15 4 1 1 1 tests passed, 0 1 5 0 0 0 0 0 0 failed

All the 6 wf failed due to timeout. Running them again gave:
6 6 6 5 2 tests passed, 0 0 0 0 0 failed

if this PR is a backport please specify the original PR and why you need to backport that PR:

Before submitting your pull requests, make sure you followed this checklist:

verify that the PR is really intended for the chosen branch
verify that changes follow CMS Naming, Coding, And Style Rules
verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

cmsbuild · 2020-07-07T14:10:29Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-07-07T14:15:07Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30575/16810

This PR adds an extra 60KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/Eras/python/Era_Run3_cff.py modified in PR(s): PPS: direct simulation update A #30466

cmsbuild · 2020-07-07T14:15:32Z

A new Pull Request was created by @mundim for master.

It involves the following packages:

Configuration/Eras
Configuration/StandardSequences
EventFilter/CTPPSRawToDigi
EventFilter/RawDataCollector
IOMC/RandomEngine
SimG4Core/Application
SimGeneral/MixingModule
SimPPS/Configuration
SimPPS/PPSPixelDigiProducer
SimTransport/PPSProtonTransport

@perrotta, @smuzaffar, @civanch, @Dr15Jones, @makortel, @emeschi, @mdhildreth, @cmsbuild, @mommsen, @franzoni, @silviodonato, @slava77, @jpata, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@fabiocos, @makortel, @felicepantaleo, @forthommel, @jan-kaspar, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @wddgit, @lecriste, @mtosi, @dgulhan, @slomeo this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

civanch · 2020-07-08T09:21:59Z

SimG4Core/Application/python/g4SimHits_cfi.py

@@ -575,6 +575,23 @@
 run2_common.toModify( g4SimHits.HFShowerLibrary, FileName = 'SimG4CMS/Calo/data/HFShowerLibrary_npmt_noatt_eta4_16en_v4.root' )
 run2_common.toModify( g4SimHits.HFShower, ProbMax = 0.5)

+
+from Configuration.Eras.Modifier_ctpps_2021_cff import ctpps_2021


@mundim , if the modifier is configured in this way, then central detector simulation is disabled and only PPS simulation is enabled. In that case, this modifier should be applied only for specific ctpps_2021 simulation. I assume it is the case and in the normal WF used in tests this should not be used.

Hi @civanch (sorry for the late reply, my laptop had to be taken to the maintenance, I'm using a tablet).

I'm not sure I understood the problem. I just picked up from the comments in the PR 29916, as suggested by @makortel for Run2 (#29916 (comment)) and replaced ctpps_2016 by ctpps_2021.

Do I have to make others changes or the rebase as your next message is enough?

@civanch Could you clarify in which way the customization below disables central detector simulation? Is it perhaps the g4simhits.Generator.HepMCProductLabel = 'LHCTransport?

I don' t believe so. We have done private pps simulation with this label and the central CMS simulation was not disabled.

civanch · 2020-07-08T09:24:50Z

@mundim , rebase is needed.

cmsbuild · 2020-07-10T22:42:52Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-07-10T22:48:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30575/16921

This PR adds an extra 60KB to repository

cmsbuild · 2020-07-10T22:49:22Z

Pull request #30575 was updated. @perrotta, @smuzaffar, @civanch, @Dr15Jones, @makortel, @emeschi, @mdhildreth, @cmsbuild, @mommsen, @franzoni, @silviodonato, @slava77, @jpata, @fabiocos, @davidlange6 can you please check and sign again.

civanch · 2020-07-10T23:04:19Z

please test

cmsbuild · 2020-07-10T23:04:47Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-07-10-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/7876/console Started: 2020/07/11 02:30

smorovic · 2020-08-29T15:59:51Z

+1

perrotta · 2020-08-29T18:15:34Z

+1

ctppsPixelRawData included in the ctppsRawData Task for Run3
digi's still left as placeholders, while waiting for an updated PPS geometry in the DB
Jenkins tests pass and show no differences in reco outputs

perrotta · 2020-08-29T18:18:20Z

@mundim the static analyzer shows that double theta is uselessly defined (L25) and modified (L35-36) in SimTransport/PPSProtonTransport/src/BaseProtonTransport.cc

It could be cleaned either here, or in the next PR, if you have to touch the same file

mundim · 2020-08-29T18:25:29Z

@mundim the static analyzer shows that double theta is uselessly defined (L25) and modified (L35-36) in SimTransport/PPSProtonTransport/src/BaseProtonTransport.cc

It could be cleaned either here, or in the next PR, if you have to touch the same file

If you don't mind, I prefer to leave it to the next PR (which will be very small, I mean, very few changes). This one has already a huge history, any change will restart it again. What do you think? And still, it is planned to change the transport again soon, when realistic optics will be available for run3, so we will have at least two opportunities to address this problem.

perrotta · 2020-08-29T19:10:37Z

@mundim the static analyzer shows that double theta is uselessly defined (L25) and modified (L35-36) in SimTransport/PPSProtonTransport/src/BaseProtonTransport.cc
It could be cleaned either here, or in the next PR, if you have to touch the same file

If you don't mind, I prefer to leave it to the next PR (which will be very small, I mean, very few changes). This one has already a huge history, any change will restart it again. What do you think? And still, it is planned to change the transport again soon, when realistic optics will be available for run3, so we will have at least two opportunities to address this problem.

Sure, I only wanted to point it to you, in case you missed it!

mundim · 2020-08-29T19:13:45Z

@mundim the static analyzer shows that double theta is uselessly defined (L25) and modified (L35-36) in SimTransport/PPSProtonTransport/src/BaseProtonTransport.cc
It could be cleaned either here, or in the next PR, if you have to touch the same file

If you don't mind, I prefer to leave it to the next PR (which will be very small, I mean, very few changes). This one has already a huge history, any change will restart it again. What do you think? And still, it is planned to change the transport again soon, when realistic optics will be available for run3, so we will have at least two opportunities to address this problem.

Sure, I only wanted to point it to you, in case you missed it!

thanks... I'll do it right away and leave it in git for the proper time to submit. I'll need to scan all PR's because there are other issues that were left behind for "future PR". As soon as we get this integrated, it will be time to address them all.

civanch · 2020-08-30T11:47:55Z

@makortel , @Dr15Jones , @silviodonato , @qliphy , please check and sign this PR - it stay too long before can be merged, now is in a good shape and include needed modifications for PPS simulation. Following PR will be needed, when GT with full Run-3 geometry will be available.

silviodonato · 2020-08-31T07:22:29Z

+operations

silviodonato · 2020-09-01T09:28:29Z

Kind reminder @cms-sw/core-l2

makortel · 2020-09-01T13:01:23Z

IOMC/RandomEngine/python/IOMC_cff.py

@@ -19,7 +19,7 @@
    ),
    LHCTransport = cms.PSet(
        initialSeed = cms.untracked.uint32(87654321),
-        engineName = FastSimEngine
+        engineName = cms.untracked.string('TRandom3')


Do we really want to use TRandom3 engine? @civanch

civanch · 2020-09-01T13:31:05Z

@makortel , initially it was told that Hector transport may be done only with TRandom3. I do not think that migration is impossible. Migration from hard-coded TRandom3 to a general scheme of CMSSW should be done in a separate PR, dedicated only for that. Let us sign this and making an issue.

makortel · 2020-09-01T13:48:45Z

+1

Thanks @civanch for the explanation.

cmsbuild · 2020-09-01T13:49:08Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-09-01T14:30:12Z

+1

cmsbuild added this to the CMSSW_11_2_X milestone Jul 7, 2020

cmsbuild added code-checks-pending comparison-pending core-pending daq-pending operations-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending labels Jul 7, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 7, 2020

civanch reviewed Jul 8, 2020

View reviewed changes

mundim force-pushed the PPSFullSim4Run3 branch from f3b1da9 to 6fcff41 Compare July 10, 2020 22:42

cmsbuild added code-checks-pending and removed code-checks-approved labels Jul 10, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 10, 2020

cmsbuild added tests-started and removed tests-pending labels Jul 10, 2020

cmsbuild added daq-approved and removed daq-pending labels Aug 29, 2020

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Aug 29, 2020

cmsbuild added operations-approved and removed operations-pending labels Aug 31, 2020

cmsbuild mentioned this pull request Sep 1, 2020

Initial Changes for Incoporating GEM GE0 into the Reco Pipeline #31309

Merged

makortel reviewed Sep 1, 2020

View reviewed changes

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Sep 1, 2020

cmsbuild added orp-approved and removed orp-pending labels Sep 1, 2020

cmsbuild merged commit 4d5966f into cms-sw:master Sep 1, 2020

This was referenced Nov 6, 2020

g4SimHitsCTPPSPixelHits not found #31991

Closed

Update of GEN-SIM files used in runTheMatrix --useInputs and of the pileup samples (Run-3 only) #32125

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrating PPS full sim in Run3 (only) #30575

Integrating PPS full sim in Run3 (only) #30575

mundim commented Jul 7, 2020 •

edited

Loading

cmsbuild commented Jul 7, 2020

cmsbuild commented Jul 7, 2020

cmsbuild commented Jul 7, 2020

civanch Jul 8, 2020

mundim Jul 9, 2020

makortel Jul 9, 2020

mundim Jul 9, 2020

civanch commented Jul 8, 2020

cmsbuild commented Jul 10, 2020

cmsbuild commented Jul 10, 2020

cmsbuild commented Jul 10, 2020

civanch commented Jul 10, 2020

cmsbuild commented Jul 10, 2020 •

edited

Loading

smorovic commented Aug 29, 2020

perrotta commented Aug 29, 2020

perrotta commented Aug 29, 2020

mundim commented Aug 29, 2020

perrotta commented Aug 29, 2020

mundim commented Aug 29, 2020

civanch commented Aug 30, 2020

silviodonato commented Aug 31, 2020

silviodonato commented Sep 1, 2020

makortel Sep 1, 2020

civanch commented Sep 1, 2020

makortel commented Sep 1, 2020

cmsbuild commented Sep 1, 2020

silviodonato commented Sep 1, 2020

Integrating PPS full sim in Run3 (only) #30575

Integrating PPS full sim in Run3 (only) #30575

Conversation

mundim commented Jul 7, 2020 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jul 7, 2020

cmsbuild commented Jul 7, 2020

cmsbuild commented Jul 7, 2020

civanch Jul 8, 2020

Choose a reason for hiding this comment

mundim Jul 9, 2020

Choose a reason for hiding this comment

makortel Jul 9, 2020

Choose a reason for hiding this comment

mundim Jul 9, 2020

Choose a reason for hiding this comment

civanch commented Jul 8, 2020

cmsbuild commented Jul 10, 2020

cmsbuild commented Jul 10, 2020

cmsbuild commented Jul 10, 2020

civanch commented Jul 10, 2020

cmsbuild commented Jul 10, 2020 • edited Loading

smorovic commented Aug 29, 2020

perrotta commented Aug 29, 2020

perrotta commented Aug 29, 2020

mundim commented Aug 29, 2020

perrotta commented Aug 29, 2020

mundim commented Aug 29, 2020

civanch commented Aug 30, 2020

silviodonato commented Aug 31, 2020

silviodonato commented Sep 1, 2020

makortel Sep 1, 2020

Choose a reason for hiding this comment

civanch commented Sep 1, 2020

makortel commented Sep 1, 2020

cmsbuild commented Sep 1, 2020

silviodonato commented Sep 1, 2020

mundim commented Jul 7, 2020 •

edited

Loading

cmsbuild commented Jul 10, 2020 •

edited

Loading