PPS Pixel data unpacking #35067

fabferro · 2021-08-30T08:44:43Z

PR description:

This PR modifies the PPS pixel unpacking code in view of the changes for Run3.
The changes concern 13 out of 64 bits of the data word that encode the pixel coordinates: the "double column-pixel id" convention is changed into a "column-row" one, due to the use of a slightly different readout chip (PROC600 instead of PsiDIG46).
The back compatibility with Run2 data is ensured by means of era modifiers: Run3 convention becomes the default one and Run2 one is triggered by means of ctpps_201* era modifiers.
The part of the code packing the simulated data is also changed accordingly.

PR validation:

The detectors are not yet installed. They will be installed beginning of November 21. Private validation on lab data, Run2 data and full simulation was done.

if this PR is a backport please specify the original PR and why you need to backport that PR:

No backport expected.

fabferro · 2021-08-30T08:45:15Z

@jan-kaspar @mundim @clemencia @AndreaBellora FYI

cmsbuild · 2021-08-30T08:52:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35067/24949

This PR adds an extra 24KB to repository

cmsbuild · 2021-08-30T08:53:19Z

A new Pull Request was created by @fabferro (Fabrizio Ferro) for master.

It involves the following packages:

EventFilter/CTPPSRawToDigi (reconstruction)

@jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @jan-kaspar this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

slava77 · 2021-08-30T13:14:30Z

@cmsbuild please test

cmsbuild · 2021-08-30T16:19:44Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2bbeda/18131/summary.html
COMMIT: 9832be4
CMSSW: CMSSW_12_1_X_2021-08-29-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35067/18131/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 3000352
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3000324
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

jpata · 2021-08-31T07:49:27Z

Could a unit test / test config be useful here?

jpata · 2021-09-02T11:15:01Z

Kind ping @fabferro. Just to clarify, I think this looks fine from the reco point of view, but typically we try to ask for the possibility (if it's not unreasonably complicated or useless) to have a test for new functionality, to be able to (at least in principle) run the code we sign. I think in this case a small test config on some testbeam data file could be helpful. Let me know what you think!

fabferro · 2021-09-02T13:16:12Z

Kind ping @fabferro. Just to clarify, I think this looks fine from the reco point of view, but typically we try to ask for the possibility (if it's not unreasonably complicated or useless) to have a test for new functionality, to be able to (at least in principle) run the code we sign. I think in this case a small test config on some testbeam data file could be helpful. Let me know what you think!

It sounds reasonable. I'll ask PPS online people to provide me a(nother) data file and a temporary DAQ map.
Where can I put them for your convenience?

jpata · 2021-09-02T14:01:34Z

Could something be done in analogy to PR#34759, https://github.com/ChrisMisan/cmssw/blob/465b1657fb94e540dd43becf6ccb9073654b2e42/RecoPPS/Local/test/totemTiming_digiConverter_reco_cfg.py?

The test file could then perhaps be located in /eos/cms/store/group/dpg_ctpps/.

jpata · 2021-09-02T14:02:19Z

EventFilter/CTPPSRawToDigi/python/ctppsRawToDigi_cff.py

+from Configuration.Eras.Modifier_ctpps_2016_cff import ctpps_2016
+from Configuration.Eras.Modifier_ctpps_2017_cff import ctpps_2017
+from Configuration.Eras.Modifier_ctpps_2018_cff import ctpps_2018
+(ctpps_2016 | ctpps_2017 | ctpps_2018).toModify(ctppsPixelDigis, isRun3 = cms.bool(False) )


Slava pointed to me that the type does not need (should not) be specified here, sorry for missing it earlier.

Suggested change

(ctpps_2016 | ctpps_2017 | ctpps_2018).toModify(ctppsPixelDigis, isRun3 = cms.bool(False) )

(ctpps_2016 | ctpps_2017 | ctpps_2018).toModify(ctppsPixelDigis, isRun3 = False )

I missed it too when I re-read the code before submission. Slava never misses! ;-)
Thanks.
BTW, it works fine as well...

cmsbuild · 2021-09-02T14:22:44Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35067/25032

This PR adds an extra 12KB to repository

cmsbuild · 2021-09-02T14:23:05Z

Pull request #35067 was updated. @jpata, @cmsbuild, @slava77 can you please check and sign again.

fabferro · 2021-09-03T09:19:48Z

The test config just added reads lab data from /eos/cms/store/group/dpg_ctpps/comm_ctpps/PixelAlive_562_RAW.root and produces a root file with DIGIs applying a dummy DAQ map (/eos/cms/store/group/dpg_ctpps/comm_ctpps/CTPPSPixel_DAQMapping_AnalysisMask.db). The output, if limited to a few events only, can be manually compared with a binary-to-txt-dump file /eos/cms/store/group/dpg_ctpps/comm_ctpps/Run562_dumpBinary.txt.

cmsbuild · 2021-09-03T09:21:14Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35067/25052

This PR adds an extra 16KB to repository

cmsbuild · 2021-09-03T09:21:39Z

Pull request #35067 was updated. @jpata, @cmsbuild, @slava77 can you please check and sign again.

jpata · 2021-09-03T11:01:42Z

@cmsbuild please test

cmsbuild · 2021-09-03T14:08:56Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2bbeda/18281/summary.html
COMMIT: 20537f2
CMSSW: CMSSW_12_1_X_2021-09-02-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35067/18281/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 3 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 3000404
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3000382
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

jpata · 2021-09-03T14:19:39Z

+reconstruction

for 20537f2
no relevant reco changes expected or observed (1325.81 is from Non-reproducibility in DeepTau in 1325.81 #32628)
a test configuration is provided and runs
slides in https://indico.cern.ch/event/1071803/contributions/4507619/attachments/2301676/3915262/PPS_Reco_030921.pdf

cmsbuild · 2021-09-03T14:20:03Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2021-09-04T06:24:45Z

+1

fabferro added 3 commits August 27, 2021 09:48

pixel unpacking for Run3

718e0d8

code format changes

d02f748

adding const

9832be4

cmsbuild added this to the CMSSW_12_1_X milestone Aug 30, 2021

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Aug 30, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 30, 2021

cmsbuild added tests-started and removed tests-pending labels Aug 30, 2021

cmsbuild added tests-approved and removed tests-started labels Aug 30, 2021

jpata reviewed Sep 2, 2021

View reviewed changes

bug fix

2c1a831

cmsbuild added code-checks-pending tests-pending and removed tests-approved code-checks-approved labels Sep 2, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 2, 2021

adding test config

20537f2

cmsbuild added code-checks-pending and removed code-checks-approved labels Sep 3, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 3, 2021

cmsbuild added tests-started and removed tests-pending labels Sep 3, 2021

cmsbuild added tests-approved and removed tests-started labels Sep 3, 2021

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Sep 3, 2021

cmsbuild added orp-approved and removed orp-pending labels Sep 4, 2021

cmsbuild merged commit f18a472 into cms-sw:master Sep 4, 2021

cmsbuild mentioned this pull request Sep 4, 2021

Updates python packages to their latest version cms-sw/cmsdist#7263

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PPS Pixel data unpacking #35067

PPS Pixel data unpacking #35067

fabferro commented Aug 30, 2021

fabferro commented Aug 30, 2021

cmsbuild commented Aug 30, 2021

cmsbuild commented Aug 30, 2021

slava77 commented Aug 30, 2021

cmsbuild commented Aug 30, 2021

jpata commented Aug 31, 2021

jpata commented Sep 2, 2021

fabferro commented Sep 2, 2021

jpata commented Sep 2, 2021

jpata Sep 2, 2021

fabferro Sep 2, 2021

cmsbuild commented Sep 2, 2021

cmsbuild commented Sep 2, 2021

fabferro commented Sep 3, 2021

cmsbuild commented Sep 3, 2021

cmsbuild commented Sep 3, 2021

jpata commented Sep 3, 2021

cmsbuild commented Sep 3, 2021

jpata commented Sep 3, 2021

cmsbuild commented Sep 3, 2021

perrotta commented Sep 4, 2021

	(ctpps_2016 \| ctpps_2017 \| ctpps_2018).toModify(ctppsPixelDigis, isRun3 = cms.bool(False) )
	(ctpps_2016 \| ctpps_2017 \| ctpps_2018).toModify(ctppsPixelDigis, isRun3 = False )

PPS Pixel data unpacking #35067

PPS Pixel data unpacking #35067

Conversation

fabferro commented Aug 30, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

fabferro commented Aug 30, 2021

cmsbuild commented Aug 30, 2021

cmsbuild commented Aug 30, 2021

slava77 commented Aug 30, 2021

cmsbuild commented Aug 30, 2021

Comparison Summary

jpata commented Aug 31, 2021

jpata commented Sep 2, 2021

fabferro commented Sep 2, 2021

jpata commented Sep 2, 2021

jpata Sep 2, 2021

Choose a reason for hiding this comment

fabferro Sep 2, 2021

Choose a reason for hiding this comment

cmsbuild commented Sep 2, 2021

cmsbuild commented Sep 2, 2021

fabferro commented Sep 3, 2021

cmsbuild commented Sep 3, 2021

cmsbuild commented Sep 3, 2021

jpata commented Sep 3, 2021

cmsbuild commented Sep 3, 2021

Comparison Summary

jpata commented Sep 3, 2021

cmsbuild commented Sep 3, 2021

perrotta commented Sep 4, 2021