(13_0_X) streamer format padding backport #41186
Conversation
…ch could be there due to concenating two files with a header file new streamer unit test for padded files write out files with padding. Used for unit test checking that such files can be opened and read address comments improved unit tests Note: fixed conflict with the unit test due to improved merged into master Conflicts: IOPool/Streamer/test/RunSimple_NewStreamer.sh
Conflicts: DQMServices/StreamerIO/test/RunStreamer.sh
Conflicts: DQMServices/StreamerIO/test/RunStreamer.sh
Conflicts: CalibCalorimetry/EcalLaserSorting/test/RunStreamer.sh
|
A new Pull Request was created by @smorovic (Srecko Morovic) for CMSSW_13_0_X. It involves the following packages:
@smuzaffar, @Dr15Jones, @makortel, @pmandrik, @emanueleusai, @tvami, @cmsbuild, @saumyaphor4252, @syuvivida, @francescobrivio, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks. cms-bot commands are listed here
|
|
please test |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e7589d/31590/summary.html Comparison SummarySummary:
|
|
backport of #41125 |
|
@smorovic in the tests I see a failure in HLT/SiStrips for wf 11634.911. |
|
I don't think this is related. I'm not familiar how this test is implemented, but I'd be surprised if it used streamer files in any way. |
|
+1 |
|
+1 |
I opened an issue #41200 to document it (and have a reference if other PRs see it). I agree this PR can't really be causing the differences (streamer files are not part of the workflow in any way). |
|
+core |
cmsbuild
added
core-approved
fully-signed
and removed
core-pending
pending-signatures
labels
|
This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2) |
|
+1
|
PR description:
Adding support for padding in the Streamer format. Padding bytes (0xff) can be appended after the end of each event or INIT message. Intention is to add them at the end of the file (4096-byte blocks) to round the file size so it is compatible with O_DIRECT copy from DAQ RUBUs to Lustre system in hope this will reduce deadtime issues seen recently.
Streamer Input sources will automatically detect padding which will (typically) be appended to streams transferred to Tier-0. Also an option was added to generate files which contain padding after each event or INI object.
Update: DQM and EcalCalibration unit tests were extended with the padded format test as well.
PR validation:
Unit tests added to generate and read padded streamer file. Patch tested with 13_1_X (PR #41125) with production streamer files without padding and padded files produced in a test system.
If this PR is a backport please specify the original PR and why you need to backport that PR. =
Backport of #41125 for 13_0_X
Adding support for padded files for this release cycle so format can be used Online for the HLT output merging and parsed by Tier-0 and other streamer clients.