[Boost] Update boost 1.80.0

[smuzaffar]

No description provided.

[smuzaffar]

please test

[cmsbuild]

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_12_6_X/master.

@smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/27506/summary.html
COMMIT: ffd32b5
CMSSW: CMSSW_12_6_X_2022-09-13-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8084/27506/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 8 differences found in the comparisons
• DQMHistoTests: Total files compared: 51
• DQMHistoTests: Total histograms compared: 3618326
• DQMHistoTests: Total failures: 8
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3618296
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
• Checked 212 log files, 49 edm output root files, 51 DQM output files
• TriggerResults: no differences found

[smuzaffar]

@ggovi @cms-sw/db-l2 any objections on updating boost to version 1.80?

[francescobrivio]

@smuzaffar is the target of the boost update CMSSW_12_6_X?

[smuzaffar]

yes @francescobrivio 12.6.X

[ggovi]

@smuzaffar need to run some test, will let you know asap

[smuzaffar]

test parameters:

• full_cmssw = true

[smuzaffar]

please test for CMSSW_12_6_CLANG_X

[tvami]

assign db

[cmsbuild]

New categories assigned: db

@ggovi,@francescobrivio,@malbouis,@saumyaphor4252,@tvami you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/27522/summary.html
COMMIT: ffd32b5
CMSSW: CMSSW_12_6_CLANG_X_2022-09-13-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8084/27522/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @missirol support XOR operator in triggerExpression::Parser cmssw#39377
• @mmusich Miscellaneous updates to TkHistoMap cmssw#39369
• @francescobrivio Update PPS DMD tag, remove deprecated SiStrip DMD tag cmssw#39380
• @civanch Fixed compilation in Geant4 IBs cmssw#39376
• @cecilecaillol [L1T] all modules swicthed to lower case and l1t cmssw#39244
• @cms-sw [XRootD] Update to version 5.5.0 #8081

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/27522/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/27522/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test test_TkHistoMap had ERRORS

Comparison Summary

Summary:

• You potentially added 90842 lines to the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 16795 differences found in the comparisons
• DQMHistoTests: Total files compared: 51
• DQMHistoTests: Total histograms compared: 3618326
• DQMHistoTests: Total failures: 95017
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3523286
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
• Checked 212 log files, 49 edm output root files, 51 DQM output files
• TriggerResults: found differences in 2 / 50 workflows

[tvami]

@smuzaffar is there a reason why this couldnt go into the first 13_X_Y release? I think it would be more natural to keep the 12_X_Y with the same boost. Thanks.

[smuzaffar]

@smuzaffar is there a reason why this couldnt go into the first 13_X_Y release? I think it would be more natural to keep the 12_X_Y with the same boost. Thanks.

no there is no specific reason. We can include it either in 13_X_Y or include it in 12.6.DEVEL to give you more time to test it. Do you have any specific reason to keep same boost for 12.X (we are going to update root for 12.X which might be a bigger jump as compare to boost update :-) )?

[francescobrivio]

@smuzaffar is there a reason why this couldnt go into the first 13_X_Y release? I think it would be more natural to keep the 12_X_Y with the same boost. Thanks.

no there is no specific reason. We can include it either in 13_X_Y and include it in 12.6.DEVEL to give you more time to test it. Do you have any specific reason to keep same boost for 12.X (we are going to update root for 12.X which might be a bigger jump as compare to boost update :-) )?

boost updates always require special handling, mainly because of the O2Os which might disrupt (read kill) data-taking.
Now, as far as I know, 12_6_X isn't meant for data-taking, but things can change depending on the LHC schedule, so I agree with @tvami that postponing this to 13_X_X would make our (alcadb) life much easier.

[fwyzard]

12.6.x aims to be out in December... I don't think there are any chances it can be used for data taking this year.

[tvami]

for AlCaDB the boost change is much more relevant than the root change. If the conditions are produced with a new boost that cannot be read by the older releases, so that's why I'd prefer to have a clear boundary at the 13_X_Y

[ggovi]

From my first investigation it looks like the transition 1.78 -> 1.80 does not break the serialisation. I mean, it seems that 1.78 CAN read payloads written in 1.80. However, I would prefer to make more tests with 'heavier' payloads.
BTW, running the condTestRegression.py on singularity with arch=el8_amd64_gcc10, I get the failure:
===== Test "condTestRegression" ====
FATAL: container creation failed: mount /etc/tnsnames.ora->/etc/tnsnames.ora error: while mounting /etc/tnsnames.ora: destination /etc/tnsnames.ora doesn't exist in container

init() took 0.650 ms

================================================================================

[fwyzard]

Makes sense... in fact, there are no differences in boost serialisation between 1.78.0 and 1.80.0: boostorg/serialization@boost-1.78.0...boost-1.80.0 .

[smuzaffar]

@ggovi , Looks like your system is missing /etc/tnsnames.ora , I would suggest to copy it from lxplus. OR use cmsdev nodes

[ggovi]

@smuzaffar I'm running on cmsdev23

[smuzaffar]

@ggovi , how do you start singularity? For me cmssw-el8 works

[muzaffar@cmsdev32 ~]$ cmssw-el8
Singularity> ls -l /etc/tnsnames.ora
-rw-r--r--. 1 root root 213978 Sep 16 06:11 /etc/tnsnames.ora

[smuzaffar]

please test

[smuzaffar]

please test for CMSSW_13_0_CLANG_X

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/29259/summary.html
COMMIT: ffd32b5
CMSSW: CMSSW_13_0_X_2022-11-25-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8084/29259/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @bsunanda Phase2-hgx330 Transfer most of the HGCal TB related geometry file to the new branch Geometry/HGCalTBCommonData cmssw#40122
• @smorovic (DAQ) File based protocol update cmssw#40099
• @wddgit Allow SwitchProducer cases to declare different transient products cmssw#40104
• @cms-tsg-storm Use LFN for EDM files of HLT-Validation and HLT-addon tests cmssw#40147
• @cms-sw SCRAM V2 disto via latest cmssw releases #8187
• @cms-sw Update tag for L1Trigger-TrackFindingTracklet to V00-03-00 #8199

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/29259/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/29259/git-merge-result

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 8 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3417239
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3417208
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 206 log files, 48 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[smuzaffar]

@tvami @ggovi , CMSSW_13_0_X is open now, any objections on integrating boost 1.80 in to it?

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/29260/summary.html
COMMIT: ffd32b5
CMSSW: CMSSW_13_0_CLANG_X_2022-11-24-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8084/29260/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @bsunanda Phase2-hgx330 Transfer most of the HGCal TB related geometry file to the new branch Geometry/HGCalTBCommonData cmssw#40122
• @smorovic (DAQ) File based protocol update cmssw#40099
• @wddgit Allow SwitchProducer cases to declare different transient products cmssw#40104
• @cms-tsg-storm Use LFN for EDM files of HLT-Validation and HLT-addon tests cmssw#40147
• @cms-sw SCRAM V2 disto via latest cmssw releases #8187
• @cms-sw Update tag for L1Trigger-TrackFindingTracklet to V00-03-00 #8199

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/29260/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6b84d4/29260/git-merge-result

Comparison Summary

Summary:

• You potentially added 78525 lines to the logs
• Reco comparison results: 22902 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3417239
• DQMHistoTests: Total failures: 44165
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3373051
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 206 log files, 48 edm output root files, 48 DQM output files
• TriggerResults: found differences in 2 / 46 workflows

[francescobrivio]

@tvami @ggovi , CMSSW_13_0_X is open now, any objections on integrating boost 1.80 in to it?

@smuzaffar given the update has been moved to 13_X and some preliminary checks from Giacomo are successful (#8084 (comment)) I think we don't have any objections.
But I do see that the tests report some unitTest failures for:
cms/8084/CLANG/el8_amd64_gcc10/unittest — Some unit tests were failed.
Unfortunately I can't find them in the logs...maybe they come from an older test trial?

[smuzaffar]

thanks @francescobrivio for quick reply.

Unfortunately I can't find them in the logs...maybe they come from an older test trial?

Yes those were old test, latest tests #8084 (comment) and #8084 (comment) (with full cmssw build) show that all unit tests passed.

[francescobrivio]

+db

• differences observed with full release build ([Boost] Update boost 1.80.0 #8084 (comment)) are not observed in "standard tests" ([Boost] Update boost 1.80.0 #8084 (comment))
• @ggovi since this will be merged in 13_0_X only this gives you a bit more time to perform more extensive tests on the new boost version if needed

[smuzaffar]

+externals

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_13_0_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)