Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(DAQ) increase monitoring limit on number of streams (13_2_X) #42902

Merged
merged 1 commit into from
Sep 29, 2023
Merged

(DAQ) increase monitoring limit on number of streams (13_2_X) #42902

merged 1 commit into from
Sep 29, 2023

Conversation

smorovic
Copy link
Contributor

PR description:

Increases the number of placeholders in the monitoring data reserved mostly for output modules (therefore HLT streams).
With more streams planning to be used in heavy-ion, 64 is no longer sufficient.

PR validation:

Tested in a DAQ test setup with full chain HLT and monitoring.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #42901

@cmsbuild cmsbuild added this to the CMSSW_13_2_X milestone Sep 28, 2023
@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 28, 2023 •
edited
Loading

A new Pull Request was created by @smorovic (Srecko Morovic) for CMSSW_13_2_X.

It involves the following packages:

  • EventFilter/Utilities (daq)

@smorovic, @emeschi, @cmsbuild can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@smorovic
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4f8cb4/34964/summary.html
COMMIT: 7011c91
CMSSW: CMSSW_13_2_X_2023-09-28-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42902/34964/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4f8cb4/34964/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4f8cb4/34964/git-merge-result

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 5 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3198243
  • DQMHistoTests: Total failures: 4
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3198217
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 207 log files, 159 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

@smorovic
Copy link
Contributor Author

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

@missirol
Copy link
Contributor

@cms-sw/orp-l2 , could this simple PR be included in the upcoming 13_2_X release ? It's relevant to HLT-online operations.

@mandrenguyen
Copy link
Contributor

+1 from HI for what it's worth

@antoniovilela
Copy link
Contributor

@cms-sw/orp-l2 , could this simple PR be included in the upcoming 13_2_X release ? It's relevant to HLT-online operations.

How critical is this? We just started the new build, before which we explicitly asked what else was urgently needed.

@missirol
Copy link
Contributor

How critical is this?

@smorovic can explain in detail how this relates to the monitoring of the HLT streams used online (I do not know). The update is relevant because we might deploy online an HLT menu with more than 64 streams in the very next days. If this update will not be in 13_2_5, HLT might ask for a patch release as soon as tomorrow.

We just started the new build, before which we explicitly asked what else was urgently needed.

I don't know where this was discussed. We simply communicated this to the ORM as soon as we could, to avoid asking another release right after 13_2_5 for such a simple update.

@antoniovilela
Copy link
Contributor

How critical is this?

@smorovic can explain in detail how this relates to the monitoring of the HLT streams used online (I do not know). The update is relevant because we might deploy online an HLT menu with more than 64 streams in the very next days. If this update will not be in 13_2_5, HLT might ask for a patch release as soon as tomorrow.

We just started the new build, before which we explicitly asked what else was urgently needed.

I don't know where this was discussed. We simply communicated this to the ORM as soon as we could, to avoid asking another release right after 13_2_5 for such a simple update.

We discussed with ORM.

Let's make a patch with this in tomorrow.

@missirol
Copy link
Contributor

Let's make a patch with this in tomorrow.

Sounds good, thanks !

@mandrenguyen
Copy link
Contributor

urgent

@rappoccio
Copy link
Contributor

urgent

  • Urgently requested for datataking.

@perrotta
Copy link
Contributor

backport of #42901

@antoniovilela
Copy link
Contributor

@smorovic
Could you quickly check why the unit test failed in the master IB:
https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc11/CMSSW_13_3_X_2023-09-28-2300/unitTestLogs/EventFilter/Utilities#/

@smorovic
Copy link
Contributor Author

@antoniovilela
On a first look it doesn't look related, segfault is in a call to read a parameterSet parameter.

#3  0x0000151905ebb34b in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02804/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_X_2023-09-28-2300/lib/el8_amd64_gcc11/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  0x000015190a35e5da in __memcmp_avx2_movbe () from /lib64/libc.so.6
#6  0x000015190cf65940 in std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, edm::Entry>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, edm::Entry> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, edm::Entry> > >::find(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02804/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_X_2023-09-28-2300/lib/el8_amd64_gcc11/libFWCoreParameterSet.so
#7  0x000015190cf5d28e in edm::ParameterSet::retrieveUntracked(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02804/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_X_2023-09-28-2300/lib/el8_amd64_gcc11/libFWCoreParameterSet.so
#8  0x000015190cf5d2d5 in edm::ParameterSet::getEntryPointerOrThrow_(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02804/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_X_2023-09-28-2300/lib/el8_amd64_gcc11/libFWCoreParameterSet.so
#9  0x000015190cf5d339 in edm::ParameterSet::getEntryPointerOrThrow_(char const*) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02804/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_X_2023-09-28-2300/lib/el8_amd64_gcc11/libFWCoreParameterSet.so
#10 0x000015190cf67379 in bool edm::ParameterSet::getUntrackedParameter<bool>(char const*) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02804/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_X_2023-09-28-2300/lib/el8_amd64_gcc11/libFWCoreParameterSet.so
#11 0x000015190534de1d in edm::StreamerOutputModuleCommon::StreamerOutputModuleCommon(edm::ParameterSet const&, std::vector<std::pair<edm::BranchDescription const*, edm::EDGetToken>, std::allocator<std::pair<edm::BranchDescription const*, edm::EDGetToken> > > const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02804/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_X_2023-09-28-2300/lib/el8_amd64_gcc11/libIOPoolStreamer.so
#12 0x000015190563c76c in evf::GlobalEvFOutputModule::globalBeginRun(edm::RunForOutput const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02804/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_X_2023-09-28-2300/lib/el8_amd64_gcc11/pluginEventFilterUtilitiesPlugins.so

@smorovic smorovic mentioned this pull request Sep 29, 2023
Closed
@antoniovilela
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit a326461 into cms-sw:CMSSW_13_2_X Sep 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
backport-ok daq-approved fully-signed orp-approved tests-approved urgent
Projects
None yet
Milestone
CMSSW_13_2_X
Development

Successfully merging this pull request may close these issues.
7 participants
@smorovic @cmsbuild @missirol @mandrenguyen @antoniovilela @rappoccio @perrotta