Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DQM module for PPS random stream - backport #41049

Merged
merged 10 commits into from
Mar 28, 2023

Conversation

AndreaBellora
Copy link
Contributor

This PR is a backport of #40895

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 14, 2023

A new Pull Request was created by @AndreaBellora for CMSSW_13_0_X.

It involves the following packages:

  • DQM/CTPPS (dqm)
  • DQM/Integration (dqm)

@emanueleusai, @cmsbuild, @syuvivida, @rvenditti, @micsucmed, @pmandrik can you please review it and eventually sign? Thanks.
@batinkov, @battibass, @fabferro, @grzanka, @threus, @francescobrivio this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@syuvivida
Copy link
Contributor

Hi, after building 13_0_0 with this PR at our playback, I was trying to test this client with the streamer provided by AlcaDB, but then found all the streamers at /eos/cms/store/t0streamer/Data/ALCAPPSExpress/000/ are from 2022 runs (created with 12_4_X or even older versions). In this case, we will not be able to read these streamers due to a mismatch of streamer format. @francescobrivio do we have this type of AlcaPPS streamers that are built with 13_0_X?

Also to double check @AndreaBellora did you test your module and client in 13_0_X using these streamers at eos?
Thanks!!

Eiko

@AndreaBellora
Copy link
Contributor Author

Hi @syuvivida, I tested the code as mentioned above, but I couldn't really run the client (I need some guidance to do it).
I think that we cannot have yet the streamers in the right format, the HLT has been approved, but it still needs to be put in a menu, and then we will be able to get a bit of data for testing.
I also add @vavati, as she's following the HLT part.

@vavati
Copy link

vavati commented Mar 15, 2023

@syuvivida : i made the request in https://its.cern.ch/jira/browse/CMSHLT-2650 but for the time being no feedback.
@trtomei please let us know when we could run and produce the correct streamer files.

@vavati
Copy link

vavati commented Mar 15, 2023

@syuvivida
Somebody here seems to suggest that there could be a way to convert streamers_12_X in streamers_13_X
https://cms-talk.web.cern.ch/t/cannot-read-streamer-files-from-cmssw-12-4-11/20212/6
I guess the problem is not only for this PR right? I cannot believe that all tests with "streamers" are blocked....

@syuvivida
Copy link
Contributor

@vavati When we switched from 12_4_X to 12_6_X for MWGR, at that time, HLT had to re-run 12_6_X at Hilton on an old 12_4_X collision streamer to make it work. On the other hand, there seems to be no streamer mismatch for 12_6_X vs 13_0_X; so far we are able to run 13_0_0 on the 12_6_3 MWGR cosmic run streamers.

@vavati
Copy link

vavati commented Mar 15, 2023 via email

@syuvivida
Copy link
Contributor

@vavati I meant to say there is a mismatch between 12_4_X streamers and newer release(either 12_6_X or 13_0_X), but not 12_6_X with 13_0_X.
If someone could reproduce these PPS streamers and provide us streamers either with 12_6_X or 13_0_X, we can test.

@vavati
Copy link

vavati commented Mar 16, 2023

@syuvivida I don't think it is up to us to provide a conversion of data streamer. The issue is not created by out PR but somewhere else. If data 2022 cannot be used to test then we wait for the new data.
As soon as HLT will provide the stream we will create the new file.

@emanueleusai
Copy link
Member

@vavati this really depends on how urgently you want this implemented. If this can wait until we get new 13_0 streamers then that's the easiest solution.

@emanueleusai
Copy link
Member

hold

  • waiting for 13_0 streamers to test at P5

@cmsbuild
Copy link
Contributor

Pull request has been put on hold by @emanueleusai
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

@cmsbuild cmsbuild added the hold label Mar 16, 2023
@emanueleusai
Copy link
Member

please test

@emanueleusai
Copy link
Member

type ctpps

@cmsbuild cmsbuild added the ctpps label Mar 16, 2023
@cmsbuild
Copy link
Contributor

Pull request #41049 was updated. @emanueleusai, @cmsbuild, @syuvivida, @rvenditti, @micsucmed, @pmandrik can you please check and sign again.

@AndreaBellora
Copy link
Contributor Author

@AndreaBellora The output of ppsrandom client at the playback when I ran this client with the other 26 clients is here: /afs/cern.ch/work/s/syu/public/pps_output/DQM_V0001_PPSRANDOM_R000519240.root But we want to remind you, this test is done with the "cosmic runs with PPS random triggers" 365045. A test with online DQM stream and a test with beam will provide more information and are still required.

Concerning this, I checked the output and everything looks right in the file

@vavati
Copy link

vavati commented Mar 23, 2023

@syuvivida Please it is up to DQM to organize the test. Today HLT said that the key can prepared to make the live test
At the moment the discussion seems to be here
https://its.cern.ch/jira/browse/CMSHLT-2650
There are too many threads with this issue and not all people are following.
Concerning the future tests with beam, well we need first beam. So keep it in DQM list and at the first occasion do the tests.

@syuvivida
Copy link
Contributor

@AndreaBellora DQM has made a production release. If HLT is ready to join, we could switch the production and include pps temporarily,
would you or someone from PPS connect to the CR zoom?

@cmsbuild
Copy link
Contributor

Pull request #41049 was updated. @emanueleusai, @cmsbuild, @syuvivida, @rvenditti, @micsucmed, @pmandrik can you please check and sign again.

@emanueleusai
Copy link
Member

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-279dea/31573/summary.html
COMMIT: a55d2c1
CMSSW: CMSSW_13_0_X_2023-03-23-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41049/31573/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 7 lines from the logs
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3552993
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3552971
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 213 log files, 164 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

@syuvivida
Copy link
Contributor

@AndreaBellora were you able to test the commit (adding colz, etc) you made yesterday using the streamers we copied to /eos/cms/store/group/comm_dqm/pps_input/?

Eiko

@AndreaBellora
Copy link
Contributor Author

@syuvivida yes, and things turned out as expected. On our side, no further change is needed, besides @francescobrivio suggestions, which will come in a separate PR

@emanueleusai
Copy link
Member

Very well then. I will complete the review soon so we can merge this.

@emanueleusai
Copy link
Member

+1

@emanueleusai
Copy link
Member

unhold

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 9d98e63 into cms-sw:CMSSW_13_0_X Mar 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants