add `commissioning_run` as `runType` and use it in SiStrip online DQM client [12.0.X] #35365

mmusich · 2021-09-22T07:20:25Z

PR description:

This PR adds a new DQM runType commissioning_run in order to define a new clause in the SiStrip online client to be used when there are no physics triggers.

PR validation:

At the moment none. to be further tested

if this PR is a backport please specify the original PR and why you need to backport that PR:

Original PR: #34832, to be tested in 2021 Commissioning data-taking

cmsbuild · 2021-09-22T07:20:46Z

A new Pull Request was created by @mmusich (Marco Musich) for CMSSW_12_0_X.

It involves the following packages:

DQM/Integration (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please review it and eventually sign? Thanks.
@battibass, @threus, @batinkov, @francescobrivio this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from [RFC] add commissioning_run as DQM runType and use it in SiStrip online DQM client #34832

mmusich · 2021-09-22T07:21:26Z

@cmsbuild, please test

cmsbuild · 2021-09-22T10:42:23Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f04dd6/18820/summary.html
COMMIT: da80a05
CMSSW: CMSSW_12_0_X_2021-09-21-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35365/18820/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2998536
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

qliphy · 2021-09-30T01:12:11Z

please test

cmsbuild · 2021-09-30T06:11:10Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f04dd6/19260/summary.html
COMMIT: da80a05
CMSSW: CMSSW_12_0_X_2021-09-29-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35365/19260/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 11
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2998530
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 38 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

jfernan2 · 2021-09-30T07:06:16Z

@qliphy we need to test this PR at P5, however for a full test, a Global Run of data taking where the DAQ shifter declares commissioning_run would be really desired, if I got it right

francescobrivio · 2021-09-30T12:18:44Z

@mmusich is this urgent for the beam test in October?
I believe this is currently the only missing PR in 12_0_X before building the new release (FYI @perrotta @qliphy )

mmusich · 2021-09-30T14:27:24Z

@francescobrivio it's not urgent for the beam test.
It's urgent for debugging some issues in the Strip APV emulation s/w that can be done at any time when the Strip is taking data during one of this "commissioning" runs.
At any rate the mechanism needs to be validated by requiring one of these runs at P5 and frankly I don't know where we stand with that...
@tsusa

francescobrivio · 2021-09-30T15:19:09Z

@francescobrivio it's not urgent for the beam test. It's urgent for debugging some issues in the Strip APV emulation s/w that can be done at any time when the Strip is taking data during one of this "commissioning" runs. At any rate the mechanism needs to be validated by requiring one of these runs at P5 and frankly I don't know where we stand with that... @tsusa

ok I understand! thanks for the explanation @mmusich!

tsusa · 2021-10-05T13:06:22Z

@francescobrivio it's not urgent for the beam test. It's urgent for debugging some issues in the Strip APV emulation s/w that can be done at any time when the Strip is taking data during one of this "commissioning" runs. At any rate the mechanism needs to be validated by requiring one of these runs at P5 and frankly I don't know where we stand with that... @tsusa

ok I understand! thanks for the explanation @mmusich!

@francescobrivio it's not urgent for the beam test. It's urgent for debugging some issues in the Strip APV emulation s/w that can be done at any time when the Strip is taking data during one of this "commissioning" runs. At any rate the mechanism needs to be validated by requiring one of these runs at P5 and frankly I don't know where we stand with that... @tsusa

the validation can be done when Strip detector is switched on (planned for Wednesday/Thursday this week)

tvami · 2021-10-05T13:45:21Z

@tsusa how was DQM supposed to validate this, is it ok to replay a cosmics run? @cms-sw/dqm-l2

mmusich · 2021-10-05T14:07:07Z

@tvami the test needs a special DAQ configuration in order to fall into the new run Type proposed here. That's why it can't be done with past data

tvami · 2021-10-05T14:23:45Z

ok, Tanja edited her message (#35365 (comment)), earlier it said "but we need the PR merged (and 12_0_3)".

ok so we wait until Wednesday/Thursday this week, validate and then merge? Is that the plan?

tvami · 2021-10-12T15:21:05Z

We are a week later, I wonder what the status for this is?

francescobrivio · 2021-10-14T10:02:03Z

FYI @mmusich @tsusa

From Tracker elog http://cmsonline.cern.ch/cms-elog/1124978:

Test ongoing for the new DQM setting "comissioning_run"
Tracker in global with run key set to DEFAULT mode (ZS) and with 400Hz random trigger rate.
Run# is 345678"

mmusich · 2021-10-14T11:59:08Z

Thanks @ahmad3213 your information appears to be correct, while the one reported above from @francescobrivio seems not.

francescobrivio · 2021-10-14T12:00:15Z

Thanks @ahmad3213 your information appears to be correct, while the one reported above from @francescobrivio seems not.

Yea you are right, I just copy/pasted the Tracker elog message assuming it was correct, but I did not double checked. Sorry about that!

…oduced in cms-sw#34832

cmsbuild · 2021-10-15T14:20:36Z

Pull request #35365 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please check and sign again.

mmusich · 2021-10-15T14:22:57Z

in the last commit I provide a reconstruction scenario for the new run_type (in this case cosmics reconstruction) for the visualization clients.
The change is also proposed in master at #35691

DQM/Integration/python/clients/visualization-live_cfg.py

mmusich · 2021-10-16T14:36:19Z

@cmsbuild please test

cmsbuild · 2021-10-16T17:53:05Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f04dd6/19689/summary.html
COMMIT: da18851
CMSSW: CMSSW_12_0_X_2021-10-16-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35365/19689/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2998536
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

jfernan2 · 2021-10-18T13:33:55Z

+1
Tested in Online DQM at P5

cmsbuild · 2021-10-18T13:34:16Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-10-19T00:55:27Z

+1

cmsbuild added this to the CMSSW_12_0_X milestone Sep 22, 2021

cmsbuild added backport-ok dqm-pending orp-pending pending-signatures tests-pending labels Sep 22, 2021

mmusich mentioned this pull request Sep 22, 2021

add commissioning_run as runType and use it in SiStrip online DQM client [11.3.X] #34838

Closed

cmsbuild added tests-started and removed tests-pending labels Sep 22, 2021

cmsbuild added tests-approved and removed tests-started labels Sep 22, 2021

cmsbuild added tests-started and removed tests-approved labels Sep 30, 2021

cmsbuild added tests-approved and removed tests-started labels Sep 30, 2021

mmusich added 2 commits October 15, 2021 16:17

add commissioning_run as runType and use it in SiStrip online DQM client

8cfd41b

provide a reconstruction scenario for the commissioning_run type intr…

da18851

…oduced in cms-sw#34832

mmusich force-pushed the sistripCommmissioningRunsDQM_12_0_X branch from da80a05 to da18851 Compare October 15, 2021 14:20

cmsbuild added tests-pending and removed tests-approved labels Oct 15, 2021

cmsbuild removed the backport-ok label Oct 15, 2021

mmusich mentioned this pull request Oct 15, 2021

add ED clients commissioning run type & fix def SPLASH settings #35688

Merged

francescobrivio reviewed Oct 15, 2021

View reviewed changes

DQM/Integration/python/clients/visualization-live_cfg.py Show resolved Hide resolved

cmsbuild added tests-started and removed tests-pending labels Oct 16, 2021

cmsbuild added tests-approved and removed tests-started labels Oct 16, 2021

jfernan2 mentioned this pull request Oct 18, 2021

provide a reconstruction scenario for the commissioning_run type introduced in #34832 #35691

Merged

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Oct 18, 2021

cmsbuild added orp-approved and removed orp-pending labels Oct 19, 2021

cmsbuild merged commit d7de0f6 into cms-sw:CMSSW_12_0_X Oct 19, 2021

mmusich deleted the sistripCommmissioningRunsDQM_12_0_X branch October 19, 2021 06:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add `commissioning_run` as `runType` and use it in SiStrip online DQM client [12.0.X] #35365

add `commissioning_run` as `runType` and use it in SiStrip online DQM client [12.0.X] #35365

mmusich commented Sep 22, 2021 •

edited

Loading

cmsbuild commented Sep 22, 2021 •

edited

Loading

mmusich commented Sep 22, 2021

cmsbuild commented Sep 22, 2021

qliphy commented Sep 30, 2021

cmsbuild commented Sep 30, 2021

jfernan2 commented Sep 30, 2021

francescobrivio commented Sep 30, 2021

mmusich commented Sep 30, 2021 •

edited

Loading

francescobrivio commented Sep 30, 2021

tsusa commented Oct 5, 2021 •

edited

Loading

tvami commented Oct 5, 2021

mmusich commented Oct 5, 2021 •

edited

Loading

tvami commented Oct 5, 2021 •

edited

Loading

tvami commented Oct 12, 2021

francescobrivio commented Oct 14, 2021

mmusich commented Oct 14, 2021 •

edited

Loading

francescobrivio commented Oct 14, 2021 •

edited

Loading

cmsbuild commented Oct 15, 2021

mmusich commented Oct 15, 2021 •

edited

Loading

mmusich commented Oct 16, 2021

cmsbuild commented Oct 16, 2021

jfernan2 commented Oct 18, 2021

cmsbuild commented Oct 18, 2021

qliphy commented Oct 19, 2021

add commissioning_run as runType and use it in SiStrip online DQM client [12.0.X] #35365

add commissioning_run as runType and use it in SiStrip online DQM client [12.0.X] #35365

Conversation

mmusich commented Sep 22, 2021 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Sep 22, 2021 • edited Loading

mmusich commented Sep 22, 2021

cmsbuild commented Sep 22, 2021

Comparison Summary

qliphy commented Sep 30, 2021

cmsbuild commented Sep 30, 2021

Comparison Summary

jfernan2 commented Sep 30, 2021

francescobrivio commented Sep 30, 2021

mmusich commented Sep 30, 2021 • edited Loading

francescobrivio commented Sep 30, 2021

tsusa commented Oct 5, 2021 • edited Loading

tvami commented Oct 5, 2021

mmusich commented Oct 5, 2021 • edited Loading

tvami commented Oct 5, 2021 • edited Loading

tvami commented Oct 12, 2021

francescobrivio commented Oct 14, 2021

mmusich commented Oct 14, 2021 • edited Loading

francescobrivio commented Oct 14, 2021 • edited Loading

cmsbuild commented Oct 15, 2021

mmusich commented Oct 15, 2021 • edited Loading

mmusich commented Oct 16, 2021

cmsbuild commented Oct 16, 2021

Comparison Summary

jfernan2 commented Oct 18, 2021

cmsbuild commented Oct 18, 2021

qliphy commented Oct 19, 2021

add `commissioning_run` as `runType` and use it in SiStrip online DQM client [12.0.X] #35365

add `commissioning_run` as `runType` and use it in SiStrip online DQM client [12.0.X] #35365

mmusich commented Sep 22, 2021 •

edited

Loading

cmsbuild commented Sep 22, 2021 •

edited

Loading

mmusich commented Sep 30, 2021 •

edited

Loading

tsusa commented Oct 5, 2021 •

edited

Loading

mmusich commented Oct 5, 2021 •

edited

Loading

tvami commented Oct 5, 2021 •

edited

Loading

mmusich commented Oct 14, 2021 •

edited

Loading

francescobrivio commented Oct 14, 2021 •

edited

Loading

mmusich commented Oct 15, 2021 •

edited

Loading