reduce verbosity of `reHLT` customisation in `SKIM` step #40599

missirol · 2023-01-24T09:45:13Z

PR description:

This PR is a follow-up of #40481, addressing #40481 (comment).

It reduces the verbosity of cmsDriver.py when the re-HLT customisation is applied to the SKIM step. In this case, it produces only a 1-line message, instead of 1 line for every affected Sequence of the SKIM step.

Merely technical. No changes expected, except for a reduction of printouts from cmsDriver.py.

PR validation:

Some basic manual checks with runTheMatrix.py and cmsDriver.py.

If this PR is a backport, please specify the original PR and why you need to backport that PR. If this PR will be backported, please specify to which release cycle the backport is meant for:

N/A

missirol · 2023-01-24T09:45:49Z

Configuration/Applications/python/ConfigBuilder.py

-        self.additionalCommands.append('process.%s.visit(ConfigBuilder.MassSearchReplaceProcessNameVisitor("%s", "%s", whitelist = ("subSystemFolder",)))'% (sequence,HLTprocess, proc))
-
+        self.additionalCommands.append(
+          'process.%s.visit(ConfigBuilder.MassSearchReplaceProcessNameVisitor("%s", "%s", whitelist = ("subSystemFolder",), verbose = %s))'% (sequence, HLTprocess, proc, verboseVisit))


This adds a bit of text to (some of) the cfgs produced by cmsDriver.py. Specifically, it adds a , verbose = False in the reHLT-related customisations inside the cfg.

cmsbuild · 2023-01-24T09:56:38Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40599/33855

This PR adds an extra 36KB to repository

cmsbuild · 2023-01-24T09:57:06Z

A new Pull Request was created by @missirol (Marino Missiroli) for master.

It involves the following packages:

Configuration/Applications (operations)

@cmsbuild, @perrotta, @rappoccio, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @fabiocos this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

missirol · 2023-01-24T21:56:13Z

please test

cmsbuild · 2023-01-25T00:19:25Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-208f86/30150/summary.html
COMMIT: b3800af
CMSSW: CMSSW_13_0_X_2023-01-24-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40599/30150/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 11 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555495
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3555464
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

perrotta · 2023-01-25T10:42:45Z

test parameters:

workflows = 4.17

perrotta · 2023-01-25T10:42:59Z

please test

missirol · 2023-01-25T11:39:33Z

hold

Still reviewing the outputs of the tests (specifically, the logs).

cmsbuild · 2023-01-25T11:39:57Z

Pull request has been put on hold by @missirol
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

cmsbuild · 2023-01-25T15:00:39Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-208f86/30161/summary.html
COMMIT: b3800af
CMSSW: CMSSW_13_0_X_2023-01-24-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40599/30161/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3613452
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3613425
DQMHistoTests: Total skipped: 24
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 216 log files, 166 edm output root files, 50 DQM output files
TriggerResults: no differences found

missirol · 2023-01-26T15:56:37Z

There is something I don't understand about the logs from runTheMatrix.py. Take wf 136.731, step 3:

I don't understand why 1. does not include printouts from cmsDriver.py like Step: RECO Spec:. It might be due to this PR, but I do not see how.

missirol · 2023-01-26T15:57:59Z

No significant changes to the logs found

@smuzaffar, I have a couple of questions. In this PR we expect a reduction in the logs of the matrix tests.

Does the bot only report increases in the logs, or also reductions? (here?)
In the outputs of PR tests (e.g. here), can one find some 'global' stats on the logs? (total number of lines, or similar)

cmsbuild · 2023-01-29T09:07:41Z

Pull request #40599 was updated. @perrotta, @rappoccio, @fabiocos, @davidlange6 can you please check and sign again.

cmsbuild · 2023-01-29T13:22:57Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-208f86/30231/summary.html
COMMIT: a2127b3
CMSSW: CMSSW_13_0_X_2023-01-28-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40599/30231/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 9 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3613452
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3613419
DQMHistoTests: Total skipped: 24
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 216 log files, 166 edm output root files, 50 DQM output files
TriggerResults: no differences found

missirol · 2023-01-29T13:31:00Z

The RelVals-INPUT error is unrelated (the tests passed in #40599 (comment), and there were really no changes since then).

No significant changes to the logs found

As noted in #40599 (comment), I'm not sure the bot reports reductions in the logs.

One can now see the reduction in the logs due to this PR, e.g. this vs baseline.

I don't have other updates planned for this PR.

perrotta · 2023-01-29T13:44:30Z

+1

cmsbuild · 2023-01-29T13:44:57Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing).

perrotta · 2023-01-29T13:46:54Z

merge

smuzaffar · 2023-01-29T16:20:44Z

Does the bot only report increases in the logs, or also reductions? (here?)

@missirol , yes bot was only reporting the increase in logs output but I now have updated it to report any reduction too. From the comparison tests log I see that 1286 log lines were removed.

Logs: -1286 False 216 299 0
Events: True 166 0.0 49 [] []
SUMMARY No significant changes to the logs found

In the outputs of PR tests (e.g. here), can one find some 'global' stats on the logs? (total number of lines, or similar)

you can find some information by going to Comparison with the baseline and then look into validateJR/logRootQA-events.log file

missirol · 2023-01-29T16:35:42Z

great, thanks @smuzaffar !

smuzaffar · 2023-02-01T07:56:24Z

There is something I don't understand about the logs from runTheMatrix.py. Take wf 136.731, step 3:

latest tests of this PR

baseline (from IB)

latest tests of another, unrelated, PR

I don't understand why 1. does not include printouts from cmsDriver.py like Step: RECO Spec:. It might be due to this PR, but I do not see how.

I was also looking in to it and now after reading this comment I think the problem is that bot might have set PYTHONUNBUFFERED only for IB tests. This explains why we see extra printouts in IB logs and not in PR logs. I will update bot to set PYTHONUNBUFFERED for PR too.

missirol · 2023-02-01T08:12:51Z

Thanks for having a look. Indeed, that's the only explaination I could think of, but I could not see where PYTHONUNBUFFERED is set for the IB tests. Out of curiosity, where is that done?

smuzaffar · 2023-02-01T08:34:26Z

all IB tests use https://github.com/cms-sw/cms-bot/blob/master/run-ib-testbase.sh to create dev area and this scripts sets export PYTHONUNBUFFERED=1

smuzaffar · 2023-02-01T08:35:04Z

by the way, bot pr cms-sw/cms-bot#1927 should fix this for PR tests too

missirol · 2023-02-01T08:39:44Z

Sounds good. Thanks for the info!

cmsbuild added this to the CMSSW_13_0_X milestone Jan 24, 2023

cmsbuild added code-checks-pending operations-pending orp-pending pending-signatures tests-pending labels Jan 24, 2023

missirol commented Jan 24, 2023

View reviewed changes

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 24, 2023

cmsbuild added tests-started and removed tests-pending labels Jan 24, 2023

cmsbuild added tests-approved and removed tests-started labels Jan 25, 2023

cmsbuild added tests-started and removed tests-approved labels Jan 25, 2023

cmsbuild added the hold label Jan 25, 2023

cmsbuild added tests-rejected and removed tests-started labels Jan 25, 2023

cmsbuild mentioned this pull request Jan 26, 2023

Add JeProfService to allow dumping of Jemalloc heap profiles during running. #40624

Merged

cmsbuild added tests-rejected and removed tests-started labels Jan 29, 2023

cmsbuild added fully-signed operations-approved orp-approved and removed operations-pending pending-signatures orp-pending labels Jan 29, 2023

cmsbuild merged commit 562a8e8 into cms-sw:master Jan 29, 2023

missirol deleted the devel_lessVerboseReHLTOfSkim branch January 29, 2023 15:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

reduce verbosity of `reHLT` customisation in `SKIM` step #40599

reduce verbosity of `reHLT` customisation in `SKIM` step #40599

missirol commented Jan 24, 2023

missirol Jan 24, 2023

cmsbuild commented Jan 24, 2023

cmsbuild commented Jan 24, 2023

missirol commented Jan 24, 2023

cmsbuild commented Jan 25, 2023

perrotta commented Jan 25, 2023

perrotta commented Jan 25, 2023

missirol commented Jan 25, 2023

cmsbuild commented Jan 25, 2023

cmsbuild commented Jan 25, 2023

missirol commented Jan 26, 2023

missirol commented Jan 26, 2023

cmsbuild commented Jan 29, 2023

cmsbuild commented Jan 29, 2023

missirol commented Jan 29, 2023 •

edited

Loading

perrotta commented Jan 29, 2023

cmsbuild commented Jan 29, 2023

perrotta commented Jan 29, 2023

smuzaffar commented Jan 29, 2023 •

edited

Loading

missirol commented Jan 29, 2023

smuzaffar commented Feb 1, 2023

missirol commented Feb 1, 2023

smuzaffar commented Feb 1, 2023

smuzaffar commented Feb 1, 2023

missirol commented Feb 1, 2023

reduce verbosity of reHLT customisation in SKIM step #40599

reduce verbosity of reHLT customisation in SKIM step #40599

Conversation

missirol commented Jan 24, 2023

PR description:

PR validation:

If this PR is a backport, please specify the original PR and why you need to backport that PR. If this PR will be backported, please specify to which release cycle the backport is meant for:

missirol Jan 24, 2023

Choose a reason for hiding this comment

cmsbuild commented Jan 24, 2023

cmsbuild commented Jan 24, 2023

missirol commented Jan 24, 2023

cmsbuild commented Jan 25, 2023

Comparison Summary

perrotta commented Jan 25, 2023

perrotta commented Jan 25, 2023

missirol commented Jan 25, 2023

cmsbuild commented Jan 25, 2023

cmsbuild commented Jan 25, 2023

RelVals-INPUT

Comparison Summary

missirol commented Jan 26, 2023

missirol commented Jan 26, 2023

cmsbuild commented Jan 29, 2023

cmsbuild commented Jan 29, 2023

RelVals-INPUT

Comparison Summary

missirol commented Jan 29, 2023 • edited Loading

perrotta commented Jan 29, 2023

cmsbuild commented Jan 29, 2023

perrotta commented Jan 29, 2023

smuzaffar commented Jan 29, 2023 • edited Loading

missirol commented Jan 29, 2023

smuzaffar commented Feb 1, 2023

missirol commented Feb 1, 2023

smuzaffar commented Feb 1, 2023

smuzaffar commented Feb 1, 2023

missirol commented Feb 1, 2023

reduce verbosity of `reHLT` customisation in `SKIM` step #40599

reduce verbosity of `reHLT` customisation in `SKIM` step #40599

missirol commented Jan 29, 2023 •

edited

Loading

smuzaffar commented Jan 29, 2023 •

edited

Loading