Fix testCSCTriggerMapping test: avoid reading invalid data e.g empty line #43269

smuzaffar · 2023-11-14T15:06:01Z

The empty or invalid lines in the mapping files were causing unit test to fail for NON-LTO build. This test was reading the following data

L30:      1       3       5       0       9       1       3       5       0       9
L31:# ================================================================================
L32:
L33:#Example of a more elaborate trigger schemes

As there was no check for empty lines (or invalid format lines) the test continued to process the L32 and has reused the i1, i2, i3, i4, i5, i6, i7, i8, i9 read from the line L30 and set i10=0 . With some debug statements, the output from the tests was

LINE:      1       3       5       0       9       1       3       5       0       9
i1-10: 1 3 5 0 9 1 3 5 0 9
LINE: # ================================================================================
LINE:
i1-10: 1 3 5 0 9 1 3 5 0 0

Local tests shows that unit test works in NONLTO IBs with this fix

cmsbuild · 2023-11-14T15:11:51Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43269/37670

This PR adds an extra 12KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43269/37670/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43269/37670/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

…line

cmsbuild · 2023-11-14T15:21:13Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43269/37672

This PR adds an extra 12KB to repository

cmsbuild · 2023-11-14T15:21:36Z

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for master.

It involves the following packages:

CondFormats/CSCObjects (db, alca)

@consuegs, @cmsbuild, @perrotta, @francescobrivio, @saumyaphor4252 can you please review it and eventually sign? Thanks.
@ptcox, @rsreds, @giovanni-mocellin, @seemasharmafnal, @missirol, @yuanchao, @tocheng, @PonIlya, @mmusich this is something you requested to watch as well.
@antoniovilela, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

smuzaffar · 2023-11-14T15:22:23Z

please test

smuzaffar · 2023-11-14T15:22:52Z

please test for CMSSW_14_0_NONLTO_X

smuzaffar · 2023-11-14T15:30:50Z

for default (LTO based IBs) this test shows the following output

LINE:      1       3       5       0       9       1       3       5       0       9
i1-10:1 3 5 0 9 1 3 5 0 9
LINE:# ================================================================================
LINE:
i1-10:1 3 5 0 9 1 3 5 0 9
CSCTriggerMapping: ERROR, already have key = 604026112

So empty line is still getting processed but is >> i1 >> i2 >> i3 >> i4 >> i5 >> i6 >> i7 >> i8 >> i9 >> i10; was not overriding the values of i1-i10 variables that is why test was running for default IBs

smuzaffar · 2023-11-14T16:08:52Z

CondFormats/CSCObjects/src/CSCTriggerMappingFromFile.cc

@@ -22,7 +22,8 @@ void CSCTriggerMappingFromFile::fill(void) {
      if (line[0] != commentFlag[0]) {
        int i1, i2, i3, i4, i5, i6, i7, i8, i9, i10;
        std::istringstream is(line);
-        is >> i1 >> i2 >> i3 >> i4 >> i5 >> i6 >> i7 >> i8 >> i9 >> i10;
+        if (!(is >> i1 >> i2 >> i3 >> i4 >> i5 >> i6 >> i7 >> i8 >> i9 >> i10))
+          continue;


@cms-sw/alca-l2 , should we throw an exception/LogError instead of silently ignoring the bad input?

ptcox · 2023-11-14T16:42:51Z

Thanks, Shahzad. The simple trap is fine. No need to do any sophisticated error trapping. Those tests are of historical interest only (I think).

perrotta · 2023-11-14T17:40:11Z

Those tests are of historical interest only (I think).

Could you please elaborate? If really those tests are not needed, then I would simply remove them, and save time to maintainers

cmsbuild · 2023-11-14T17:54:33Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-01d980/35803/summary.html
COMMIT: c6fb83c
CMSSW: CMSSW_14_0_X_2023-11-14-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43269/35803/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 115 lines from the logs
Reco comparison results: 5 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3363028
DQMHistoTests: Total failures: 4
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3363002
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

ptcox · 2023-11-14T18:02:17Z

Andrea, I know of nobody having made use of those tests in at least 10 years. Mainly of course because code stabilized and is very rarely changed. With luck, over the next year, we'll find somebody to go through it all, and rationalize and otherwise clean it up.

cmsbuild · 2023-11-14T18:10:49Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-01d980/35804/summary.html
COMMIT: c6fb83c
CMSSW: CMSSW_14_0_NONLTO_X_2023-11-13-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43269/35804/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 222 lines from the logs
Reco comparison results: 1273 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3363028
DQMHistoTests: Total failures: 17134
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3345872
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

perrotta · 2023-11-15T09:31:27Z

+1

Minimal fix to avoid unit test to fail in NON-LTO build

cmsbuild · 2023-11-15T09:31:52Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

smuzaffar · 2023-11-17T11:09:56Z

@cms-sw/orp-l2 , can you please review/sign this?

rappoccio · 2023-11-17T14:43:32Z

+1

cmsbuild added this to the CMSSW_14_0_X milestone Nov 14, 2023

cmsbuild added alca-pending db-pending pending-signatures tests-pending orp-pending code-checks-pending labels Nov 14, 2023

smuzaffar mentioned this pull request Nov 14, 2023

Exception of category CSCTriggerNumbering::InvalidInput in unit test testCSCTriggerMapping #43031

Closed

cmsbuild added code-checks-rejected and removed code-checks-pending labels Nov 14, 2023

Fix testCSCTriggerMapping test: avoid reading invalid data e.g empty …

c6fb83c

…line

smuzaffar force-pushed the fix-testCSCTriggerMapping branch from a091c7d to c6fb83c Compare November 14, 2023 15:16

cmsbuild added code-checks-pending and removed code-checks-rejected labels Nov 14, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 14, 2023

cmsbuild added tests-started and removed tests-pending labels Nov 14, 2023

smuzaffar commented Nov 14, 2023

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Nov 14, 2023

cmsbuild added alca-approved fully-signed db-approved and removed alca-pending db-pending pending-signatures labels Nov 15, 2023

cmsbuild added orp-approved and removed orp-pending labels Nov 17, 2023

cmsbuild merged commit dadc634 into cms-sw:master Nov 17, 2023

cmsbuild mentioned this pull request Nov 17, 2023

add verbose option to BeamSpotRcdPrinter and use it in the unit tests #43291

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix testCSCTriggerMapping test: avoid reading invalid data e.g empty line #43269

Fix testCSCTriggerMapping test: avoid reading invalid data e.g empty line #43269

smuzaffar commented Nov 14, 2023 •

edited

Loading

cmsbuild commented Nov 14, 2023

cmsbuild commented Nov 14, 2023

cmsbuild commented Nov 14, 2023

smuzaffar commented Nov 14, 2023

smuzaffar commented Nov 14, 2023

smuzaffar commented Nov 14, 2023

smuzaffar Nov 14, 2023 •

edited

Loading

ptcox commented Nov 14, 2023

perrotta commented Nov 14, 2023

cmsbuild commented Nov 14, 2023

ptcox commented Nov 14, 2023

cmsbuild commented Nov 14, 2023

perrotta commented Nov 15, 2023

cmsbuild commented Nov 15, 2023

smuzaffar commented Nov 17, 2023

rappoccio commented Nov 17, 2023

Fix testCSCTriggerMapping test: avoid reading invalid data e.g empty line #43269

Fix testCSCTriggerMapping test: avoid reading invalid data e.g empty line #43269

Conversation

smuzaffar commented Nov 14, 2023 • edited Loading

cmsbuild commented Nov 14, 2023

cmsbuild commented Nov 14, 2023

cmsbuild commented Nov 14, 2023

smuzaffar commented Nov 14, 2023

smuzaffar commented Nov 14, 2023

smuzaffar commented Nov 14, 2023

smuzaffar Nov 14, 2023 • edited Loading

Choose a reason for hiding this comment

ptcox commented Nov 14, 2023

perrotta commented Nov 14, 2023

cmsbuild commented Nov 14, 2023

Comparison Summary

ptcox commented Nov 14, 2023

cmsbuild commented Nov 14, 2023

Comparison Summary

perrotta commented Nov 15, 2023

cmsbuild commented Nov 15, 2023

smuzaffar commented Nov 17, 2023

rappoccio commented Nov 17, 2023

smuzaffar commented Nov 14, 2023 •

edited

Loading

smuzaffar Nov 14, 2023 •

edited

Loading