testGetPayloadData (getPayloadData.py) unit failure

[qliphy]

There is a unit failure In CMSSW_12_0_X_2021-07-15-2300
https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc900/CMSSW_12_0_X_2021-07-15-2300/unitTestLogs/CondCore/Utilities#/181-181
Traceback (most recent call last):
File "", line 1, in
KeyError: 'cond::BasicPayload'
getPayloadData.py --discover test not passed... found no entries
---> test testGetPayloadData had ERRORS

This can also be reproduced locally:
cmsrel CMSSW_12_0_X_2021-07-15-2300
cd CMSSW_12_0_X_2021-07-15-2300/src
cmsenv
git cms-addpkg CondCore/Utilities
cd CondCore/Utilities/test/
source test_getPayloadData.sh

[cmsbuild]

A new Issue was created by @qliphy Qiang Li.

@Dr15Jones, @perrotta, @dpiparo, @silviodonato, @smuzaffar, @makortel, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[qliphy]

assign db

[cmsbuild]

New categories assigned: db

@ggovi you have been requested to review this Pull request/Issue and eventually sign? Thanks

[qliphy]

@mmusich Please have a look. Thanks!

[mmusich]

@qliphy just about the second part of your report, that's not correct

The issue was not there in previous IB. However, there are other errors seem not to be reported in IB:
(I have tested both CMSSW_12_0_X_2021-07-15-1100 and CMSSW_12_0_X_2021-07-14-1100 )
---> Passed test of 'getPayloadData.py --plugin pluginBasicPayload_PayloadInspector --plot plot_BasicPayload_data0_withInput --tag BasicPayload_v0 --input_params {"Factor":"1","Offset":"2","Scale":"3"} --time_type Run --iovs {"start_iov": "1", "end_iov": "101"} --db Prod --test'
Traceback (most recent call last):
File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-14-1100/bin/slc7_amd64_gcc900/getPayloadData.py", line 278, in
result = deserialize_iovs(args.db, args.plugin, args.plot, tags, args.time_type, input_params)
File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-14-1100/bin/slc7_amd64_gcc900/getPayloadData.py", line 48, in decorated
result = f( *fargs, **fkwargs )
File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-14-1100/bin/slc7_amd64_gcc900/getPayloadData.py", line 106, in deserialize_iovs
success = plot.process(db_name, tags)
RuntimeError: An exception of category 'ConditionDatabase' occurred.
Exception Message:
Tag "BasicPayload" has not been found in the database. from IOVProxy::load

This is actually expected, because the test is checking for failures when being passed wrong input:

cmssw/CondCore/Utilities/test/test_getPayloadData.sh

Lines 63 to 73 in 11928c0

	########################################
	# Test BasicPayload with wrong inputs
	########################################
	check_for_failure getPayloadData.py \
	--plugin pluginBasicPayload_PayloadInspector \
	--plot plot_BasicPayload_data0 \
	--tag BasicPayload \
	--time_type Run \
	--iovs '{"start_iov": "1", "end_iov": "101"}' \
	--db Prod \
	--test ;

Please remove that part from the report, because it's very confusing!

[qliphy]

@mmusich ok, thanks. I will remove that part.

[mmusich]

About the issue in the last IB, I think I understand the cause.
In general, if there are no changes in any of the plugins to be "discovered" (as it has been the case until now) the script will look for all the plugins available, see e.g. here:

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc900/CMSSW_12_0_X_2021-07-15-1100/unitTestLogs/CondCore/Utilities#/172.

But specifically in CMSSW_12_0_X_2021-07-15-2300 two PRs changing plugins in CondCore/SiPixelPlugins have been merged (#34375 and #34457), and as a consequence the script only discovers the plugins in that folder, see:

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc900/CMSSW_12_0_X_2021-07-15-2300/unitTestLogs/CondCore/Utilities#/172

as a result the specific plugin used for the test is missing and the test fails.
I suspect that the test will restart to work at the next IB, but again every time someone updates the plugins it will fail again.
I think this is a by-product of the behavior introduced in #34257.
I'll try to think how to avoid that, but suggestions are welcome.

[mmusich]

see #34515