Deal with invalid blocks in WMBS for Dataset start policy

[amaltaro]

Fixes #11822

Status

not-tested

Description

This PR fixes work acquisition from Local WorkQueue into WMBS in the case of an invalid block (a block with only invalid files). This fix applies to workflows with Dataset starting policy, i.e., workflows with a single workqueue element where the actual input is a dataset (instead of the usual block).

In addition, a performance improvement is provided to the listFilesInBlock DBSReader wrapper API, such that we don't try to resolve the lumis in a block with only invalid files.

Is it backward compatible (if not, which system it affects?)

YES

Related PRs

None

External dependencies / deployment changes

None

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 1 new failures
  • 1 tests no longer failing
  • 1 changes in unstable tests
• Python3 Pylint check: failed
  • 2 warnings and errors that must be fixed
  • 10 warnings
  • 147 comments to review
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded
  • 6 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14744/artifact/artifacts/PullRequestReport.html

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 1 tests no longer failing
  • 1 changes in unstable tests
• Python3 Pylint check: failed
  • 2 warnings and errors that must be fixed
  • 10 warnings
  • 147 comments to review
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded
  • 6 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14746/artifact/artifacts/PullRequestReport.html

[amaltaro]

After some checks, I applied this patch to submit7 and I confirm it works as expected.

I also think that we should patch all the other agents with this PR to ensure that there are no such workflows stuck in the agents.