-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DQM bin-by-bin comparisons broken in last PRs? #42050
Comments
A new Issue was created by @mmusich Marco Musich. @Dr15Jones, @perrotta, @dpiparo, @rappoccio, @makortel, @smuzaffar can you please review it and eventually sign/assign? Thanks. cms-bot commands are listed here |
assign dqm,core |
New categories assigned: dqm,core @tjavaid,@micsucmed,@nothingface0,@rvenditti,@Dr15Jones,@smuzaffar,@emanueleusai,@syuvivida,@makortel,@pmandrik you have been requested to review this Pull request/Issue and eventually sign? Thanks |
@cms-sw/dqm-l2 any update (or acknowledgement on this)? |
Hello, We’re looking into it. In the meantime the link to “comparisons failed” contains essentially the same plots and they do not appear empty, see e.g.: |
So, in CMSSW_13_2_X_2023-06-18-2300 the bin by bin comparison worked correctly, while in CMSSW_13_2_X_2023-06-20-1100 it did not (according to the issue description). |
Hmm that is a very good lead. On the other hand, last year we had a similar problem and it was caused by a corrupted database. We had to delete it and recreate it. |
This doesn't seem to be completely correct. while in the DQM bin-by-bin comparisons of the same PR I see more workflows with changes: perhaps something to be investigated in another issue? |
Yes the two comparison methods have always shown differences. It might have to do with the different thresholds the two methods use to define what’s different/failed, but is was never really investigated (not even by previous DQM people afaik). I usually check both and I found out that actual differences in dqm plots show up in both. I believe “Bin-to-bin” tends to be more noisy (or sensitive) and pick up stuff that is not a real discrepancy. |
Btw it looks like the indexing db is broken. Last time it was fixed by deleting and recreating the index file but we were strongly advised against doing that by SW experts, so we are looking for other solutions . |
Hello, |
OK, thanks I will close this issue for the time being. |
Title says it all, I wanted to check the outcome of the comparisons in a few PR-s but despite the counts of differences being non-zero they look empty.
Examples:
@rerecoZeroBias
DQM sequence #42031 (link)In other PRs it seems to work:
The text was updated successfully, but these errors were encountered: