Skip to content

Commit

Permalink
[ML] Modify thresholds for normalization triggers (#33663)
Browse files Browse the repository at this point in the history
[ML] Modify thresholds for normalization triggers

The (arbitrary) threshold factors used to judge if scores have
changed significantly enough to trigger a look-back renormalization have
been changed to values that reduce the frequency of such
renormalizations.

Added a clause to treat changes in scores as a 'big change' if it would
result in a change of severity reported in the UI.

Also altered the clause affecting small scores so that a change should
be considered big if scores have changed by at least 1.5. 

Relates elastic/machine-learning-qa#263
  • Loading branch information
edsavage committed Sep 25, 2018
1 parent faf34c1 commit 2b43c8a
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -190,23 +190,34 @@ private double mergeRecursively(Iterator<NormalizerResult> scoresIter, Normaliza
* Encapsulate the logic for deciding whether a change to a normalized score
* is "big".
* <p>
* Current logic is that a big change is a change of at least 1 or more than
* than 50% of the higher of the two values.
* Current logic is that a change is considered big if any of the following criteria are met:
* <ul>
* <li>the change would result in a change of colour in the UI
* (e.g. severity would be changed from WARNING to MINOR)</li>
* <li>the change is at least 1.5</li>
* <li>the change in values is greater than 67% of the higher of the two values.</li>
* </ul>
* These values have been chosen through a process of experimentation, in particular it was desired to reduce
* the number of updates written to the results index due to renormalization events for performance reasons
* while not changing the normalized scores greatly
*
* @param oldVal The old value of the normalized score
* @param newVal The new value of the normalized score
* @return true if the update is considered "big"
*/
private static boolean isBigUpdate(double oldVal, double newVal) {
if (Math.abs(oldVal - newVal) >= 1.0) {
if ((int) (oldVal / 25.0) != (int) (newVal / 25.0)) {
return true;
}
if (Math.abs(oldVal - newVal) >= 1.5) {
return true;
}
if (oldVal > newVal) {
if (oldVal * 0.5 > newVal) {
if (oldVal * 0.33 > newVal) {
return true;
}
} else {
if (newVal * 0.5 > oldVal) {
if (newVal * 0.33 > oldVal) {
return true;
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ public class NormalizerTests extends ESTestCase {
private static final String INDEX_NAME = "foo-index";
private static final String QUANTILES_STATE = "someState";
private static final int BUCKET_SPAN = 600;
private static final double INITIAL_SCORE = 2.0;
private static final double INITIAL_SCORE = 3.0;
private static final double FACTOR = 2.0;

private Bucket generateBucket(Date timestamp) throws IOException {
Expand Down

0 comments on commit 2b43c8a

Please sign in to comment.