From b7d6cecce87141761c532acfccf7a72152402e67 Mon Sep 17 00:00:00 2001 From: Expertium <83031600+Expertium@users.noreply.github.com> Date: Fri, 13 Dec 2024 19:22:04 +0300 Subject: [PATCH] Update Benchmark.md --- Benchmark.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Benchmark.md b/Benchmark.md index 8e93645..bd03238 100644 --- a/Benchmark.md +++ b/Benchmark.md @@ -48,7 +48,7 @@ Next is AUC (Area Under the Curve). Unlike the previous two metrics, AUC is not 1. AUC measures how well an algorithm can tell classes apart; in our case, classes are "recalled" and "forgotten." You can think of AUC as a measure of how well the algorithm can draw a boundary between two classes, such that all members of class 1 are on one side of the boundary and all members of class 2 are on the other side. 2. AUC ranges from 0 to 1, but in practice it's almost always greater than 0.5. AUC less than 0.5 indicates that the algorithm is performing worse than random. Higher is better. -AUC can be rather unintuitive in some cases. Exaggerated example: suppose you have an algorithm that always outputs a 99% probability of having cancer for people who do have cancer and a 98% probability of having cancer for people who do not have cancer. It never outputs 98% for those who do have cancer, and it never outputs 99% for those who do. What do you think is the AUC of this algorithm? Answer: 1.0, because it can perfectly distinguish between these two classes, even if the calibration is absolutely terrible. AUC doesn't tell us anything about calibration, only about discrimination. +AUC can be rather unintuitive in some cases. Exaggerated example: suppose you have an algorithm that always outputs a 99% probability of having cancer for people who do have cancer and a 98% probability of having cancer for people who do not have cancer. It never outputs 98% for those who do have cancer, and it never outputs 99% for those who don't. What do you think is the AUC of this algorithm? Answer: 1.0, because it can perfectly distinguish between these two classes, even if the calibration is absolutely terrible. AUC doesn't tell us anything about calibration, only about discrimination. Below is a diagram that explains AUC.