Skip to content

Commit

Permalink
Update 7d-OOD-detection-distance.md
Browse files Browse the repository at this point in the history
  • Loading branch information
qualiaMachine authored Dec 19, 2024
1 parent 6064725 commit 71d5e10
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions episodes/7d-OOD-detection-distance.md
Original file line number Diff line number Diff line change
Expand Up @@ -382,9 +382,9 @@ plt.show()
After plotting the Mahalanobis distances for in-distribution (ID) and out-of-distribution (OOD) data, we may observe some overlap between the two distributions. This overlap reveals one of the limitations of distance-based methods: **the separability of ID and OOD data is highly dependent on the quality of the feature representations**. The model's learned features might not adequately distinguish between ID and OOD data, especially when OOD samples share semantic or structural similarities with ID data.

### A solution? Contrastive learning
In classical training regimes, models are trained with a *limited worldview*. They learn to distinguish between pre-defined classes based only on the data they’ve seen during training. You don't know what you don't know!
In classical training regimes, models are trained with a *limited worldview*. They learn to distinguish between pre-defined classes based only on the data they’ve seen during training, and simply don't know what they don't know.

For instance, consider a child learning to identify animals based on a set of flashcards with pictures of cats, dogs, and birds. If you show them a picture of a fox or a turtle, they might struggle because their understanding is constrained by the categories they’ve been explicitly taught. This is analogous to the way models trained with supervised learning approach classification—they build decision boundaries tailored to the training classes but struggle with new, unseen data.
An analogy: consider a child learning to identify animals based on a set of flashcards with pictures of cats, dogs, and birds. If you show them a picture of a fox or a turtle, they might struggle because their understanding is constrained by the categories they’ve been explicitly taught. This is analogous to the way models trained with supervised learning approach classification—they build decision boundaries tailored to the training classes but struggle with new, unseen data.

Now, consider teaching the child differently. Instead of focusing solely on identifying "cat" or "dog," you teach them to group animals by broader characteristics—like furry vs. scaly or walking vs. swimming. This approach helps the child form a more generalized understanding of the world, enabling them to recognize new animals by connecting them to familiar patterns. Contrastive learning aims to achieve something similar for machine learning models.

Expand Down

0 comments on commit 71d5e10

Please sign in to comment.