```
For instance, to train a [PatchCore](https://github.com/openvinotoolkit/anomalib/tree/development/anomalib/models/patchcore) model, the following command would be run:
+
```bash
anomalib fit --config ./configs/model/patchcore.yaml
```
The new CLI approach offers a lot more flexibility, details of which are explained in the [documentation](https://pytorch-lightning.readthedocs.io/en/stable/common/lightning_cli.html).
-## Inference
-### β οΈ Anomalib < v.0.4.0
+# Inference
+
+## β οΈ Anomalib < v.0.4.0
+
Anomalib includes multiple tools, including Lightning, Gradio, and OpenVINO inferencers, for performing inference with a trained model.
The following command can be used to run PyTorch Lightning inference from the command line:
@@ -205,7 +216,7 @@ python tools/inference/gradio_inference.py \
--weights ./results/padim/mvtec/bottle/weights/model.ckpt
```
-## Hyperparameter Optimization
+# Hyperparameter Optimization
To run hyperparameter optimization, use the following command:
@@ -217,7 +228,7 @@ python tools/hpo/sweep.py \
For more details refer the [HPO Documentation](https://openvinotoolkit.github.io/anomalib/guides/hyperparameter_optimization.html)
-## Benchmarking
+# Benchmarking
To gather benchmarking data such as throughput across categories, use the following command:
@@ -228,7 +239,7 @@ python tools/benchmarking/benchmark.py \
Refer to the [Benchmarking Documentation](https://openvinotoolkit.github.io/anomalib/guides/benchmarking.html) for more details.
-## Logging Images
+# Logging Images
You can save images locally or to a logger such TensorBoard or Weights and Biases by setting the following configuration.
@@ -239,18 +250,19 @@ logging:
```
For more information on logging images, refer to the [Logging Documentation](https://openvinotoolkit.github.io/anomalib/guides/logging.html)
-___
-## Datasets
+---
+
+# Datasets
`anomalib` supports MVTec AD [(CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/) and BeanTech [(CC-BY-SA)](https://creativecommons.org/licenses/by-sa/4.0/legalcode) for benchmarking and `folder` for custom dataset training/inference.
-### [MVTec AD Dataset](https://www.mvtec.com/company/research/datasets/mvtec-ad)
+## [MVTec AD Dataset](https://www.mvtec.com/company/research/datasets/mvtec-ad)
MVTec AD dataset is one of the main benchmarks for anomaly detection, and is released under the
Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License [(CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/).
-### Image-Level AUC
+## Image-Level AUC
| Model | | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
| ------------- | ------------------ | :-------: | :-------: | :-------: | :-----: | :-------: | :-------: | :-----: | :-------: | :-------: | :------: | :-------: | :-------: | :-------: | :--------: | :--------: | :-------: |
@@ -279,7 +291,7 @@ Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
| STFPM | Wide ResNet-50 | 0.903 | 0.987 | **0.989** | 0.980 | 0.966 | 0.956 | 0.966 | 0.913 | 0.956 | 0.974 | 0.961 | 0.946 | 0.988 | 0.178 | 0.807 | 0.980 |
| STFPM | ResNet-18 | 0.951 | 0.986 | 0.988 | 0.991 | 0.946 | 0.949 | 0.971 | 0.898 | 0.962 | 0.981 | 0.942 | 0.878 | 0.983 | 0.983 | 0.838 | 0.972 |
-### Image F1 Score
+## Image F1 Score
| Model | | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
| ------------- | ------------------ | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :--------: | :--------: | :-------: |
@@ -296,9 +308,11 @@ Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
| DFKDE | ResNet-18 | 0.872 | 0.864 | 0.844 | 0.854 | 0.960 | 0.898 | 0.942 | 0.793 | 0.908 | 0.827 | 0.894 | 0.916 | 0.859 | 0.853 | 0.756 | 0.916 |
| GANomaly | | 0.834 | 0.864 | 0.844 | 0.852 | 0.836 | 0.863 | 0.863 | 0.760 | 0.905 | 0.777 | 0.894 | 0.916 | 0.853 | 0.833 | 0.571 | 0.881 |
-## Reference
+# Reference
+
If you use this library and love it, use this to cite it π€
-```
+
+```tex
@misc{anomalib,
title={Anomalib: A Deep Learning Library for Anomaly Detection},
author={Samet Akcay and
diff --git a/anomalib/models/cflow/README.md b/anomalib/models/cflow/README.md
index e8d3cfe139..3b97c2d7a2 100644
--- a/anomalib/models/cflow/README.md
+++ b/anomalib/models/cflow/README.md
@@ -1,4 +1,4 @@
-# Real-Time Unsupervised Anomaly Detection via Conditional Normalizing Flows
+# Real-Time Unsupervised Anomaly Detection via Conditional Normalizing Flows
This is the implementation of the [CFLOW-AD](https://arxiv.org/pdf/2107.12571v1.pdf) paper. This code is modified form of the [official repository](https://github.com/gudovskiy/cflow-ad).
@@ -26,19 +26,19 @@ All results gathered with seed `42`.
| | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
| -------------- | :---: | :----: | :---: | :-----: | :---: | :---: | :----: | :---: | :-----: | :------: | :-------: | :---: | :---: | :--------: | :--------: | :----: |
-| Wide ResNet-50 | 0.962 | 0.986 | 0.962 | 1.0 | 0.999 | 0.993 | 1.0 | 0.893 | 0.945 | 1.0 | 0.995 | 0.924 | 0.908 | 0.897 | 0.943 | 0.984 |
+| Wide ResNet-50 | 0.962 | 0.986 | 0.962 | 1.0 | 0.999 | 0.993 | 1.0 | 0.893 | 0.945 | 1.0 | 0.995 | 0.924 | 0.908 | 0.897 | 0.943 | 0.984 |
### Pixel-Level AUC
| | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
| -------------- | :---: | :----: | :---: | :-----: | :---: | :---: | :----: | :---: | :-----: | :------: | :-------: | :---: | :---: | :--------: | :--------: | :----: |
-| Wide ResNet-50 | 0.971 | 0.986 | 0.968 | 0.993 | 0.968 | 0.924 | 0.981 | 0.955 | 0.988 | 0.990 | 0.982 | 0.983 | 0.979 | 0.985 | 0.897 | 0.980 |
+| Wide ResNet-50 | 0.971 | 0.986 | 0.968 | 0.993 | 0.968 | 0.924 | 0.981 | 0.955 | 0.988 | 0.990 | 0.982 | 0.983 | 0.979 | 0.985 | 0.897 | 0.980 |
### Image F1 Score
| | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
| -------------- | :---: | :----: | :---: | :-----: | :---: | :---: | :----: | :---: | :-----: | :------: | :-------: | :---: | :---: | :--------: | :--------: | :----: |
-| Wide ResNet-50 | 0.944 | 0.972 | 0.932 | 1.000 | 0.988 | 0.967 | 1.000 | 0.832 | 0.939 | 1.000 | 0.979 | 0.924 | 0.971 | 0.870 | 0.818 | 0.967 |
+| Wide ResNet-50 | 0.944 | 0.972 | 0.932 | 1.000 | 0.988 | 0.967 | 1.000 | 0.832 | 0.939 | 1.000 | 0.979 | 0.924 | 0.971 | 0.870 | 0.818 | 0.967 |
### Sample Results
diff --git a/anomalib/models/components/freia/README.md b/anomalib/models/components/freia/README.md
index b9ef90a20b..9fa6281739 100644
--- a/anomalib/models/components/freia/README.md
+++ b/anomalib/models/components/freia/README.md
@@ -1,5 +1,7 @@
-## FrEIA
+# FrEIA
+
This sub-package contains freia packages to use within flow-based algorithms such as Cflow.
## Description
+
[FrEIA](https://github.com/VLL-HD/FrEIA) package is currently not available in pypi to install via pip. The only way to install it is `pip install git+https://github.com/VLL-HD/FrEIA.git`. PyPI, however, does not support installing packages from git links. Due to this limitation, anomalib cannot be updated on PyPI. To avoid this, `anomalib` contains some of the [FrEIA](https://github.com/VLL-HD/FrEIA) modules to facilitate CFlow training/inference.
diff --git a/anomalib/models/draem/README.md b/anomalib/models/draem/README.md
index 8780574c98..d888f1860f 100644
--- a/anomalib/models/draem/README.md
+++ b/anomalib/models/draem/README.md
@@ -11,6 +11,7 @@ DRAEM is a reconstruction based algorithm that consists of a reconstructive subn
For optimal results, DRAEM requires specifying the path to a folder of image data that will be used as the source of the anomalous pixel regions in the simulated anomaly images. The path can be specified by editing the value of the `model.anomaly_source_path` parameter in the `config.yaml` file. The authors of the original paper recommend using the [DTD](https://www.robots.ox.ac.uk/~vgg/data/dtd/) dataset as anomaly source.
## Architecture
+
![DRAEM Architecture](../../../docs/source/images/draem/architecture.png "DRAEM Architecture")
## Usage
diff --git a/anomalib/models/fastflow/README.md b/anomalib/models/fastflow/README.md
index c79bc53249..057269f359 100644
--- a/anomalib/models/fastflow/README.md
+++ b/anomalib/models/fastflow/README.md
@@ -1,6 +1,6 @@
# FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows
-This is the implementation of the [FastFlow](https://arxiv.org/abs/2111.07677) paper. This code is developed by utilizing the torch model implemented in [https://github.com/gathierry/FastFlow](https://github.com/gathierry/FastFlow).
+This is the implementation of the [FastFlow](https://arxiv.org/abs/2111.07677) paper. This code is developed by utilizing the torch model implemented in [https://github.com/gathierry/FastFlow](https://github.com/gathierry/FastFlow).
Model Type: Segmentation
@@ -23,6 +23,7 @@ All results gathered with seed `0`.
## [MVTec AD Dataset](https://www.mvtec.com/company/research/datasets/mvtec-ad)
---
+
**NOTE**
When the numbers are produced, early stopping callback (patience: 3) is used. It might be possible to achieve higher-metrics by increasing the patience.
@@ -50,7 +51,6 @@ When the numbers are produced, early stopping callback (patience: 3) is used. It
| Zipper | 0.878 | 0.951 | 0.981 | 0.977 |
| Average | | | | |
-
### Pixel-Level AUC
| | ResNet-18 | Wide ResNet50 | DeiT | CaiT |
@@ -72,9 +72,8 @@ When the numbers are produced, early stopping callback (patience: 3) is used. It
| Zipper | 0.965 | 0.985 | 0.978 | 0.979 |
| Average | | | | |
-
-
### Image F1 Score
+
| | ResNet-18 | Wide ResNet50 | DeiT | CaiT |
| ---------- | :-------: | :-----------: | :---: | :---: |
| Bottle | 0.976 | 0.952 | 0.741 | 0.977 |
@@ -95,6 +94,7 @@ When the numbers are produced, early stopping callback (patience: 3) is used. It
| Average | | | | |
### Pixel F1 Score
+
| | ResNet-18 | Wide ResNet50 | DeiT | CaiT |
| ---------- | :-------: | :-----------: | :---: | :---: |
| Bottle | 0.670 | 0.733 | 0.753 | 0.725 |
@@ -114,7 +114,6 @@ When the numbers are produced, early stopping callback (patience: 3) is used. It
| Zipper | 0.492 | 0.621 | 0.522 | 0.504 |
| Average | | | | |
-
### Sample Results
![Sample Result 1](../../../docs/source/images/fastflow/results/0.png "Sample Result 1")
diff --git a/anomalib/models/ganomaly/README.md b/anomalib/models/ganomaly/README.md
index 563fc502eb..215674c81f 100644
--- a/anomalib/models/ganomaly/README.md
+++ b/anomalib/models/ganomaly/README.md
@@ -26,12 +26,12 @@ All results gathered with seed `42`.
### Image-Level AUC
-| | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
-| -------------- | :---: | :----: | :---: | :-----: | :---: | :---: | :----: | :---: | :-----: | :------: | :-------: | :---: | :---: | :--------: | :--------: | :----: |
-| | 0.421 | 0.203 | 0.404 | 0.413 | 0.408 | 0.744 | 0.251 | 0.457 | 0.682 | 0.537 | 0.270 | 0.472 | 0.231 | 0.372 | 0.440 | 0.434 |
+| | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
+| --- | :---: | :----: | :---: | :-----: | :---: | :---: | :----: | :---: | :-----: | :------: | :-------: | :---: | :---: | :--------: | :--------: | :----: |
+| | 0.421 | 0.203 | 0.404 | 0.413 | 0.408 | 0.744 | 0.251 | 0.457 | 0.682 | 0.537 | 0.270 | 0.472 | 0.231 | 0.372 | 0.440 | 0.434 |
### Image F1 Score
-| | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
-| -------------- | :---: | :----: | :---: | :-----: | :---: | :---: | :----: | :---: | :-----: | :------: | :-------: | :---: | :---: | :--------: | :--------: | :----: |
-| | 0.834 | 0.864 | 0.844 | 0.852 | 0.836 | 0.863 | 0.863 | 0.760 | 0.905 | 0.777 | 0.894 | 0.916 | 0.853 | 0.833 | 0.571 | 0.881 |
+| | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
+| --- | :---: | :----: | :---: | :-----: | :---: | :---: | :----: | :---: | :-----: | :------: | :-------: | :---: | :---: | :--------: | :--------: | :----: |
+| | 0.834 | 0.864 | 0.844 | 0.852 | 0.836 | 0.863 | 0.863 | 0.760 | 0.905 | 0.777 | 0.894 | 0.916 | 0.853 | 0.833 | 0.571 | 0.881 |
diff --git a/docs/blog/001-train-custom-dataset/README.md b/docs/blog/001-train-custom-dataset/README.md
index 8d8bf76de0..9559f205de 100644
--- a/docs/blog/001-train-custom-dataset/README.md
+++ b/docs/blog/001-train-custom-dataset/README.md
@@ -1,24 +1,27 @@
# How to Train Your Custom Dataset with Anomalib
+
## Introducing Anomalib
-A Deep Learning Library called Anomalib was released by a group of Intel AI researchers in early 2022. The library includes cutting-edge algorithms for detecting anomalies in image datasets. Anomalib is a comprehensive end-to-end solution that includes many features for achieving the highest accuracy model as well as inference deployment code with Intel's OpenVino Toolkit.
+A Deep Learning Library called Anomalib was released by a group of Intel AI researchers in early 2022. The library includes cutting-edge algorithms for detecting anomalies in image datasets. Anomalib is a comprehensive end-to-end solution that includes many features for achieving the highest accuracy model as well as inference deployment code with Intel's OpenVino Toolkit.
-## Why a dedicated library for Anomaly Detection?
+## Why a dedicated library for Anomaly Detection
Anomaly detection is the process of identifying anomalous items in a stream of input data. An example of a real-world anomaly detection problem is industrial defect detection, where the aim is to identify anomalous products in the output of a production line. In this type of application, anomalous items occur much less frequently than normal items, which makes it challenging to collect sufficient representative samples of the anomalous class. On top of that, the anomalous class is not well defined, and can contain a wide range of different defect types. These characteristics make it difficult to solve anomaly detection problems with classical, supervised methods. Instead, anomaly detection algorithms usually rely on unsupervised techniques to learn an implicit representation of normality, the normality model. During inference, new samples are compared against this normality model to determine if they belong to the normal or anomalous category.
-Anomalib aims to collect the most recent deep-learning based anomaly detection algorithms with the purpose of providing a tool that makes it easy to benchmark different anomaly detection algorithms on public and custom datasets. Anomalib is continuously updated with the latest State-of-the-Art algorithms, and contains several tools and interfaces that can be used to run experiments and create and test new algorithms.
+Anomalib aims to collect the most recent deep-learning based anomaly detection algorithms with the purpose of providing a tool that makes it easy to benchmark different anomaly detection algorithms on public and custom datasets. Anomalib is continuously updated with the latest State-of-the-Art algorithms, and contains several tools and interfaces that can be used to run experiments and create and test new algorithms.
## How to train a Custom dataset with Anomalib
Anomalib supports a number of datasets in various formats, including the state-of-the-art anomaly detection benchmarks such as MVTec AD and BeanTech. For those who would like to use the library on their custom datasets, anomalib also provides a `Folder` datamodule that can load datasets from a folder on a file system. The scope of this post will be to train anomalib models on custom datasets using the `Folder` datamodule.
### Step 1: Install Anomalib
+
#### Option - 1 : PyPI
+
Anomalib can be installed from PyPI via the following:
```bash
@@ -26,7 +29,9 @@ pip install anomalib
```
#### Option - 2: Editable Install
+
Alternatively, it is also possible to do editable install:
+
```bash
git clone https://github.com/openvinotoolkit/anomalib.git
cd anomalib
@@ -34,24 +39,29 @@ pip install -e .
```
### Step 2: Collect Your Custom Dataset
+
Anomalib supports multiple image extensions such as `".jpg", ".jpeg", ".png", ".ppm", ".bmp", ".pgm", ".tif", ".tiff", and ".webp"`. A dataset can be collected from images that have any of these extensions.
### Step 3: Format your dataset
+
Depending on the use-case and collection, custom datasets can have different formats, some of which are listed below:
-- A dataset with good and bad images.
-- A dataset with good and bad images as well as mask ground-truths for pixel-wise evaluation.
-- A dataset with good and bad images that is already split into training and testing sets.
-Each of these use-cases is addressed by anomalib's `Folder` datamodule. Let's focus on the first use-case as an example of end-to-end model training and inference. In this post, we will use a toy dataset which you can download from [here](https://openvinotoolkit.github.io/anomalib/_downloads/3f2af1d7748194b18c2177a34c03a2c4/hazelnut_toy.zip). The dataset consists of several folders, each containing a set of images. The _colour_ and the _crack_ folders represent two kinds of defects. We can ignore the _masks_ folder for now.
+- A dataset with good and bad images.
+- A dataset with good and bad images as well as mask ground-truths for pixel-wise evaluation.
+- A dataset with good and bad images that is already split into training and testing sets.
+
+Each of these use-cases is addressed by anomalib's `Folder` datamodule. Let's focus on the first use-case as an example of end-to-end model training and inference. In this post, we will use a toy dataset which you can download from [here](https://openvinotoolkit.github.io/anomalib/_downloads/3f2af1d7748194b18c2177a34c03a2c4/hazelnut_toy.zip). The dataset consists of several folders, each containing a set of images. The _colour_ and the _crack_ folders represent two kinds of defects. We can ignore the _masks_ folder for now.
Load your data to the following directory structure. Anomalib will use all images in the _colour_ folder as part of the validation dataset and then randomly split the good images for training and validation.
-```
+
+```bash
Hazelnut_toy
βββ colour
βββ good
```
### Step 4: Modify Config File
+
A YAML configuration file is necessary to run training for Anomalib. The training configuration parameters are categorized into 5 sections: `dataset`, `model`, `project`, `logging`, `trainer`.
To get Anomalib functionally working with a custom dataset, one only needs to change the `dataset` section of the configuration file.
@@ -103,19 +113,22 @@ model:
```
### Step 5: Run training
+
As per the config file, move `Hazelnut_toy` to the datasets section in the main root directory of anomalib, and then run
```bash
python tools/train.py --config custom_padim.yaml
```
-### Step 6: Interpret Results
+### Step 6: Interpret Results
+
Anomalib will print out results of the trained model on the validation dataset. The printed metrics are dependent on the task mode chosen. The classification example provided in this tutorial prints out two scores: F1 and AUROC. The F1 score is a metric which values both the precision and recall, more information on its calculation can be found in this [blog](https://towardsdatascience.com/understanding-accuracy-recall-precision-f1-scores-and-confusion-matrices-561e0f5e328c).
**Additional Info**
Not only does Anomalib classify whether a part is defected or not, it can also be used to segment the defects as well. To do this, simply add a folder called _mask_ at the same directory level as the _good_ and _colour_ folders. This folder should contain binary images for the defects in the _colour_ folder. Here, the white pixels represent the location of the defect. Populate the mask field in the config file with `mask` and change the task to segmentation to see Anomalib segment defects.
-```
+
+```bash
Hazelnut_toy
βββ colour
β βββ 00.jpg
@@ -137,12 +150,14 @@ Here is an example of the generated results for a toy dataset containing Hazelnu
## Logging and Experiment Management
+
While it is delightful to know how good your model performed on your preferred metric, it is even more exciting to see the predicted outputs. Anomalib provides a couple of ways to log and track experiments. These can be used individually or in a combination. As of the current release, you can save images to a local folder, or upload to weights and biases, or TensorBoard.
To select where you would like to save the images, change the `log_images_to` parameter in the `project` section in the config file.
For example, setting the following `log_images_to: ["local"]` will result in saving the images in the results folder as shown in the tree structure below:
-```
+
+```bash
results
βββ padim
βββ Hazelnut_toy
@@ -160,13 +175,15 @@ results
```
### Logging to Tensorboard and/or W&B
-To use TensorBoard and/or W&B logger, ensure that the logger parameter is set to `tensorboard`, `wandb` or `[tensorboard, wandb]` in the `logging` section of the config file.
-An example configuration for saving to TensorBoard is shown in the figure below. Similarly after setting logger to `wandb` you will see the images on your wandb project dashboard.
+To use TensorBoard and/or W&B logger, ensure that the logger parameter is set to `tensorboard`, `wandb` or `[tensorboard, wandb]` in the `logging` section of the config file.
+
+An example configuration for saving to TensorBoard is shown in the figure below. Similarly after setting logger to `wandb` you will see the images on your wandb project dashboard.
+
```yaml
logging:
- log_images_to: [tensorboard]
- logger: tensorboard # options: [tensorboard, wandb, csv] or combinations.
+ log_images_to: [tensorboard]
+ logger: tensorboard # options: [tensorboard, wandb, csv] or combinations.
```
@@ -174,6 +191,7 @@ logging:
### Hyper-Parameter Optimization
+
It is very rare to find a model which works out of the box for a particular dataset. However, fortunately, we support tools which work out of the box to help tune the models in Anomalib to your particular dataset. As of the publication of this blog post, Anomalib supports [weights and biases](https://wandb.ai/) for hyperparameter optimization. To get started have a look at `sweep.yaml` located at `tools/hpo`. It provides a sample of how one can define a hyperparameter sweep.
```yaml
@@ -196,18 +214,20 @@ The observation_budget informs wandb about the number of experiments to run. The
To run a sweep, you can just call,
-```
+```bash
python tools/hpo/wandb_sweep.py --model padim --config ./path_to_config.yaml --sweep_config tools/hpo/sweep.yaml"
```
In case `model_config` is not provided, the script looks at the default config location for that model. Note, you will need to have logged into a wandb account to use HPO search and view the results.
A sample run is visible in the screenshot below.
+
## Benchmarking
+
To add to the suit of experiment tracking and optimization, anomalib also includes a benchmarking script for gathering results across different combinations of models, their parameters, and dataset categories. The model performance and throughputs are logged into a csv file that can also serve as a means to track model drift. Optionally, these same results can be logged to Weights and Biases and TensorBoard. A sample configuration file is shown in the screenshot below.
```yaml
diff --git a/docs/source/guides/structure_of_documentation.md b/docs/source/guides/structure_of_documentation.md
index 36d5a87acf..502eeaa425 100644
--- a/docs/source/guides/structure_of_documentation.md
+++ b/docs/source/guides/structure_of_documentation.md
@@ -1,13 +1,13 @@
# Structure of the Documentation
-
This documentation is divided into the following sections:
+
1. Getting Started
-2. Models
-3. API Reference
-4. Tutorials
-5. Guides
-6. Research
+1. Models
+1. API Reference
+1. Tutorials
+1. Guides
+1. Research
## Getting Started
@@ -23,6 +23,7 @@ responsibility to update this page when a new model is added to the repo.
This page lists all the modules, classes and functions available within the `anomalib` package. This page is update
automatically for the following modules:
+
- config
- core
- datasets
diff --git a/docs/source/guides/using_pre_commit.md b/docs/source/guides/using_pre_commit.md
index 63c1bfd9ab..0d1a58ba26 100644
--- a/docs/source/guides/using_pre_commit.md
+++ b/docs/source/guides/using_pre_commit.md
@@ -1,23 +1,24 @@
(pre-commit_hooks)=
+
# Pre-commit Hooks
First of all, you will need to install development requirements. This will also install the pre-commit pip package
in your python environment
-```pip install -r requirements/dev.txt```
+`pip install -r requirements/dev.txt`
Then, install pre-commit hooks using the following command:
-```pre-commit install```
+`pre-commit install`
Pre-commit hooks will run several formatters, linters and type checkers each time you commit some changes to a branch. Some tools like black and isort will automatically format your files to enforce consistent formatting within the repo. Other tools will provide a list of errors and warnings which you will be required to address before being able to make the commit.
In some cases it might be desired to commit your changes even though some of the checks are failing. For example when you want to address the pre-commit issues at a later time, or when you want to commit a work-in-progress. In these cases, you can skip the pre-commit hooks by adding the `--no-verify` parameter to the commit command.
-```git commit -m 'WIP commit' --no-verify```
+`git commit -m 'WIP commit' --no-verify`
When doing so, please make sure to revisit the issues at a later time. A good way to check if all issues have been addressed before making an MR is to run tox.
Apart from tox, you can also run `pre-commit` on all the files to check formatting and style issues. To do this you can use
-```pre-commit run --all```
+`pre-commit run --all`
diff --git a/docs/source/guides/using_tox.md b/docs/source/guides/using_tox.md
index 44e2dbb410..788a78bacf 100644
--- a/docs/source/guides/using_tox.md
+++ b/docs/source/guides/using_tox.md
@@ -1,12 +1,15 @@
(using_tox)=
+
# Using Tox
The quality of code for `anomalib` maintained by using tools such as `black`, `mypy`, and `flake8`. In order to enforce these, we use Tox. This is also the reason why we don't have dependencies such as `pytest` in the `requirements.txt` file.
What is tox?
-> `tox` aims to automate and standardize testing in Python. It is part of a larger vision of easing the packaging, testing and release process of Python software.
+> `tox` aims to automate and standardize testing in Python. It is part of a larger vision of easing the packaging, testing and release process of Python software.
+>
> It is a generic virtualenv management and test command line tool you can use for:
+>
> - checking that your package installs correctly with different Python versions and interpreters
> - running your tests in each of the environments, configuring your test tool of choice
> - acting as a frontend to Continuous Integration servers, greatly reducing boilerplate and merging CI and shell-based testing. - from the [docs](https://tox.readthedocs.io/)
@@ -18,20 +21,29 @@ See [getting started](#setting-up-and-getting-started) to dive in and get `tox`
Setting up tox is easy
1. Create new Conda environment
+
`conda create -n toxenv python=3.8`
-2. Activate the environment
+1. Activate the environment
+
`conda activate toxenv`
-3. Install tox
+1. Install tox
+
`pip install tox`
-4. Run
+1. Run
+
`tox`
It should setup the environments and install the dependencies.
-```{note} All developers are required to run tox before creating their MR. If you have setup pre-commit hooks then this step can be skipped.
-```
+---
+
+**NOTE**
+
+All developers are required to run tox before creating their MR. If you have setup pre-commit hooks then this step can be skipped.
+
+---
## Brief Explanation of `tox.ini`
@@ -46,7 +58,7 @@ All `tox` files start with
`envlist` This defines all the environments that tox should create. Take for example that we want to create an environment to run `black` formatter. We can name this environment as:
-```
+```ini
[tox]
envlist =
black_env
@@ -54,7 +66,7 @@ envlist =
Then to define the commands in this environment all we need to do is start a block with `[testenv:black_env]` and then define the dependencies and call the commands.
-```
+```ini
[testenv:black_env]
deps = black
commands = black .
diff --git a/docs/source/research/benchmark.md b/docs/source/research/benchmark.md
index ad31647816..354f0d30ba 100644
--- a/docs/source/research/benchmark.md
+++ b/docs/source/research/benchmark.md
@@ -27,6 +27,7 @@
| PatchCore | Wide ResNet-50 | 0.955 | 0.988 | 0.903 | 0.990 | 0.957 | 0.936 | 0.972 | 0.950 | 0.968 | 0.974 | 0.960 | 0.948 | 0.917 | 0.969 | 0.913 | 0.976 |
| PaDiM | ResNet-18 | 0.968 | 0.984 | 0.918 | **0.994** | 0.934 | 0.947 | 0.983 | 0.965 | 0.984 | 0.978 | 0.970 | 0.957 | 0.978 | 0.988 | 0.968 | 0.979 |
| **PaDiM** | **Wide ResNet-50** | **0.979** | **0.991** | 0.970 | 0.993 | 0.955 | **0.957** | **0.985** | **0.970** | **0.988** | **0.985** | **0.982** | **0.966** | **0.988** | **0.991** | **0.976** | **0.986** |
+
### Image F1 Score
| Model | | Avg | Carpet | Grid | Leather | Tile | Wood | Bottle | Cable | Capsule | Hazelnut | Metal Nut | Pill | Screw | Toothbrush | Transistor | Zipper |
diff --git a/docs/source/research/citation.md b/docs/source/research/citation.md
index 2485b53bdc..5aced2cab5 100644
--- a/docs/source/research/citation.md
+++ b/docs/source/research/citation.md
@@ -2,7 +2,7 @@
You can cite this repository as
-```
+```tex
@misc{anomalib,
title={Anomalib: A Deep Learning Library for Anomaly Detection},
author={Samet Akcay and
diff --git a/markdownlint.rb b/markdownlint.rb
new file mode 100644
index 0000000000..4515a6a379
--- /dev/null
+++ b/markdownlint.rb
@@ -0,0 +1,62 @@
+################################################################################
+# Modified using
+# https://github.com/jumanjihouse/pre-commit-hooks/blob/master/ci/jumanjistyle.rb
+################################################################################
+
+
+# frozen_string_literal: true
+
+################################################################################
+# Style file for markdownlint.
+#
+# https://github.com/markdownlint/markdownlint/blob/master/docs/configuration.md
+#
+# This file is referenced by the project `.mdlrc`.
+################################################################################
+
+#===============================================================================
+# Start with all built-in rules.
+# https://github.com/markdownlint/markdownlint/blob/master/docs/RULES.md
+all
+
+#===============================================================================
+# Override default parameters for some built-in rules.
+# https://github.com/markdownlint/markdownlint/blob/master/docs/creating_styles.md#parameters
+
+# Allow both fenced and indented code blocks.
+# rule 'MD046', style: ['fenced', 'indented']
+
+# Ignore line length in code blocks.
+rule 'MD013', :line_length => 1000
+# rule 'MD013', code_blocks: false
+
+#===============================================================================
+# Exclude the rules I disagree with.
+
+# IMHO it's easier to read lists like:
+# * outmost indent
+# - one indent
+# - second indent
+# * Another major bullet
+exclude_rule 'MD004' # Unordered list style
+
+# I prefer two blank lines before each heading.
+exclude_rule 'MD012' # Multiple consecutive blank lines
+
+# This is not useful for some files such as `CHANGELOG.md`
+exclude_rule 'MD024' # Multiple headers with the same content
+
+# I find it necessary to use '
' to force line breaks.
+exclude_rule 'MD033' # Inline HTML
+
+# If a page is printed, it helps if the URL is viewable.
+exclude_rule 'MD034' # Bare URL used
+
+# Some md files have comments or links at the top of the files.
+exclude_rule 'MD041' # First line in file should be a top level header
+
+#===============================================================================
+# Exclude rules for pragmatic reasons.
+
+# Either disable this one or MD024 - Multiple headers with the same content.
+exclude_rule 'MD036' # Emphasis used instead of a header
diff --git a/notebooks/100_datamodules/README.md b/notebooks/100_datamodules/README.md
index 5423c8488a..e9cf385435 100644
--- a/notebooks/100_datamodules/README.md
+++ b/notebooks/100_datamodules/README.md
@@ -1,13 +1,14 @@
# Anomalib DataModules
+
The notebooks in this section demonstrate the mechanics of anomalib data modules, with a specific focus on benchmarks such as MVTec AD, BTech, and custom datasets via the Folder module. Anomalib data modules are structured as follows: Each data collection implements the Torch Dataset and the PyTorch Lightning DataModule objects.
The Torch Dataset inherits `torch.utils.data.Dataset` and implement the `__len__` and `__getitem__` methods. This implementation might therefore be utilized not just for anomalib, but also for other implementations.
-The DataModule implementation inherits the PyTorch Lightning `DataModule` object. The advantage of this class is that it organizes each step of data from download to creating the Torch dataloader.Β
+The DataModule implementation inherits the PyTorch Lightning `DataModule` object. The advantage of this class is that it organizes each step of data from download to creating the Torch dataloader.
Overall, a data implementation has the following structure:
-```
+```bash
anomalib
βββ __init__.py
βββ data
diff --git a/notebooks/README.md b/notebooks/README.md
index 7163c35f5f..2625ab725d 100644
--- a/notebooks/README.md
+++ b/notebooks/README.md
@@ -15,10 +15,10 @@
| Folder | [103_folder](100_datamodules/103_folder.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/development/notebooks/100_datamodules/103_folder.ipynb) |
## 2. Models
-| Notebook | GitHub | Colab |
-| ------------ | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| Model | [201_fastflow](200_models/201_fastflow.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/development/notebooks/200_models/201_fastflow.ipynb) |
+| Notebook | GitHub | Colab |
+| -------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Model | [201_fastflow](200_models/201_fastflow.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/development/notebooks/200_models/201_fastflow.ipynb) |
## 3. Benchmarking and Hyperparameter Optimization
diff --git a/tools/benchmarking/README.md b/tools/benchmarking/README.md
index d54e254253..8b93aba717 100644
--- a/tools/benchmarking/README.md
+++ b/tools/benchmarking/README.md
@@ -5,31 +5,10 @@ These bash scripts will assist in measuring the training performance of the anom
The python script (`benchmark.py`) will assist in computing metrics for all the models in the repository.
## Usage
-Run the train.sh with the same args as the tools/train.py. Refer to [`../README.md`](https://github.com/openvinotoolkit/anomalib/blob/development/README.md) for those details.
-
-Note: To collect memory read/write numbers, run the script with sudo privileges. Otherwise, those values will be blank.
-
-```
-sudo -E ./train.sh # Train STFPM on MVTec AD leather
-
-sudo -E ./train.sh --config
-
-sudo -E ./train.sh --model stfpm
-```
-
-The training script will create an output directory in this location, and inside it will have a time stamped directory for each training run you do. You can find the raw logs in there, as well as any errors captured in the train.log file.
-
-For post processing, run the post-process.sh script with the results directory you want to post process.
-
-```
-./post-process.sh ./output/2021Aug31_2351
-```
-
----
To use the python script, run it from the root directory.
-```
+```bash
python tools/benchmarking/benchmark.py
```