diff --git a/.github/workflows/CI.yaml b/.github/workflows/ci.yaml similarity index 100% rename from .github/workflows/CI.yaml rename to .github/workflows/ci.yaml diff --git a/docs/datasets/mhist.md b/docs/datasets/mhist.md index f327080a..0e53b051 100644 --- a/docs/datasets/mhist.md +++ b/docs/datasets/mhist.md @@ -58,7 +58,7 @@ Please create a root folder, e.g. `mhist`, and download all the files there, whi We work with the splits provided by the data source. Since no "validation" split is provided, we use the "test" split as validation split. - Train split: `annotations.csv` :: "Partition" == "train" - - Valdation split: `annotations.csv` :: "Partition" == "test" + - Validation split: `annotations.csv` :: "Partition" == "test" | Splits | Train | Validation | |----------|-----------------|--------------| diff --git a/docs/datasets/total_segmentator.md b/docs/datasets/total_segmentator.md index f72d2332..de31c94c 100644 --- a/docs/datasets/total_segmentator.md +++ b/docs/datasets/total_segmentator.md @@ -1,6 +1,6 @@ # TotalSegmentator -The TotalSegmentator dataset is an radiology image-segmentation dataset with 1228 3D images and corresponding masks with 117 different anatomical structures. It can be used for segmentation and multilabel classification tasks. +The TotalSegmentator dataset is a radiology image-segmentation dataset with 1228 3D images and corresponding masks with 117 different anatomical structures. It can be used for segmentation and multilabel classification tasks. ## Raw data diff --git a/docs/index.md b/docs/index.md index d0f5d826..3e2d0dbe 100644 --- a/docs/index.md +++ b/docs/index.md @@ -90,7 +90,7 @@ For more details on the FM-backbones and instructions to replicate the results, *Note that the current version of eva implements the task- & model-independent and fixed default set up following the standard evaluation protocol proposed by [1] and described in the table below. We selected this approach to prioritize reliable, robust and fair FM-evaluation while being in line with common literature. Additionally, with future versions we are planning to allow the use of cross-validation and hyper-parameter tuning to find the optimal setup to achieve best possible performance on the implemented downstream tasks.* -With the FM as input, *eva* computes embeddings for all WSI patches which are then used as input to train a downstream head consisting of a single linear layer in a supervised setup for each of the benchmark datasets. We use early stopping with a patience of 5% of the maximal number of epochs. +With a provided FM, *eva* computes embeddings for all input images (WSI patches) which are then used to train a downstream head consisting of a single linear layer in a supervised setup for each of the benchmark datasets. We use early stopping with a patience of 5% of the maximal number of epochs. | | | |-------------------------|---------------------------| @@ -104,7 +104,7 @@ With the FM as input, *eva* computes embeddings for all WSI patches which are th | **Base learning rate** | 0.01 | | **Learning Rate** | [Base learning rate] * [Batch size] / [Base batch size] | | **Max epochs** | [Number of samples] * [Number of steps] / [Batch size] | -| **Early stopping** | 10% * [Max epochs] | +| **Early stopping** | 5% * [Max epochs] | | **Optimizer** | SGD | | **Momentum** | 0.9 | | **Weight Decay** | 0.0 | diff --git a/docs/user-guide/advanced/replicate_evaluations.md b/docs/user-guide/advanced/replicate_evaluations.md index e30f9e08..898accc2 100644 --- a/docs/user-guide/advanced/replicate_evaluations.md +++ b/docs/user-guide/advanced/replicate_evaluations.md @@ -1,6 +1,6 @@ # Replicate evaluations -To produce the evaluation results presented [here](../index.md), you can run *eva* with the settings below. +To produce the evaluation results presented [here](../../index.md#evaluation-results), you can run *eva* with the settings below. Make sure to replace `` in the commands below with `bach`, `crc`, `mhist` or `patch_camelyon`. diff --git a/docs/user-guide/tutorials/evaluate_resnet.md b/docs/user-guide/tutorials/evaluate_resnet.md index 4f66068f..d1765385 100644 --- a/docs/user-guide/tutorials/evaluate_resnet.md +++ b/docs/user-guide/tutorials/evaluate_resnet.md @@ -2,7 +2,7 @@ If you read [How to use eva](../getting-started/how_to_use.md) and followed the Tutorials to this point, you might ask yourself why you would not always use the *offline* workflow to run a complete evaluation. An *offline*-run stores the computed embeddings and runs faster than the *online*-workflow which computes a backbone-forward pass in every epoch. -One use case for the *online*-workflow is the evaluation of a supervised ML model that does not rely on an backbone/head architecture. To demonstrate this, lets train a ResNet 18 from [Pytoch Image Models (timm)](https://timm.fast.ai/). +One use case for the *online*-workflow is the evaluation of a supervised ML model that does not rely on an backbone/head architecture. To demonstrate this, let's train a ResNet 18 from [Pytoch Image Models (timm)](https://timm.fast.ai/). To do this we need to create a new config-file: @@ -21,18 +21,18 @@ Now let's adapt the new `bach.yaml`-config to the new model: path: timm.create_model arguments: model_name: resnet18 - num_classes: &NUM_CLASSES 2 + num_classes: &NUM_CLASSES 4 drop_rate: 0.0 pretrained: false ``` -To reduce training time, lets overwrite some of the default parameters. In the terminal where you run ***eva***, set: +To reduce training time, let's overwrite some of the default parameters. In the terminal where you run *eva*, set: ``` export OUTPUT_ROOT=logs/resnet/bach -export MAX_STEPS=20 -export LR_VALUE=0.1 +export MAX_STEPS=50 +export LR_VALUE=0.01 ``` Now train and evaluate the model by running: ``` eva fit --config configs/vision/resnet18/bach.yaml ``` -Once the run is complete, take a look at the results in `logs/resnet/bach//results.json`. How does the performance compare to the results observed in the previous tutorials? +Once the run is complete, take a look at the results in `logs/resnet/bach//results.json` and check out the tensorboard with `tensorboard --logdir logs/resnet/bach`. How does the performance compare to the results observed in the previous tutorials? diff --git a/docs/user-guide/tutorials/offline_vs_online.md b/docs/user-guide/tutorials/offline_vs_online.md index 635c34d6..a4d75d8c 100644 --- a/docs/user-guide/tutorials/offline_vs_online.md +++ b/docs/user-guide/tutorials/offline_vs_online.md @@ -12,7 +12,7 @@ If you have not yet downloaded the BACH data to your machine, open `configs/visi ### 1. Compute the embeddings -First, lets use the `predict`-command to download the data and compute embeddings. In this example we use a randomly initialized `dino_vits16` as backbone. +First, let's use the `predict`-command to download the data and compute embeddings. In this example we use a randomly initialized `dino_vits16` as backbone. Open a terminal in the folder where you installed *eva* and run: ``` @@ -37,7 +37,7 @@ Once the session is complete, verify that: Now we can use the `fit`-command to evaluate the FM on the precomputed embeddings. -To ensure a quick run for the purpose of this exercise, lets overwrite some of the default parameters. In the terminal where you run *eva*, set: +To ensure a quick run for the purpose of this exercise, let's overwrite some of the default parameters. In the terminal where you run *eva*, set: ``` export MAX_STEPS=20 export LR_VALUE=0.1