Pytorch implementation for the manuscript "General stain deconvolution of histopathology images with physics-guided deep learning" The current implementation supports deconvolution of brightfield images stained with two or more colors, and it will be extended to support darkfield images soon.
Python version requirement: 3.9+
The recommended installation is through Anaconda or anaconda/miniconda
Installation takes 15-30 minutes on a normal desktop assuming good internet connection.
$ conda env create --name pgdips
$ conda activate pgdips
$ pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116
$ pip3 install -r requirements.txt
NOTE: Please check if your graphics card is compatible with the following the packages:
cudatoolkit=11.6andpytorch 1.13.0. If your GPU will not support them, you can downgradepytorch,cudaandcudnn.
Alternatively, you can install all required packages specified in environment.yml.
$ conda env create --name pgdips --file environment.yml
$ conda activate pgdips
comming soon
A user-friendly web interface for PGDIPS is available through the CODIDO platform: https://www.codido.co/marketplace/browse.
Train PGDIPS on provided target images:
$ python pgdips.py --config=he.cfg
The following example target images are available:
[--config] # he.cfg, ihc.cfg, alcianblue.cfg, DeepLIIF6.cfg, fibronectin.cfg, SMACD34.cfg, trichrome.cfg
Training on your own data (minimum 10x resolution, ideally 20x-40x) by creating a configuration file
2. Create a configuration file for your own data based on the following template and save it as config\[config_name].
[system]
input = [image_name]
input_folder = ./input
output_folder = ./output
[preprocess]
input_depth = 3
input_size = 512
roi_selection = centercrop
x_coor_start = 1
y_coor_start = 1
[stain]
num_stains = 2
fixed_color = False
default_stain_vector_1 = 0.60, 0.75, 0.29
default_stain_vector_2 = 0.21, 0.91, 0.36
use_spectral_correction_factor = False
[network]
iterations = 6000
keep_color_iterations = 2000
init_learning_rate = 0.0005
plot_during_training = True
show_every = 400
[loss]
weight_l1 = 1
weight_exclusion = 0.3
weight_mse = 1
input = [image_name]# name of the input/target image in theinput_folderinput_folder = ./input# where input images are storedoutput_folder = ./output# where output files should be storedinput_depth = 3# input image color channels,1for grayscale and3for rgbinput_size = 512#256,512,1024, etc # dimensions of input imageroi_selection = centercrop# how to select from the target image, pick from the following:centercrop,crop,resizex_coor_start = 1# x-coordinate of the top left pixel for cropping, only applicable whenroi_selectioniscropy_coor_start = 1# y-coordinate of the top left pixel for cropping, only applicable whenroi_selectioniscropnum_stains = 2# number of stains,2or3fixed_color = False# whether to NOT learn stain color vectors. Only useTrueif you know that the default stain vectors are the true stain color vectorsdefault_stain_vector_1 = 0.60, 0.75, 0.29# default stain color vector for stain 1, no need to change thesedefault_stain_vector_2 = 0.21, 0.91, 0.36# default stain color vector for stain 2 no need to change theseuse_spectral_correction_factor = False#TrueorFalse, whether to apply the spectral correction factoriterations = 6000# number of iterations,6000iters is a good place to startkeep_color_iterations = 2000# how long are the stain vectors regulated by the default stain vectorsplot_during_training = True# save intermediate plots?show_every = 400# how often to save intermediate plotsweight_l1 = 1# weight for the l1 reconstruction losssweight_exclusion = 0.3# weight for exlusion loss, increase this if concentration maps overlap and decrease this when one or more c-maps are emptyweight_mse = 1# weight for the l2 color fixing loss
NOTE: The primary criteria for selecting ROI is to make sure that the ROI contains enough (and ideally relatively equal amounts of) concentrations of all the target stains.
centercropis the default option.resizeis not recommended as it changes the spectral bias (i.e. deep image prior) of the image and may require chaning the network structure.
NOTE: Input size determines runtime and VRAM bottleneck. A GPU with less than 6 GB VRAM can handle input size of 1024 but 512 is recommended.
$ python pgdips.py --config=[config_name]
The cropped original image will be automatically opened once you launch the command. The default output files (saved in the output folder) include the following:
CONFIGNAME_Con1_Iterations.png: concentration map for estimated stain 1CONFIGNAME_Con2_Iterations.png: concentration map for estimated stain 2CONFIGNAME_Stained1_Iterations.png: colored concentration map (concentration map multiplied by the corresponding color vector) for estimated stain 1CONFIGNAME_Stained2_Iterations.png: colored concentration map for estimated stain 2CONFIGNAME_All_Iterations.png: reconstructed image with background illunimation removedCONFIGNAME_WB_Iterations.png: reconstructed image (including estimated background illunimation)CONFIGNAME_original.png: cropped/resized original imageCONFIG_mse.png: a graph for the mean squared error between original image and reconstructed image through different iterations.CONFIGNAME_out.csv: a spreadsheet containing estimated physics parameters at different iterations, including S1R, S1G, S1B (color vectors for stain 1 in RGB channels), S2R, S2G, S2B (color vectors for stain 2)，SCF1, SCF2 (spectral correction factor for stain 1 and 2, respectively), BGR, BGG, BGB (Background illunimation in RGB channels)
-
Double check the quality of input image: Please double check the preprocessed (cropped) image for stain deconvolution, which is stored in the output folder. PGDIPS only has issues in two known cases: 1) when the colors of two stains are too similiar 2) when the distribution of two stains are too imbalanced (for example when there are too much hematoxylin and too little DAB stain in the input image).
-
Run PGDIPS again with parameters learned from the first run: After running PGDIPS the physics parameters such as stain color vectors and background correction factors are saved in the output folder. Running PGDIPS again using these parameters (for example by setting default color vectors 1 and 2 to the estimated color vectors from pervious rounds) will help improve the assignment of structures to correct colors, as the model will start from correct colors at the beginning of training.
-
Tune the weights of the different loss functions: We found that
weight_l1=1andweight_mse=1works in all our experiments.weight_exclusioncan have a big impact on the estimated concentration maps. If one of the concentration maps is too blurry or empty, try tuningweight_exclusiondown. -
Rerun PGDIPS: As the input of DIP modules are random noise, when the initialization of the random noise went wrong, the concentration maps may contains black hole artifacts. This is very rare and can usually solved by running PGDIPS again.
© Sunnybrook Research Institute - PGDIPS code is distributed under BSD 3 with Commons Clause license, and is available for non-commercial academic purposes.
Bugs can be reported in the GitHub Issues tab.
If you have any questions about this code, I am happy to answer your issues or emails (to [email protected]).
The work conducted by Jianan Chen was funded by grants from the Martel lab.
This code is inspired by DoubleDIP.
If you use the code or results in your research, please use the following BibTeX entry.
@article {ChenPGDIPS,
author = {Chen, Jianan and Liu, Lydia Y and Han, Wenchao and Wang, Dan and Cheung, Alison M and Tsui, Hubert and Martel, Anne L},
title = {General stain deconvolution of histopathology images with physics-guided deep learning},
elocation-id = {2022.12.06.519385},
year = {2022},
doi = {10.1101/2022.12.06.519385},
publisher = {Cold Spring Harbor Laboratory},
URL = {https://www.biorxiv.org/content/early/2022/12/10/2022.12.06.519385},
eprint = {https://www.biorxiv.org/content/early/2022/12/10/2022.12.06.519385.full.pdf},
journal = {bioRxiv}
}