Prepare for release

.gitignore

@@ -9,10 +9,10 @@ results/
 .DS_Store
 
 # private only
-# ./exp_logs.cvs
-# ./exp_logs.xlsx
-# ./scripts/metrics/run_local.sh
-# ./scripts/test_daily.sh
-# ./RF.sh
-# ./configs/RestoreFormer++_cf.yaml
-# ./configs/RestoreFormer++.yaml
+./exp_logs.cvs
+./exp_logs.xlsx
+./scripts/metrics/run_local.sh
+./scripts/test_daily.sh
+./RF.sh
+./configs/RestoreFormer++_cf.yaml
+./configs/RestoreFormer++.yaml

README.md

@@ -1,17 +1,21 @@
-# Updating
-- 20221003 We provide the link of the [test datasets](#testset).
-- 20220924 We add the code for [**metrics**](#metrics) in scripts/metrics.
+# ToDo List
 
+- [ ] Provide gradio model
+- [x] 20230914 Upload model
+- [x] 20230914 Realse Code
+- [x] 20221120 Introducing the project.
 
-# RestoreFormer
+# RestoreFormer++
 
-This repo includes the source code of the paper: "[RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs](https://arxiv.org/pdf/2201.06374.pdf)" (CVPR 2022) by Zhouxia Wang, Jiawei Zhang, Runjian Chen, Wenping Wang, and Ping Luo.
+This repo is a official implementation of "[RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Paris](https://arxiv.org/pdf/2308.07228.pdf)". 
 
-![](assets/figure1.png)
 
-**RestoreFormer** tends to explore fully-spatial attentions to model contextual information and surpasses existing works that use local operators. It has several benefits compared to prior arts. First, it incorporates a multi-head coross-attention layer to learn fully-spatial interations between corrupted queries and high-quality key-value pairs. Second, the key-value pairs in RestoreFormer are sampled from a reconstruction-oriented high-quality dictionary, whose elements are rich in high-quality facial features specifically aimed for face reconstruction.
 
-<!-- ![](assets/framework.png "Framework")-->
+[**RestoreFormer++**](https://arxiv.org/pdf/2308.07228.pdf) is an extension of our [**RestoreFormer**](https://openaccess.thecvf.com/content/CVPR2022/papers/Wang_RestoreFormer_High-Quality_Blind_Face_Restoration_From_Undegraded_Key-Value_Pairs_CVPR_2022_paper.pdf). It proposes to restore a degraded face image with both fidelity and realness by using the powerful fully-spacial attention mechanisms to model the abundant contextual information in the face and its interplay with our reconstruction-oriented high-quality priors. Besides, it introduces an extending degrading model (**EDM**) that contains more realistic degraded scenarios for training data synthesizing, which helps to enhance its robustness and generalization towards real-world scenarios. Our results compared with the state-of-the-art methods and performance with/without EDM are in following:
+
+![images/fig1.png](images/fig1.png)
+
+![images/fig3.png](images/fig3.png)
 
 ## Environment
 
@@ -26,38 +30,109 @@ This repo includes the source code of the paper: "[RestoreFormer: High-Quality B
 ## Preparations of dataset and models
 
 **Dataset**: 
-- Training data: Both **HQ Dictionary** and **RestoreFormer** in our work are trained with **FFHQ** which attained from [FFHQ repository](https://github.com/NVlabs/ffhq-dataset). The original size of the images in FFHQ are 1024x1024. We resize them to 512x512 with bilinear interpolation in our work. Link this dataset to ./data/FFHQ/image512x512.
+- Training data: Both **ROHQD** and **RestoreFormer++** in our work are trained with **FFHQ** which attained from [FFHQ repository](https://github.com/NVlabs/ffhq-dataset). The original size of the images in FFHQ are 1024x1024. We resize them to 512x512 with bilinear interpolation in our work. Link this dataset to ./data/FFHQ/image512x512.
 - <a id="testset">Test data</a>: [CelebA-Test](https://pan.baidu.com/s/1iUvBBFMkjgPcWrhZlZY2og?pwd=test), [LFW-Test](http://vis-www.cs.umass.edu/lfw/#views), [WebPhoto-Test](https://xinntao.github.io/projects/gfpgan), and [CelebChild-Test](https://xinntao.github.io/projects/gfpgan)
 
-**Model**: Both pretrained models used for training and the trained model of our RestoreFormer can be attained from [Google Driver](https://drive.google.com/file/d/13ir-VvdDNOA3LeoNbx4Y530rvIheyhRM/view?usp=sharing) or [BaiduYun (code x6nn)](https://pan.baidu.com/s/1EO7_1dYyCuORpPNosQgogg?pwd=x6nn). Link these models to ./experiments.
+**Model**: 
+Both pretrained models used for training and the trained model of our RestoreFormer and RestoreFormer++ can be attained from [Google Driver](https://connecthkuhk-my.sharepoint.com/:f:/g/personal/wzhoux_connect_hku_hk/EkZhGsLBtONKsLlWRmf6g7AB_VOA_6XAKmYUXLGKuNBsHQ?e=ic2LPl). Link these models to ./experiments.
 
 ## Test
     sh scripts/test.sh
 
+scripts/test.sh
+
+    exp_name='RestoreFormer'
+    exp_name='RestoreFormerPlusPlus'
+
+    root_path='experiments'
+    out_root_path='results'
+    align_test_path='data/test'
+    tag='test'
+
+    outdir=$out_root_path'/'$exp_name'_'$tag
+
+    if [ ! -d $outdir ];then
+        mkdir -m 777 $outdir
+    fi
+
+    CUDA_VISIBLE_DEVICES=7 python -u scripts/test.py \
+    --outdir $outdir \
+    -r $root_path'/'$exp_name'/last.ckpt' \
+    -c 'configs/'$exp_name'.yaml' \
+    --test_path $align_test_path \
+    --aligned
+
+
+- This codebase is available for both **RestoreFormer** and **RestoreFormerPlusPlus**. Determinate the specific model with *exp_name*.
+- Setting the model path with *root_path*
+- Restored results are save in *out_root_path*
+- Put the degraded face images in *test_path*
+- If the degraded face images are aligned, set *--aligned*, else remove it from the script. The provided test images in data/test are aligned.
+
 ## Training
     sh scripts/run.sh
 
-**Note**. 
-- The first stage is to attain **HQ Dictionary** by setting `conf_name` in scripts/run.sh to 'HQ\_Dictionary'. 
-- The second stage is blind face restoration. You need to add your trained HQ\_Dictionary model to `ckpt_path` in config/RestoreFormer.yaml and set `conf_name` in scripts/run.sh to 'RestoreFormer'.
-- Our model is trained with 4 V100 GPUs.
+scripts/run.sh
 
-## <a id="metrics">Metrics</a>
+    export BASICSR_JIT=True
+
+    # For RestoreFormer
+    # conf_name='HQ_Dictionary'
+    # conf_name='RestoreFormer'
+
+    # For RestoreFormer++
+    conf_name='ROHQD'
+    conf_name='RestoreFormerPlusPlus'
+
+    # gpus='0,1,2,3,4,5,6,7'
+    # node_n=1
+    # ntasks_per_node=8
+
+    root_path='PATH_TO_CHECKPOINTS'
+
+    gpus='0,'
+    node_n=1
+    ntasks_per_node=1
+
+    gpu_n=$(expr $node_n \* $ntasks_per_node)
+
+    python -u main.py \
+    --root-path $root_path \
+    --base 'configs/'$conf_name'.yaml' \
+    -t True \
+    --postfix $conf_name'_gpus'$gpu_n \
+    --gpus $gpus \
+    --num-nodes $node_n \
+    --random-seed True \
+
+- This codebase is available for both **RestoreFormer** and **RestoreFormerPlusPlus**. Determinate the training model with *conf_name*. 'HQ_Dictionary' and 'RestoreFormer' are for **RestoreFormer**, while 'ROHQD' and 'RestoreFormerPlusPlus' are for **RestoreFormerPlusPlus**.
+- While training 'RestoreFormer' or 'RestoreFormerPlusPlus', *'ckpt_path'* in the corresponding configure files in configs/ sholud be updated with the path of the trained model of 'HQ_Dictionary' or 'ROHQD'.
+
+<!-- ## <a id="metrics">Metrics</a> -->
+## Metrics
     sh scripts/metrics/run.sh
 
 **Note**. 
 - You need to add the path of CelebA-Test dataset in the script if you want get IDD, PSRN, SSIM, LIPIS.
+- Related metric models are in *./experiments/pretrained_models/*
 
 ## Citation
+    @article{wang2022restoreformer++,
+      title={RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Paris},
+      author={Wang, Zhouxia and Zhang, Jiawei and Chen, Tianshui and Wang, Wenping and Luo, Ping},
+      booktitle={arXiv preprint arXiv:2308.07228},
+      year={2023}
+    }
+
     @article{wang2022restoreformer,
       title={RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs},
       author={Wang, Zhouxia and Zhang, Jiawei and Chen, Runjian and Wang, Wenping and Luo, Ping},
       booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
       year={2022}
     }
 
-## Acknowledgement
-We thank everyone who makes their code and models available, especially [Taming Transformer](https://github.com/CompVis/taming-transformers), [basicsr](https://github.com/XPixelGroup/BasicSR), and [GFPGAN](https://github.com/TencentARC/GFPGAN).
+<!-- ## Acknowledgement
+We thank everyone who makes their code and models available, especially [Taming Transformer](https://github.com/CompVis/taming-transformers), [basicsr](https://github.com/XPixelGroup/BasicSR), and [GFPGAN](https://github.com/TencentARC/GFPGAN). -->
 
 ## Contact
-For any question, feel free to email `[email protected]` or `[email protected]`.
+For any question, feel free to email `[email protected]` or `[email protected]`.

scripts/metrics/run.sh

@@ -13,9 +13,6 @@ test_image=$test_name'_'$tag'/restored_faces'
 out_name=$test_name
 need_post=1
 
-
-
-
 # FID
 python -u scripts/metrics/cal_fid.py \
 $root'/'$test_image \