In this project we wanted to test the ability of the CounTR (Transformer-based Generalised Visual Counting) on a newly created different dataset to see if the performance would've been great also with noisy images and on objects with a very small patch size.
The dataset was made from scratch (code) starting from a FITS Image and the related segmentation mask of 25 Mpx. The Image was patched and for each object the central pixel was annotated in a numpy binary array, while for training starting, from the groundtruth array, a gaussian filter was applied after multiplying the pixels x60 in order to avoid sparsity.
The dataset has been made publicy available on Kaggle at the following link: Galaxy Counting Dataset
Original Paper Pipeline (folder)
In the original paper the model is trained on augmented images (mixing, random cropping, blending) and tested by cropping the image and summing the counts on the individual patches.
Starting from the pretrained weights, performing image reconstruction, on DOTA and CARPK, the model was finetuned for 100 epochs on the Galaxy dataset.
Galaxy-CounTR (folder)
In the personalized pipeline has been used a different augmentation pipeline (random flip, color jittering, Random Noise) and were tested multiple loss functions and Learning rate Schedulers.
The code has been fully re-implemented using PyLightning both for Training and Testing.
All the results and observations have been reported in an exhaustive report about the project, also the data preparation and ablation studies are reported in order to give a better understanding of the workflow.
For sake of curiosity this is the comparison between the model trained with the original pipeline vs our implementation:
Original Paper: CounTR: Transformer-based Generalised Visual Counting
Paper Repository: Verg-Avesta/CounTR
Paper Project Site: https://verg-avesta.github.io/CounTR_Webpage/
Galaxy Detection Paper: Galaxy detection and identification using deep learning and data augmentation
