How does training deep neural networks on a biased dataset affect the loss landscape of the network?

This repository contained the code for our project for the course "Lab Development and Application of Data Min-ing and Learning Systems: Machine Learningand Data Mining".

We investigate the loss landscape of a deep neural network trained on a biased dataset by visualizing the loss landscape using "filter-normalized random directions" as described by Li, Hao et al. “Visualizing the Loss Landscape of Neural Nets.” NeurIPS (2018).

Methodology

We trained "ResNet-20" and "ShaResNet-20" several times on "CIFAR-10 Dataset" with different types of bias (mislabeling, gaussian noise and skewness).

Implementations

• We use a slightly modified version of keras example for ResNet and CIFAR-10
• We implemented the visualization method in Tensorflow, its original implementation in PyTorch is available here
• We modified the impleemntation of the ResNet to implement the ShaResNet
• Finally we implemented the logic for adding different biases to the training dataset

Report

For detailed methodology and final results please see our final report.