- Training:
- Testing:
This notebook is used to generate Road Segmentation Mask of Arena Image using Transfer Learning Technique, utilizing U-Net as the architechure of the Deep-Learning Model
Make sure you use GPU runtime for this notebook. For Google Colab, go to Runtime -> Change runtime type and select GPU as the hardware accelerator. As using only CPU runtime will be very slow for training model.
- Testing: Road Segmentation of Arena Image using U-Net
- Table of Contents
- U-Net
- UNET - Network Architecture
- Errors
- Prediction with Smooth Blending
- Training Curves
- Output
- Training Metrics
- Video of Progress Over Epochs
- References
U-Net was developed by Olaf Ronneberger et al. for Bio Medical Image Segmentation. The architecture contains two paths. First path is the contraction path (also called as the encoder) which is used to capture the context in the image. The encoder is just a traditional stack of convolutional and max pooling layers. The second path is the symmetric expanding path (also called as the decoder) which is used to enable precise localization using transposed convolutions. Thus, it is an end-to-end fully convolutional network (FCN), i.e. it only contains Convolutional layers and does not contain any Dense layer because of which it can accept image of any size.
In the original paper, the UNet is described as follows:
UNET is a U-shaped encoder-decoder network architecture, which consists of four encoder blocks and four decoder blocks that are connected via a bridge. The encoder network (contracting path) half the spatial dimensions and double the number of filters (feature channels) at each encoder block. Likewise, the decoder network doubles the spatial dimensions and half the number of feature channels.
- Encoder Network The encoder network acts as the feature extractor and learns an abstract representation of the input image through a sequence of the encoder blocks. Each encoder block consists of two 3x3 convolutions, where each convolution is followed by a ReLU (Rectified Linear Unit) activation function. The ReLU activation function introduces non-linearity into the network, which helps in the better generalization of the training data. The output of the ReLU acts as a skip connection for the corresponding decoder block. Next, follows a 2x2 max-pooling, where the spatial dimensions (height and width) of the feature maps are reduced by half. This reduces the computational cost by decreasing the number of trainable parameters.
- Skip Connections These skip connections provide additional information that helps the decoder to generate better semantic features. They also act as a shortcut connection that helps the indirect flow of gradients to the earlier layers without any degradation. In simple terms, we can say that skip connection helps in better flow of gradient while backpropagation, which in turn helps the network to learn better representation.
- Bridge The bridge connects the encoder and the decoder network and completes the flow of information. It consists of two 3x3 convolutions, where each convolution is followed by a ReLU activation function.
- Decoder Network The decoder network is used to take the abstract representation and generate a semantic segmentation mask. The decoder block starts with a 2x2 transpose convolution. Next, it is concatenated with the corresponding skip connection feature map from the encoder block. These skip connections provide features from earlier layers that are sometimes lost due to the depth of the network. After that, two 3x3 convolutions are used, where each convolution is followed by a ReLU activation function. The output of the last decoder passes through a 1x1 convolution with sigmoid activation. The sigmoid activation function gives the segmentation mask representing the pixel-wise classification.
If the following error occurs:
Go to File: /usr/local/lib/python3.10/dist-packages/efficientnet/keras.py and change the following lines
from . import inject_keras_modules, init_keras_custom_objects--->from . import inject_keras_modules, init_tfkeras_custom_objectsinit_keras_custom_objects()--->init_tfkeras_custom_objects()
- Make smooth predictions by blending image patches, such as for image segmentation, rather than jagged ones.
- One challenge of using a U-Net for image segmentation is to have smooth predictions, especially if the receptive field of the neural network is a small amount of pixels.
- Since we only want to extract the Road Segmentation, we will utilize only the Road-Pixel Values from one-hot encoded mask, while omiting the rest of the segmented pixel values
- Do Note on testing, we deducted that for Prediction of Road on Arena Image:
LandClass of Training Data is providing most-optimized output - But Colour of #6EC1E4 (ROAD) used only for visualization
| Original Input Image |
|---|
 |
| No. of Epochs | Predicted Mask without Smooth Blending | Predicted Mask with Smooth Blending |
|---|---|---|
| 50 Epochs |  |
 |
| 100 Epochs |  |
 |
| 150 Epochs |  |
 |
| 200 Epochs |  |
 |
| No. of Epoch | Loss | Accuracy | Jaccard Coefficient | Validation Loss | Validation Accuracy | Validation Jaccard Coefficient |
|---|---|---|---|---|---|---|
| 50 | 0.9039 | 0.8566 | 0.6968 | 0.9266 | 0.8124 | 0.6594 |
| 100 | 0.8748 | 0.9012 | 0.7831 | 0.9456 | 0.7970 | 0.6562 |
| 150 | 0.8608 | 0.9318 | 0.8472 | 0.9180 | 0.8515 | 0.7443 |
| 200 | 0.8554 | 0.9407 | 0.8656 | 0.9174 | 0.8531 | 0.7465 |
- U-Net: Convolutional Networks for Biomedical Image Segmentation
- drone-images-semantic-segmentation
- Dataset: Semantic segmentation of aerial imagery
- DigitalSreeni: 228 - Semantic segmentation of aerial (satellite) imagery using U-net
- Dr. Sreenivas Bhattiprolu: GitHub - 228_semantic_segmentation_of_aerial_imagery_using_unet
- DigitalSreeni: 229 - Smooth blending of patches for semantic segmentation of large images (using U-Net)
- Dr. Sreenivas Bhattiprolu: GitHub - 229_smooth_predictions_by_blending_patches
- DigitalSreeni: Python tips and tricks - 8: Working with RGB (and Hex) masks for semantic segmentation
- U-net for image segmentation
- Vooban: GitHub - Smoothly-Blend-Image-Patches
- Kaggle API
- Medium: What is UNET?
- OpenAerialMap: Arena Image