Road_Segmentation_of_Arena_Image_using_U-Net

Testing: Road Segmentation of Arena Image using U-Net

• Training:
• Testing:

This notebook is used to generate Road Segmentation Mask of Arena Image using Transfer Learning Technique, utilizing U-Net as the architechure of the Deep-Learning Model

Make sure you use GPU runtime for this notebook. For Google Colab, go to Runtime -> Change runtime type and select GPU as the hardware accelerator. As using only CPU runtime will be very slow for training model.

Table of Contents

• Testing: Road Segmentation of Arena Image using U-Net
• Table of Contents
• U-Net
• UNET - Network Architecture
• Errors
• Prediction with Smooth Blending
• Training Curves
• Output
• Training Metrics
• Video of Progress Over Epochs
• References

U-Net

U-Net was developed by Olaf Ronneberger et al. for Bio Medical Image Segmentation. The architecture contains two paths. First path is the contraction path (also called as the encoder) which is used to capture the context in the image. The encoder is just a traditional stack of convolutional and max pooling layers. The second path is the symmetric expanding path (also called as the decoder) which is used to enable precise localization using transposed convolutions. Thus, it is an end-to-end fully convolutional network (FCN), i.e. it only contains Convolutional layers and does not contain any Dense layer because of which it can accept image of any size.

In the original paper, the UNet is described as follows:

U-Net architecture (example for 32x32 pixels in the lowest resolution). Each blue box corresponds to a multi-channel feature map. The number of channels is denoted on top of the box. The x-y-size is provided at the lower left edge of the box. White boxes represent copied feature maps. The arrows denote the different operations.

UNET - Network Architecture

UNET is a U-shaped encoder-decoder network architecture, which consists of four encoder blocks and four decoder blocks that are connected via a bridge. The encoder network (contracting path) half the spatial dimensions and double the number of filters (feature channels) at each encoder block. Likewise, the decoder network doubles the spatial dimensions and half the number of feature channels.

• Encoder Network The encoder network acts as the feature extractor and learns an abstract representation of the input image through a sequence of the encoder blocks. Each encoder block consists of two 3x3 convolutions, where each convolution is followed by a ReLU (Rectified Linear Unit) activation function. The ReLU activation function introduces non-linearity into the network, which helps in the better generalization of the training data. The output of the ReLU acts as a skip connection for the corresponding decoder block. Next, follows a 2x2 max-pooling, where the spatial dimensions (height and width) of the feature maps are reduced by half. This reduces the computational cost by decreasing the number of trainable parameters.
• Skip Connections These skip connections provide additional information that helps the decoder to generate better semantic features. They also act as a shortcut connection that helps the indirect flow of gradients to the earlier layers without any degradation. In simple terms, we can say that skip connection helps in better flow of gradient while backpropagation, which in turn helps the network to learn better representation.
• Bridge The bridge connects the encoder and the decoder network and completes the flow of information. It consists of two 3x3 convolutions, where each convolution is followed by a ReLU activation function.
• Decoder Network The decoder network is used to take the abstract representation and generate a semantic segmentation mask. The decoder block starts with a 2x2 transpose convolution. Next, it is concatenated with the corresponding skip connection feature map from the encoder block. These skip connections provide features from earlier layers that are sometimes lost due to the depth of the network. After that, two 3x3 convolutions are used, where each convolution is followed by a ReLU activation function. The output of the last decoder passes through a 1x1 convolution with sigmoid activation. The sigmoid activation function gives the segmentation mask representing the pixel-wise classification.

Errors

If the following error occurs:

Go to File: /usr/local/lib/python3.10/dist-packages/efficientnet/keras.py and change the following lines

• from . import inject_keras_modules, init_keras_custom_objects ---> from . import inject_keras_modules, init_tfkeras_custom_objects
• init_keras_custom_objects() ---> init_tfkeras_custom_objects()

Prediction with Smooth Blending

• Make smooth predictions by blending image patches, such as for image segmentation, rather than jagged ones.
• One challenge of using a U-Net for image segmentation is to have smooth predictions, especially if the receptive field of the neural network is a small amount of pixels.

Training Curves

Training and Validation Loss

Training and Validation IOU

Output:

• Since we only want to extract the Road Segmentation, we will utilize only the Road-Pixel Values from one-hot encoded mask, while omiting the rest of the segmented pixel values
• Do Note on testing, we deducted that for Prediction of Road on Arena Image: Land Class of Training Data is providing most-optimized output
• But Colour of #6EC1E4 (ROAD) used only for visualization

Original Input Image

No. of Epochs	Predicted Mask without Smooth Blending	Predicted Mask with Smooth Blending
50 Epochs
100 Epochs
150 Epochs
200 Epochs

Training Metrics

No. of Epoch	Loss	Accuracy	Jaccard Coefficient	Validation Loss	Validation Accuracy	Validation Jaccard Coefficient
50	0.9039	0.8566	0.6968	0.9266	0.8124	0.6594
100	0.8748	0.9012	0.7831	0.9456	0.7970	0.6562
150	0.8608	0.9318	0.8472	0.9180	0.8515	0.7443
200	0.8554	0.9407	0.8656	0.9174	0.8531	0.7465

Video of Progress Over Epochs

• Video of Progress Over Epochs

References

• U-Net: Convolutional Networks for Biomedical Image Segmentation
• drone-images-semantic-segmentation
• Dataset: Semantic segmentation of aerial imagery
• DigitalSreeni: 228 - Semantic segmentation of aerial (satellite) imagery using U-net
• Dr. Sreenivas Bhattiprolu: GitHub - 228_semantic_segmentation_of_aerial_imagery_using_unet
• DigitalSreeni: 229 - Smooth blending of patches for semantic segmentation of large images (using U-Net)
• Dr. Sreenivas Bhattiprolu: GitHub - 229_smooth_predictions_by_blending_patches
• DigitalSreeni: Python tips and tricks - 8: Working with RGB (and Hex) masks for semantic segmentation
• U-net for image segmentation
• Vooban: GitHub - Smoothly-Blend-Image-Patches
• Kaggle API
• Medium: What is UNET?
• OpenAerialMap: Arena Image