Optical Flow and object tracker using Lucas-Kanade Algorithm

The Lucas-Kanade method is used for optical flow estimation, to track desired features in a video. The project aims to implement the Lucas-Kanade (LK) template tracker.

• The LK tracker that is implemented here is used to track:
  • A car moving on a road,
  • Face of a baby fighting with a dragon,
  • Usain Bolt running on a track, respectively in three video sequences.

Approach and Implementation:

• To initialize the tracker a template is defined by drawing a bounding box around the object to be tracked in the first frame of the video.
• For each of the subsequent frames the tracker updates an affine transform that warps the current frame so that the template in the first frame is aligned with the warped current frame.
• At the core of the algorithm is a function affineLKtracker(img, tmp, rect, p_prev).
• This function gets as input a grayscale image of the current frame (img), the template image (tmp), the bounding box (rect) that marks the template region in tmp, and the parameters p_prev of the previous warping.
• The function iteratively (gradient descent) computes the new warping parameters p_new, and returns these parameters.
• The algorithm computes the affine transformations from the template to every frame in the sequence and draws the bounding boxes of the rectangles warped from the first frame.
• Elaborate explanation about the approach and the pipeline can be found in the report

Output:

Tracking moving car (video here):

Tracking baby face (video here):

Tracking running Bolt (video here):

Instructions to run the code:

Input dataset required for the code is present in:

• Car dataset
• Baby dataset
• Bolt dataset

Go to directory: cd 'Code/

• To run the car tracker run:

  • $ python trackCar.py

• To run the baby face tracker run:

  • $ python trackBaby.py

• To run the Bolt tracker run:

  • $ python trackBolt.py

References:

• Simon and Matthew’s paper - Lucas-Kanade 20 Years On: A Unifying Framework
• Lecture on Lucas-Kanade Tracker (KLT) by Dr. Mubarak Shah (University of Florida)
• Optical Flow with Lucas-Kanade method - OpenCV 3.4 with Python 3
• Optical-Flow using Lucas Kanade for Motion Tracking