The goals / steps of this project are the following:
- Perform a Histogram of Oriented Gradients (HOG) feature extraction on a labeled training set of images and train a classifier Linear SVM classifier
- Optionally, you can also apply a color transform and append binned color features, as well as histograms of color, to your HOG feature vector.
- Note: for those first two steps don't forget to normalize your features and randomize a selection for training and testing.
- Implement a sliding-window technique and use your trained classifier to search for vehicles in images.
- Run your pipeline on a video stream (start with the test_video.mp4 and later implement on full project_video.mp4) and create a heat map of recurring detections frame by frame to reject outliers and follow detected vehicles.
- Estimate a bounding box for vehicles detected.
Rubric Points
Here I will consider the rubric points individually and describe how I addressed each point in my implementation.
1. Explain how (and identify where in your code) you extracted HOG features from the training images.
The code for this step is contained in the third code cell of the IPython notebook.
I started by reading in all the vehicle and non-vehicle images. Here is an example of one of each of the vehicle and non-vehicle classes:
I then explored different color spaces and different skimage.hog() parameters (orientations, pixels_per_cell, and cells_per_block). I grabbed random images from each of the two classes and displayed them to get a feel for what the skimage.hog() output looks like.
Here is an example using the YCrCb color space and HOG parameters of orientations=8, pixels_per_cell=(8, 8) and cells_per_block=(2, 2):
I started with the parameters in Tips and Tricks for the Project section from the Udacity site. These worked well when using a slide window searching even even moreso when I began using HOG subsampling
3. Describe how (and identify where in your code) you trained a classifier using your selected HOG features (and color features if you used them).
In code cell 13 titled Get car and notcar feature sets I trained a linear SVM using spatial, histogram, and HOG features. I found the YCrCb color space gave me the best results with a spatial size of (32, 32) and a histogram bin size of 32.
1. Describe how (and identify where in your code) you implemented a sliding window search. How did you decide what scales to search and how much to overlap windows?
Initially I used a sliding window search that utilized multiple arrays of window sizes and had good results in terms of the output heatmap. However, the efficency was very poor and would have made a pipeline on the video feed extremely slow. As shown in code cell 19, I used the HOG subsampling technique from the Q&A video and improved my searching times significantly.
2. Show some examples of test images to demonstrate how your pipeline is working. What did you do to optimize the performance of your classifier?
Ultimately I searched on two scales, [1.5, 1,75, 2.0], using YCrCb 3-channel HOG features plus spatially binned color and histograms of color in the feature vector, which provided a nice result. Here are some example images:
1. Provide a link to your final video output. Your pipeline should perform reasonably well on the entire project video (somewhat wobbly or unstable bounding boxes are ok as long as you are identifying the vehicles most of the time with minimal false positives.)
Here's a link to my video result
2. Describe how (and identify where in your code) you implemented some kind of filter for false positives and some method for combining overlapping bounding boxes.
I recorded the positions of positive detections in each frame of the video. From the positive detections I created a heatmap and then thresholded that map to identify vehicle positions. I then used scipy.ndimage.measurements.label() to identify individual blobs in the heatmap. I then assumed each blob corresponded to a vehicle. I constructed bounding boxes to cover the area of each blob detected.
Using these heatmaps I stored the past 5 frames at any given point in the video and summed them up. This way I could increase the heatmap threshold and further filter out outliers.
Here's an example result showing the heatmap from a series of frames of video, the result of scipy.ndimage.measurements.label() and the bounding boxes then overlaid on the last frame of video:
1. Briefly discuss any problems / issues you faced in your implementation of this project. Where will your pipeline likely fail? What could you do to make it more robust?
My biggest problem intially was managing the sliding window search effectively, not only from the perspective of how accurate it is but from how performant it was. I was able to get it more performant but at the expense of performance. Performanace degredation was the biggest reason I had to use HOG subsampling.
Another big difficulty was making sure all of the multiple data points were in the format that were correct. Using jpgs and pngs and mpimg and cv2 gave trouble when making sure the formats were exact.
The remainder of the work went into fine tuning the subsampling solution as well as figuring out how many frames to keep track of along with the threshold value to eliminate spurious data.