Finding Lane Lines on the Road

By Kiril Cvetkov

---

Finding Lane Lines on the Road

The goals / steps of this project are the following:

• Building a pipeline that finds lane lines on the road
• Trying the pipeline on real videos

Youtube video

---

Reflection

1. Pipeline description. Briefly description of how the pipeline works.

My pipeline consisted of the following steps:

• Image Transformations for extracting useful structural information from the image and dramatically reduce the amount of data to be processed:

  • Transforming the image to gray-scale
  • Applying Gaussian blur to smooth the image in order to remove the noise
  • Applying Canny transform for edge detection

• Extracting Mask from region of interest, the region that will contain the lane lines

• Applying Hough transformation on the extracted mask in order to get all the candidate lines contained in the region of interest

  • Line noise removal

    After applying hough transform, we obtain the lines formed from the edges.
    A line y=mx+b is represented as (m,b), where m is the slope and b is the intercept.

    

    As we can see from the picture if we plot all the candidate lines as a points in 2d Axis, there may be a noise points, simplified we can consider them as a points that are not representing the lane lines which means their slope and intercept differs a lot and are far away from the mean.
    Below is the proposed condition for outlier detection:

  

    `m` : prameter that tells how much we scale from standard deviation
  
    `μ` : calculated mean for dataset attribute (slope, intercept)
  
    `X` : observed attribute sample (slope, intercept)
  
    `σ` : standard deviation for dataset attribute (slope, intercept)
  

  • Adding the new lines inside the frame buffer

    Once we cleaned the noise lines from the candidate lines, we should see how much they fit with all the previous lines which were inserted in the past, inside the buffer.
    If the candidate line differs a lot from the lines in the buffer, we should
    exclude the candidate, otherwise we should include them in the buffer.

  • We get the last 20 lines inserted from the buffer, and we can construct the average line

    • Mean slope calculation
    • Mean intersect calculation

  • We plot the line on our current mask

• We merge the mask together with our picture

You can see the image below, describing how the pipeline works, containing activity flow diagram

2. Potential shortcomings with my current pipeline

This pipeline is not robust to the following conditions :

• If there is huge amount of noise in the region of interest (edges which are not produced from the lanes), the algorithm
  cannot decide which one represent the lines
• Drastic changes in the lanes direction, if we are driving around twisty roads, this system won't be capable
  to detect the lines, since we are using averages from the past frames,and also the drastic changes will be considered
  as an outliers(noise)

3. Possible improvements to my pipeline

• We can obtain some slight improvements if we use Linear regression or for example Nearest Neighbour heuristic can help to find the hough point that has the most neighbours, so we can draw that point instead of calculating average, but it won't solve the main problems above.
• A possible solution would be the using of non-linear models that can learn how one lane is represented in the road, considering not only the edges, but many features as well.
  I think that Neural networks, especially Convolutional Neural Networks can be trained to get all the regions(anchors) representing the lanes, so we draw curve among the anchors, with using spline interpolations or similar numeric methods.
• I think that this model will solve the shortcomings mentioned before, the best power is that we can train the model with data generated from different conditions with different type of noise,
  so our system can be more robust and if properly trained, it can adapt to all drastic changes that can happen.