Implementing Traffic Sign Classification in Python for Computer Vision tasks in Autonomous Vehicles.
Test online here
Training YOLO v3 for Objects Detection with Custom Data. Build your own detector by labelling, training and testing on image, video and in real time with camera. Join here: https://www.udemy.com/course/training-yolo-v3-for-objects-detection-with-custom-data/
Detections on video are shown below. Trained weights for detection tasks can be found in the course mentioned above.
Explore dataset used for training and detection tasks here: https://www.kaggle.com/valentynsichkar/traffic-signs-dataset-in-yolo-format
https://www.udemy.com/course/training-yolo-v3-for-objects-detection-with-custom-data/
Short description of the content. Full codes you can find inside the course by link above:
- Loading Data
- Preprocessing Data
- Model 1
- Predicting with image from test dataset
- Predicting with user's image
- Traffic Sign Classification in Real Time
Data used for this task is German Traffic Sign Benchmarks (GTSRB).
Initially dataset consists of images in PPM format with different sizes.
For current task datasets were organized in the same way as it was done for CIFAR-10 Image Classification:
- x_train, x_validation, x_test - 4D tensors as numpy.ndarray type with shapes (12345, 3, 32, 32)
- y_train, y_validation, y_test - 1D tensors as numpy.ndarray type with shapes (12345, )
Here,
12345 - number of images / labels,
3, 32, 32 - image with 3 channels and size of 32x32 (height and width).
All tensors were put in a dictionary and were written in a pickle file:
d = {'x_train': x_train, 'y_train': y_train,
'x_validation': x_validation, 'y_validation': y_validation,
'x_test': x_test, 'y_test': y_test}Examples of Unique Traffic Signs for every class from Training dataset are shown on the figure below.
Histogram of 43 classes for training dataset with their number of examples for Traffic Signs Classification before and after Equalization by adding transformated images (brightness and rotation) from original dataset is shown on the figure below. After Equalization, training dataset has increased up to 86989 examples.
Examples of Good Quality Traffic Signs for every class to show in GUI for driver are shown on the figure below.
Following tabel represents number of class and its corresponding label (description).
| Class | Description |
|---|---|
| 0 | Speed limit (20km/h) |
| 1 | Speed limit (30km/h) |
| 2 | Speed limit (50km/h) |
| 3 | Speed limit (60km/h) |
| 4 | Speed limit (70km/h) |
| 5 | Speed limit (80km/h) |
| 6 | End of speed limit (80km/h) |
| 7 | Speed limit (100km/h) |
| 8 | Speed limit (120km/h) |
| 9 | No passing |
| 10 | No passing for vehicles over 3.5 metric tons |
| 11 | Right-of-way at the next intersection |
| 12 | Priority road |
| 13 | Yield |
| 14 | Stop |
| 15 | No vehicles |
| 16 | Vehicles over 3.5 metric tons prohibited |
| 17 | No entry |
| 18 | General caution |
| 19 | Dangerous curve to the left |
| 20 | Dangerous curve to the right |
| 21 | Double curve |
| 22 | Bumpy road |
| 23 | Slippery road |
| 24 | Road narrows on the right |
| 25 | Road work |
| 26 | Traffic signals |
| 27 | Pedestrians |
| 28 | Children crossing |
| 29 | Bicycles crossing |
| 30 | Beware of ice/snow |
| 31 | Wild animals crossing |
| 32 | End of all speed and passing limits |
| 33 | Turn right ahead |
| 34 | Turn left ahead |
| 35 | Ahead only |
| 36 | Go straight or right |
| 37 | Go straight or left |
| 38 | Keep right |
| 39 | Keep left |
| 40 | Roundabout mandatory |
| 41 | End of no passing |
| 42 | End of no passing by vehicles over 3.5 metric tons |
Prepared data is preprocessed in variety of ways and appropriate datasets are written into 'pickle' files.
Datasets data0 - data3 have RGB images and datasets data4 - data8 have Gray images.
- data0.pickle - Shuffling
- data1.pickle - Shuffling, /255.0 Normalization
- data2.pickle - Shuffling, /255.0 + Mean Normalization
- data3.pickle - Shuffling, /255.0 + Mean + STD Normalization
- data4.pickle - Grayscale, Shuffling
- data5.pickle - Grayscale, Shuffling, Local Histogram Equalization
- data6.pickle - Grayscale, Shuffling, Local Histogram Equalization, /255.0 Normalization
- data7.pickle - Grayscale, Shuffling, Local Histogram Equalization, /255.0 + Mean Normalization
- data8.pickle - Grayscale, Shuffling, Local Histogram Equalization, /255.0 + Mean + STD Normalization
Shapes of data0 - data3 are as following (RGB):
- x_train: (86989, 3, 32, 32)
- y_train: (86989,)
- x_validation: (4410, 3, 32, 32)
- y_validation: (4410,)
- x_test: (12630, 3, 32, 32)
- y_test: (12630,)
Shapes of data4 - data8 are as following (Gray):
- x_train: (86989, 1, 32, 32)
- y_train: (86989,)
- x_validation: (4410, 1, 32, 32)
- y_validation: (4410,)
- x_test: (12630, 1, 32, 32)
- y_test: (12630,)
Examples of some of them (RGB, Gray, Local Histogram Equalization) are shown on the figure below:
For Model 1 architecture will be used as it was done for CIFAR-10 Image Classification:
Input --> Conv --> ReLU --> Pool --> Affine --> ReLU --> Affine --> Softmax
For Model 1 following parameters will be used:
| Parameter | Description |
|---|---|
| Weights Initialization | HE Normal |
| Weights Update Policy | Adam |
| Activation Functions | ReLU |
| Regularization | L2 |
| Pooling | Max |
| Loss Functions | Softmax |
For Overfitting Small Data of Model 1 dataset 'data8.pickle' was chosen. Overfitting Small Data with 10 training examples and 100 epochs is shown on the figure below.
For training Model 1 dataset 'data8.pickle' was chosen as it reached the best accuracy over all datasets.
Model 1 with 'data8.pickle' dataset reached 0.989 training accuracy.
Training process of Model 1 with 17 500 iterations is shown on the figure below.
Accuracy for different datasets is shown on the figure below.
Table with training results is shown on the figure below.
| Dataset | Training Accuracy | Validation Accuracy |
|---|---|---|
| data2 | 0.977 |
0.881 |
| data3 | 0.983 |
0.889 |
| data7 | 0.988 |
0.943 |
| data8 | 0.989 |
0.924 |
Initialized and Trained filters for CNN Layer are shown on the figure below.
Feature maps of trained network for CNN Layer are shown on the figure below.
Prediction with image from Test Dataset 'data8.pickle' is shown on the figure below.
Classified as Speed limit (60km/h).
Prediction with user's image is shown on the figure below.
Classified as Keep right.
Traffic Sign Classification with Convolutional Neural Network.
Left: Original frame with Detected Sign.
Upper Right: Cut frame with Detected Sign.
Lower Right: Classified frame by ConvNet according to the Detected Sign.
Valentyn N Sichkar. Neural Networks for computer vision in autonomous vehicles and robotics // GitHub platform. DOI: 10.5281/zenodo.1317904