Vehicle, Pedestrian Detection with IR Image
Last updated
Last updated
Date: 2021-6-21
Author: 김도연, 이예
Github:
Demo Video:
This tutorial explains the FLIR image object detection using yolov5. If you want to know how to install yolov5 in your desktop, referring to this site .
This report consists of five parts.
Why do I use FLIR camera
Train FLIR dataset and obtain the training 'weight'
The simple usage of FLIP camera (FLIR A60)
Pre-processing codes
Discussion
Wanting to know only the way to train dataset, you can only refer to step 2, (Train FLIR dataset and obtain the training 'weight')
At night, it is hard to object detection because it is difficult to distinguish objects from common cameras.
But FLIR camera not affected by light
These camera can see through smoke, fog, haze, and other atmospheric obscurants better than a visible light camera can.
Before you get your dataset from Google drive, you should upload your data in your own Google drive.
For example, in this tutorial, I made data folder named 'juho' in my google drive. In 'juho', there are 'images' folder which has training image set and validateion image set and 'lables' folder which has lables of training images and labels of validation images.
Figure below shows the subfolers of 'juho' folder!
If you run this and login your ID, you will get the authorizing code
Then you can get the "drive" folder and you can access your data in Google drive
'yaml' file includes the imformation of your dataset.
'train' in 'yaml' indicates the directory of your training images!
'val' in 'yaml' indicates the directory of your validating images!
'nc' means the number of class that your data file detecting
'names' means the sort of classes your data file detecting
This stage is making image lists for your image data!
'train_img' is a training image list
'val_image' is a validating image list
In this tutorial, the image files used are '.jpg' format, you can change it to your own file format!
You should modify directory of your 'train_img' and 'val_img' in 'yaml' file (in Check yaml file session). Because directories of data files changed in Google drive. This code make the directory (path of your files) as a .txt file.
Using the 'train.txt' and 'val.txt'(in Making path text files session), you can change your 'train' and 'val' directory in 'yaml' file.
This stage is 'training stage'. If you run this code, you can get a 'weight' file for your own dataset. There are some parameters you should know.
'--img' : the image size of your data
'--batch' : set of data calculated per process.
'--epochs' : total number of data training
'--data' : the directory of your 'yaml' data file
'--cfg' : the structure of the weight model(in this tutorial 'yolo5x.yaml' used)
'--weights' : training weight (in this tutorial 'yolo5x' used)
'--name' : the name of your own weight file, the result of this training
You can move 'best.pt' file to your desktop YOLOv5 folder.
Finally, you can run your code with your own weight file!!!
Install the ebus_runtime_32-bit.4.1.6.3809.exe and run the PvSimpleUISample.exe
After, click the Select/Connect button, then connect your FLIR camera
Anaconda Prompt allows dircet access to the yolov5 file to modify the code. It can modify overall extent of yolov5 before object detection, such as what object to find. We did the following setting: 1. Set Roi to detect only the objects of lane where this vehicle is located 2. Approximate distance measurement - Display warnining message about 2m when a person or car in front of the car
Set roi region like this, in order to get the information needed only from the front of the vehicle when driving.
If the object is not in range, do not detect it. More clear detection is possible by blocking unnecessary information (such as vehicles in other lanes).
If a person or other vehicle is too close to the front of the vehicle, warning message is displayed on the screen. The distance was set at approximately 2 m.
In detect.py code, the xyxy variable is the coordinate value of the upper left and lower right points of the bounding box being detected. This allows the warning message to be displayed when the y-value of the lower right-hand spot is 30 percent from the bottom of the screen, which is thought to be 2 meters from the camera.
In this step, the overall assesment wil be conducted.
if you look at the above figure, it can be seen that there are three objects(1 person ,2 cars),but actually there are only two objects(1 person, 1 car). It means that it detect the wrong object. There are mainly two reason for this error.
Environmental effect
The FLIR measures the infra wave from the object. Therefore, it is affected the environment of the place where images are taken. the training images are taken in California , but the test images are in Pohang. For this reason, this error occurs.
Filter effect
In FLIR camera setting, there is a filter effect. The result of the picture changes depending on which filters user uses. The difference filter effect between training data and testing data make this error.
The picture above and the picture below are pictures with different filters. Because FLIR images are represented by binaries, these filter differences make a very big difference.
The distance measured in this project (2 meters apart from the camera) was about 30 percent of the screen. But this is a very rough value, and it actually needs to be more precise.
When object detection is performed, the bounding box is not exactly fitted to the object.
Only measured values can always be used in fixed positions because the camera can vary at all times depending on the height, angle, etc.
Therefore, we suggest utilizing a combination of sensors other than FLIR cameras (e.g., lidar sensors that read reflected values using laser pulses) to measure the distance precisely.
After finishing your custom training dataset,you should move the 'weight' file to your desktop The result weight file is located at '/content/yolov5/runs/train/FIRL_yolov5x/weights/ (FIRL_yolov5x is the weight file name used in this tutorial)
image from
you can download FLIR software from
download FLIR GEV Dem 1.10.0 version After download this software
