The 'heart2019' tag is stable https://github.com/lp6m/ImageDetectionHW2/tree/heart2019
HOG/BGRHSV + SVM FPGA Implementation of Object Detection
This system detects red traffic signal from USB webcam captured image in real time.
System runs on Debian/Ubuntu OS on Ultra96(rev1) development board. HW Object Detection Accelerator is implemented in FPGA.
HW Accelerator is developed by HLS, which calculates the probability of being a red traffic signal for 891(32pix*64pix) regions in (240pix*320pix) BGR image.
Detail of Algorithm is described in hls/README.md
- HLS IP Latency: min:153838 max:196936 (not fully pipelined)
- Achieved FPS: pver 142fps (including DMA data transmission time, more than 200 times faster than SW)
This system is equipped with the miniature robot car.
Youtube Link:
First create SD Card for Ultra96 following this instruction: ZynqMP-FPGA-Linux.
You must install udmabuf device driver following the instruction.
Japanese instruction is here.
After you finish setup of Debian OS on Ultra96, install OpenCV from source code like this. In my environment, I use OpenCV-3.4.1 version. It takes a while.
Copy overlay/
and app/
directory of this repository to rootfs partition of your SD card.
Steps after this should be done on Ultra96 board.
Connect Ultra96 board by USB-serial port or ssh connection.
fpga@debian-fpga$ sudo -s
root@debian-fpga$ cd ~/overlay
root@debian-fpga$ cp fpga.bin /lib/firmware/
root@debian-fpga:~/overlay# ./init.sh
hog_svm.dtb: Warning (unit_address_vs_reg): /fragment@0/__overlay__/hog_svm_0: node has a reg or ranges property, but no unit name
dma0.dtb: Warning (unit_address_vs_reg): /fragment@0/__overlay__/axi_dma_0: node has a reg or ranges property, but no unit name
FPGA configuration is done by this command.
root@debian-fpga:~# cd ~/app/hog_svm_test
root@debian-fpga:~/app/hog_svm_test# sh compile.sh
root@debian-fpga:~/app/hog_svm_test# ./a.out
1.6268558502 0.8357384801
2.9449763298 0.9500254989
2.1004710197 0.8909489512
elapsed:6.0000000000[milisec]
fps:166.6666666667[fps]
result.png
is generated as output. This file should be like this.
First plug USB webcam to Ultra96.
root@debian-fpga:~# cd ~/app/realtime_webcam
root@debian-fpga:~/app/realtime_webcam# sh compile.sh
root@debian-fpga:~/app/realtime_webcam# ./a.out
If red traffic signal is detected, the probability is printed to standard output.
To avoid false positive, detection region is used.
In this project, trained parameters of SVM used in HLS IP are saved in BRAM, and you can update parameter from SW via AXI Bram Controller.
Modify self.load_saved
in python/vdtools.py
to False
, and run python/train.py
Run python/myestimator.py
. This code extracts trained SVM paramter. Output file is genrated to output/weights.h
.
HLS IP uses 32bit fixed point (ap_fixed<32,10>). When updating the SVM weight from SW, the weight is expressed as a 32-bit unsigned int value. Therefore, it is necessary to convert the 32-bit fixed point to the 32-bit unsigned int of the same bit string.
Copy python/output/weights.h
to util/ap_fixed_convert/weight.h
. And run ap_fixed_convert on Vivado HLS, copy json string output to app/hog_svm_test/weights.json
.
Training data. This files are used python/vdtools.py
python and scikit-learn library are used to train, verify accuracy and export trained weight to cpp header file.
detail description is in python/README.md
HOG + SVM Vivado HLS project.
The detail description of algorithm used is in hls/README.md
You can create project by running create_project.tcl
from Vivado 2018.2
Some SW-level applications to verify porting from python to C++.
detail description is in cpp/README.md
FPGA bistream is configured and IPs are registered as generic-uio
device by Linux Device Tree Overlay function.
This directory contains bitstream, device tree file, and initialize shell script.
These application only works on Ultra96, containing interface layer to call FPGA IP.
Utility program.
This SVM Classifier tends to detect false positive. This may be because of the limit of Linear SVM model.