Fast and Resource Efficient Image Segmentation on Field-Programmable Gate Arrays

open_access
Open Access
Author:
Kang, Joseph
Area of Honors:
Engineering Science
Degree:
Bachelor of Science
Document Type:
Thesis
Thesis Supervisors:
Mohamed Khaled Almekkawy, Thesis Supervisor
Lucas Jay Passmore, Thesis Honors Advisor
Keywords:
Field-Programmable Gate Arrays
High-Level Synthesis
Hardware Acceleration
hls4ml
Deep Learning
Artificial Intelligence
Neural Networks
UNet
Image Segmentation
Convolutional Neural Networks
Abstract:
In recent years, deep learning has achieved state-of-the-art results across a range of domains, revolutionizing several fields. However, deploying neural networks in real-time systems with stringent latency and resource constraints remains a significant challenge owing to the high computational demands of conventional hardware such as CPUs and GPUs. Field-Programmable Gate Arrays (FPGAs) offer a promising alternative because their reconfigurability and high degree of parallelism can deliver extremely low latency and reduced power consumption. Image segmentation, especially in applications such as autonomous vehicles and medical imaging, benefits from FPGA-based neural network inference because of its time-sensitive nature and the frequent use of power-constrained devices. In this study, we demonstrate the feasibility of FPGA deployment for image segmentation by converting a Keras-based UNet model into a Hardware Description Language (HDL) using high-level synthesis (HLS). We trained a compact version of the UNet architecture on three relevant datasets: the Oxford-IIIT Pet dataset, Cardiac Acquisitions for Multi-structure Ultrasound Seg- mentation (CAMUS) dataset, and Multimodal Brain Tumor Segmentation (BraTS) dataset. Despite the model’s reduced size, it maintained high segmentation performance, as measured by the Dice coefficient, even after conversion to fixed-point arithmetic for HLS. The resulting model achieved Dice scores of 0.8396 (BraTS), 0.7615, and 0.7352 for BraTS (Oxford Pets), and 0.7352 (CAMUS). The model was synthesized into HDL (Verilog) using the hls4ml Python library targeting the xcu250-figd2104-2L-e FPGA. Compared to execution on Google Colab’s CPU, T4 GPU, and TPU v2-8, the FPGA implementation achieved dramatically reduced inference latency. For the BraTS and Oxford Pets datasets, inference took less than one millisecond—over 100 times faster than on traditional hardware. For CAMUS, the latency was approximately 8 ms, which was more than 13 times faster than the baseline results. While LUT, FF, URAM, and BRAM_18K usage remained within acceptable limits, the high number of multiplications used in the network caused DSP usage to exceed the available capacity, suggesting the need for further optimization for this specific FPGA device.