Rapid Design of Real-Time Image Fusion on FPGA using HLS and Other Techniques

During the process of implementing a parameterized hardware IP generator for an image fusion algorithm, we had a chance to test various tools and techniques such as HLS, pipelining, and PCIe logic/software porting, which we developed in a previous design project. Image fusion combines two or more images through a color transformation process. Depending on the application, different fps and/or resolution may be needed. Yet the specifics of the image-processing algorithm may frequently change causing redesign. If the target platform is FPGA, usually rapid yet optimized hardware implementation is required. All these requirements cannot be met only by HLS. Clever approaches in terms of architectural techniques such as unorthodox ways of pipelining, RTL coding, and creative ways of porting interface logic/software allowed us to meet the requirements outlined above. With all these in our arsenal, we were able to get 3 versions of the algorithm (with different fps and/or resolution) running on Cyclone IV and Arria 10 FPGAs in a fairly short amount of time. This paper explains the image fusion algorithm, our hardware architecture as well as our specific flow for rapid implementation of it.