Please use this identifier to cite or link to this item:
https://doi.org/10.21256/zhaw-26950| Publication type: | Conference paper |
| Type of review: | No review |
| Title: | Real-time image signal processor for SoC/FPGA with direct GPU communication |
| Authors: | Weiss, Richard Gelke, Hans |
| et. al: | No |
| DOI: | 10.21256/zhaw-26950 |
| Proceedings: | Proceedings of Embedded World Conference 2022 |
| Editors of the parent work: | Sikora, Axel |
| Page(s): | 666 |
| Pages to: | 669 |
| Conference details: | Embedded World Conference, Nuremberg, Germany, 21-23 June 2022 |
| Issue Date: | 23-Jun-2022 |
| Publisher / Ed. Institution: | WEKA |
| ISBN: | 978-3-645-50194-1 |
| Language: | English |
| Subjects: | ISP; FPGA; Realtime; Signal processing; Videoprocessing; Image signal processor; fpd-link; pci-e |
| Subject (DDC): | 621.3: Electrical, communications, control engineering |
| Abstract: | Many video applications require to capture images in a dynamic environment. Embedded cameras can be configured to adapt to these environments. However, the algorithm that controls these settings is often left up to the user. Additionally, no tools for preprocessing and calibration are provided as the implementation of these tools is time-consuming, especially in an FPGA environment. In this paper, an Image Signal Processor (ISP) for Xilinx FPGAs is discussed. The goal was to create a versatile toolset to improve image quality regarding the needs of typical video applications. Thus, the following features were implemented. First, automatic white balancing and automatic gain control were added to ensure that the color tone and the exposure of the captured video adapt to the environment in which the camera is used. Additionally, non-linearities and offsets of each color channel caused by the image sensor can be corrected by lookup tables. Moreover, classic image processing algorithms, such as blurring or edge detection, can be applied using the specifically designed configurable convolution filter. Furthermore, the perceived sharpness of the video stream can be increased by the sharpening feature. Finally, for grayscale videos, a histogram equalization can be used to increase the overall contrast, which is interesting for machine learning or medical applications. A glass- to-glass demonstration was built to test the ISP implementation. The demonstration consists of a CSI (Camera Serial Interface) camera connected to a video capturing card featuring a Xilinx Ultracale+ MPSoC FPGA on which the ISP was implemented. The capture card is connected to the PCIe port of a host PC, which displays the video. For the data transfer between the FPGA and the host computer, a frame-based DMA was developed. This DMA has the capability to transfer the data directly into the memory that is accessible from the GPU. The tests revealed that all developed features perform as intended, verifying the proper implementation. The various tools provided by the ISP are a solid basis for any video application. In addition, the modular design of the video pipeline and the used Xilinx AXI-Stream video interface allows adding future features with minimal effort. |
| URI: | https://digitalcollection.zhaw.ch/handle/11475/26950 |
| Fulltext version: | Published version |
| License (according to publishing contract): | Licence according to publishing contract |
| Departement: | School of Engineering |
| Organisational Unit: | Institute of Embedded Systems (InES) |
| Appears in collections: | Publikationen School of Engineering |
Files in This Item:
| File | Description | Size | Format | |
|---|---|---|---|---|
| 2022_Weiss-Gelke_Real-time-image-signal-processor-for-SoC-FPGA_EmbeddedWorld.pdf | 325.77 kB | Adobe PDF |  View/Open |
Show full item record
Weiss, R., & Gelke, H. (2022). Real-time image signal processor for SoC/FPGA with direct GPU communication [Conference paper]. In A. Sikora (Ed.), Proceedings of Embedded World Conference 2022 (pp. 666–669). WEKA. https://doi.org/10.21256/zhaw-26950
Weiss, R. and Gelke, H. (2022) ‘Real-time image signal processor for SoC/FPGA with direct GPU communication’, in A. Sikora (ed.) Proceedings of Embedded World Conference 2022. WEKA, pp. 666–669. Available at: https://doi.org/10.21256/zhaw-26950.
R. Weiss and H. Gelke, “Real-time image signal processor for SoC/FPGA with direct GPU communication,” in Proceedings of Embedded World Conference 2022, Jun. 2022, pp. 666–669. doi: 10.21256/zhaw-26950.
WEISS, Richard und Hans GELKE, 2022. Real-time image signal processor for SoC/FPGA with direct GPU communication. In: Axel SIKORA (Hrsg.), Proceedings of Embedded World Conference 2022. Conference paper. WEKA. 23 Juni 2022. S. 666–669. ISBN 978-3-645-50194-1
Weiss, Richard, and Hans Gelke. 2022. “Real-Time Image Signal Processor for SoC/FPGA with Direct GPU Communication.” Conference paper. In Proceedings of Embedded World Conference 2022, edited by Axel Sikora, 666–69. WEKA. https://doi.org/10.21256/zhaw-26950.
Weiss, Richard, and Hans Gelke. “Real-Time Image Signal Processor for SoC/FPGA with Direct GPU Communication.” Proceedings of Embedded World Conference 2022, edited by Axel Sikora, WEKA, 2022, pp. 666–69, https://doi.org/10.21256/zhaw-26950.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.