Real time motion tracking for augmented reality with TOF camera and vulkan rendering

Wegmann, Marcel; Rosenthal, Matthias

doi:10.21256/zhaw-26948

Please use this identifier to cite or link to this item: https://doi.org/10.21256/zhaw-26948

Publication type:	Conference paper
Type of review:	Peer review (abstract)
Title:	Real time motion tracking for augmented reality with TOF camera and vulkan rendering
Authors:	Wegmann, Marcel Rosenthal, Matthias
et. al:	No
DOI:	10.21256/zhaw-26948
Proceedings:	Proceedings of Embedded World Conference 2022
Editors of the parent work:	Sikora, Axel
Page(s):	661
Pages to:	665
Conference details:	Embedded World Conference, Nuremberg, Germany, 21-23 June 2022
Issue Date:	23-Jun-2022
Publisher / Ed. Institution:	WEKA
ISBN:	978-3-645-50194-1
Language:	English
Subjects:	Augmented reality; Time of flight camera; Real-time processing; CUDA; Vulkan; RANSAC; Kalman filter; Motion tracking
Subject (DDC):	006: Special computer methods
Abstract:	Augmented Reality is the concept of enhancing the real world with virtual objects or information with projections into a viewfinder or through specialized goggles. Simpler forms of Augmented Reality – like a heads-up display in a car – do not need to estimate the camera’s motion, an object, or the user. However, more elaborate implementations of Augmented Reality need to track things and, more importantly, the camera’s movement itself. The applications in which Augmented Reality could be leveraged range from social interaction over pedestrian navigation to various use cases in different professions. Multiple companies already have shown closed source or custom-tailored programming interfaces, either running on smartphones or shipped with industry-targeted goggles. The tracking of real-world objects or surfaces is possible with the provided interfaces, but the algorithms behind the different functions are corporate secrets. This paper describes an approach for an end-to-end pipeline in a prototype of an Augmented Reality platform without using commercial interfaces. A time-of-flight camera provides a depth-image that allows reconstruction of the recorded scene as a cloud of SIFT features. Frame-by-frame analysis of the point cloud estimates the camera’s motion by highly parallel processing and a three-dimensional extension of the RANSAC algorithm. An accelerometer and a gyroscope provide additional data, fused with a Kalman filter to improve the motion estimation. A regular color camera acts as a viewfinder, and Vulkan renders the result to a monitor. Enhancing the matching quality of SIFT features between consecutive frames of a time-of-flight camera using a three-dimensional RANSAC algorithm led to over two times as many correct matches.
URI:	https://digitalcollection.zhaw.ch/handle/11475/26948
Fulltext version:	Published version
License (according to publishing contract):	Licence according to publishing contract
Departement:	School of Engineering
Organisational Unit:	Institute of Embedded Systems (InES)
Appears in collections:	Publikationen School of Engineering

Files in This Item:

File	Description	Size	Format
2022_Wegmann-Rosenthal_Real-time-motion-tracking-for-augmented-reality_EmbeddedWorld.pdf		654.45 kB	Adobe PDF	View/Open

Show full item record

Wegmann, M., & Rosenthal, M. (2022). Real time motion tracking for augmented reality with TOF camera and vulkan rendering [Conference paper]. In A. Sikora (Ed.), Proceedings of Embedded World Conference 2022 (pp. 661–665). WEKA. https://doi.org/10.21256/zhaw-26948

Wegmann, M. and Rosenthal, M. (2022) ‘Real time motion tracking for augmented reality with TOF camera and vulkan rendering’, in A. Sikora (ed.) Proceedings of Embedded World Conference 2022. WEKA, pp. 661–665. Available at: https://doi.org/10.21256/zhaw-26948.

M. Wegmann and M. Rosenthal, “Real time motion tracking for augmented reality with TOF camera and vulkan rendering,” in Proceedings of Embedded World Conference 2022, Jun. 2022, pp. 661–665. doi: 10.21256/zhaw-26948.

WEGMANN, Marcel und Matthias ROSENTHAL, 2022. Real time motion tracking for augmented reality with TOF camera and vulkan rendering. In: Axel SIKORA (Hrsg.), Proceedings of Embedded World Conference 2022. Conference paper. WEKA. 23 Juni 2022. S. 661–665. ISBN 978-3-645-50194-1

Wegmann, Marcel, and Matthias Rosenthal. 2022. “Real Time Motion Tracking for Augmented Reality with TOF Camera and Vulkan Rendering.” Conference paper. In Proceedings of Embedded World Conference 2022, edited by Axel Sikora, 661–65. WEKA. https://doi.org/10.21256/zhaw-26948.

Wegmann, Marcel, and Matthias Rosenthal. “Real Time Motion Tracking for Augmented Reality with TOF Camera and Vulkan Rendering.” Proceedings of Embedded World Conference 2022, edited by Axel Sikora, WEKA, 2022, pp. 661–65, https://doi.org/10.21256/zhaw-26948.