Real time motion tracking for augmented reality with TOF camera and vulkan rendering

Wegmann, Marcel; Rosenthal, Matthias

doi:10.21256/zhaw-26948

Please use this identifier to cite or link to this item: https://doi.org/10.21256/zhaw-26948

Full metadata record

DC Field	Value	Language
dc.contributor.author	Wegmann, Marcel	-
dc.contributor.author	Rosenthal, Matthias	-
dc.date.accessioned	2023-02-15T15:28:23Z	-
dc.date.available	2023-02-15T15:28:23Z	-
dc.date.issued	2022-06-23	-
dc.identifier.isbn	978-3-645-50194-1	de_CH
dc.identifier.uri	https://digitalcollection.zhaw.ch/handle/11475/26948	-
dc.description.abstract	Augmented Reality is the concept of enhancing the real world with virtual objects or information with projections into a viewfinder or through specialized goggles. Simpler forms of Augmented Reality – like a heads-up display in a car – do not need to estimate the camera’s motion, an object, or the user. However, more elaborate implementations of Augmented Reality need to track things and, more importantly, the camera’s movement itself. The applications in which Augmented Reality could be leveraged range from social interaction over pedestrian navigation to various use cases in different professions. Multiple companies already have shown closed source or custom-tailored programming interfaces, either running on smartphones or shipped with industry-targeted goggles. The tracking of real-world objects or surfaces is possible with the provided interfaces, but the algorithms behind the different functions are corporate secrets. This paper describes an approach for an end-to-end pipeline in a prototype of an Augmented Reality platform without using commercial interfaces. A time-of-flight camera provides a depth-image that allows reconstruction of the recorded scene as a cloud of SIFT features. Frame-by-frame analysis of the point cloud estimates the camera’s motion by highly parallel processing and a three-dimensional extension of the RANSAC algorithm. An accelerometer and a gyroscope provide additional data, fused with a Kalman filter to improve the motion estimation. A regular color camera acts as a viewfinder, and Vulkan renders the result to a monitor. Enhancing the matching quality of SIFT features between consecutive frames of a time-of-flight camera using a three-dimensional RANSAC algorithm led to over two times as many correct matches.	de_CH
dc.language.iso	en	de_CH
dc.publisher	WEKA	de_CH
dc.rights	Licence according to publishing contract	de_CH
dc.subject	Augmented reality	de_CH
dc.subject	Time of flight camera	de_CH
dc.subject	Real-time processing	de_CH
dc.subject	CUDA	de_CH
dc.subject	Vulkan	de_CH
dc.subject	RANSAC	de_CH
dc.subject	Kalman filter	de_CH
dc.subject	Motion tracking	de_CH
dc.subject.ddc	006: Spezielle Computerverfahren	de_CH
dc.title	Real time motion tracking for augmented reality with TOF camera and vulkan rendering	de_CH
dc.type	Konferenz: Paper	de_CH
dcterms.type	Text	de_CH
zhaw.departement	School of Engineering	de_CH
zhaw.organisationalunit	Institute of Embedded Systems (InES)	de_CH
dc.identifier.doi	10.21256/zhaw-26948	-
zhaw.conference.details	Embedded World Conference, Nuremberg, Germany, 21-23 June 2022	de_CH
zhaw.funding.eu	No	de_CH
zhaw.originated.zhaw	Yes	de_CH
zhaw.pages.end	665	de_CH
zhaw.pages.start	661	de_CH
zhaw.parentwork.editor	Sikora, Axel	-
zhaw.publication.status	publishedVersion	de_CH
zhaw.publication.review	Peer review (Abstract)	de_CH
zhaw.title.proceedings	Proceedings of Embedded World Conference 2022	de_CH
zhaw.author.additional	No	de_CH
zhaw.display.portrait	Yes	de_CH
Appears in collections:	Publikationen School of Engineering

Files in This Item:

File	Description	Size	Format
2022_Wegmann-Rosenthal_Real-time-motion-tracking-for-augmented-reality_EmbeddedWorld.pdf		654.45 kB	Adobe PDF	View/Open

Show simple item record

Wegmann, M., & Rosenthal, M. (2022). Real time motion tracking for augmented reality with TOF camera and vulkan rendering [Conference paper]. In A. Sikora (Ed.), Proceedings of Embedded World Conference 2022 (pp. 661–665). WEKA. https://doi.org/10.21256/zhaw-26948

Wegmann, M. and Rosenthal, M. (2022) ‘Real time motion tracking for augmented reality with TOF camera and vulkan rendering’, in A. Sikora (ed.) Proceedings of Embedded World Conference 2022. WEKA, pp. 661–665. Available at: https://doi.org/10.21256/zhaw-26948.

M. Wegmann and M. Rosenthal, “Real time motion tracking for augmented reality with TOF camera and vulkan rendering,” in Proceedings of Embedded World Conference 2022, Jun. 2022, pp. 661–665. doi: 10.21256/zhaw-26948.

WEGMANN, Marcel und Matthias ROSENTHAL, 2022. Real time motion tracking for augmented reality with TOF camera and vulkan rendering. In: Axel SIKORA (Hrsg.), Proceedings of Embedded World Conference 2022. Conference paper. WEKA. 23 Juni 2022. S. 661–665. ISBN 978-3-645-50194-1

Wegmann, Marcel, and Matthias Rosenthal. 2022. “Real Time Motion Tracking for Augmented Reality with TOF Camera and Vulkan Rendering.” Conference paper. In Proceedings of Embedded World Conference 2022, edited by Axel Sikora, 661–65. WEKA. https://doi.org/10.21256/zhaw-26948.

Wegmann, Marcel, and Matthias Rosenthal. “Real Time Motion Tracking for Augmented Reality with TOF Camera and Vulkan Rendering.” Proceedings of Embedded World Conference 2022, edited by Axel Sikora, WEKA, 2022, pp. 661–65, https://doi.org/10.21256/zhaw-26948.