tob

Enhancing realism of mixed reality applications through real-time depth-imaging devices in X3D

Tobias Alexander Franke, Svenja Kahn, Manuel Olbrich and Yvonne Jung Proceedings of the 16th International Conference on 3D Web Technology

Teaser (Upper Right) Rotated 3D view with virtual trees. The depth measurements were transformed to the world coordinate system and mapped with the colors of the 2D camera. Left: Calculated camera positions (orange: color camera, blue: depth camera). (Upper Left) Raw depth image. At black pixels the Kinect depth camera could not acquire depth values. (Lower Left) Color image augmented with virtual trees (without occlusion) (Lower Right) Correct occlusion handling by mapping the depth measurements on the color image.

Abstract

Until recently, depth sensing cameras have been used almost exclusively in research due to the high costs of such specialized equipment. With the introduction of the Microsoft Kinect device, realtime depth imaging is now available for the ordinary developer at low expenses and so far it has been received with great interest from both the research and hobby developer community. The underlying OpenNI framework not only allows to extract the depth image from the camera, but also provides tracking information of gestures or user skeletons. In this paper, we present a framework to include depth sensing devices into X3D in order to enhance visual fidelity of X3D Mixed Reality applications by introducing some extensions for advanced rendering techniques. We furthermore outline how to calibrate depth and image data in a meaningful way through calibration for devices that do not already come with precalibrated sensors, as well as a discussion of some of the OpenNI functionality that X3D can benefit from in the future.

Preview

Bibtex

@inproceedings{Franke:2011:ERM:2010425.2010439,
 author = {Franke, Tobias and Kahn, Svenja and Olbrich, Manuel and Jung, Yvonne},
 title = {Enhancing realism of mixed reality applications through real-time depth-imaging devices in X3D},
 booktitle = {Proceedings of the 16th International Conference on 3D Web Technology},
 series = {Web3D '11},
 year = {2011},
 isbn = {978-1-4503-0774-1},
 location = {Paris, France},
 pages = {71--79},
 numpages = {9},
 url = {http://doi.acm.org/10.1145/2010425.2010439},
 doi = {10.1145/2010425.2010439},
 acmid = {2010439},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {X3D, augmented reality, mixed reality, rendering},
} 

Links