Award Winners
grigtod
Video URL：https://www.youtube.com/embed/7UVy7S7v9U4
Manual URL：
https://paper.dropbox.com/doc/360-stream-to-AR-app-for-image-based-lighting-and-real-timereflections- m3hXjllnTm3LberqdtK6P
Application URL：
https://www.dropbox.com/s/hgdy7mfl4u1qv0m/360%20stream%20to%20AR%20app%20 for%20image-based%20lighting%20and%20real-time%20reflections.zip?dl=0
With this proof of concept app I aimed at using the RICOH THETA S camera to capture 360 image and use it for image-based lighting and reflections in an augmented reality application.
This app is capable of loading an equirectangular image from the phone memory, downloading any image from the Internet or connecting to a PC and using a real-time video stream from the RICOH THETA S camera. The equirectangular image (or stream) is then converted into a cubemap and used for image-based lighting and reflections in the Unity3d game engine.
I find this project compelling as it’s something 360 cameras are not usually used for and augmented reality applications can surely benefit from some additional realism.
A few plug-ins were used to ease the development process as the goal of this app is to prove that the RICOH THETA S camera can be beneficially used in an augmented reality application.
The Vuforia AR plugin was used as it’s the one I’m most familiar with, but I’m also developing versions for Kudan and Wikitude.
Requirements
- Android app tested on Galaxy S6 - Android 5.1.1
- PC Streaming app tested on Windows 10.
- RICOH THETA S camera
Additional software/plug-ins used
- Unity3d 5.4
https://unity3d.com/ - RICOH THETA S Live for Unity®
http://theta360.guide/blog/unity/2015/12/17/Using-RICOH-Live-View-With-Unity.html - PanoramaToCubemap
https://www.assetstore.unity3d.com/en/#!/content/13616 - Vuforia - http://www.vuforia.com/
- ICSharpCode BZip2 for stream compression
https://icsharpcode.github.io/SharpZipLib/ - ImageVideoContactPicker to load an image from the phone memory
https://www.assetstore.unity3d.com/en/#!/content/28597
3D models used for demonstration
- Animated sphere
https://grabcad.com/library/sphere-gears-i-1
3D scanned people
https://sketchfab.com/codame
3D scanned tower
https://sketchfab.com/models/d8883fe7a08e440b8f2af4624e649c58
The technology and its direction shown in this work, "360 stream to AR app for image-based lighting and real-time reflections", will be very important in the expression in the future virtual reality and augmented reality applications. There are many submissions that focus on IBL (Image-Based Lighting) in this contest. Among the peers, this work that implements IBL for AR applications using 360-degree images of RICOH THETA was given high marks by the judges from the viewpoint of "Does it inspire imagination and sense of innovation?", and was selected as Second Prize.