Manual URL： https://drive.google.com/file/d/0B2a6aRDV_JeuQTVfWTdBblQzcVE/view?usp=sharing
360EyeToEar is a wearable camera that converts the visual information of the 360 degree surrounding into audio information, and communicates the visual information as sound to the wearer in real time.
Two units of RICOH THETA S on one's head create a spherical stereo camera system, and this enables us to measure the direction and distance to an object just as human eyes do.
The information about the object is represented by the sound that is proportional to the distance of the object and is from the direction of the object. The sound is audible from a pair of surround headphones that is combined with RICOH THETA S.
So you can grasp the situation of the 360-degree by the sound from the headphones alone.
Human visual field is said to be as wide as 130 degrees vertically, and 120 degrees horizontally. There are many dead spots in the extreme right and left, and behind. Also, the number of visually-challenged people in Japan is said to be more than a few hundred thousand. With this background, we wanted to come up with a non-visual system that can convey the information about the surrounding area that cannot be covered by human visual perception alone.
System and operation
The developed system submitted to the contest consists of a mini PC that is connected to a mobile battery, two RICOH THETA S cameras and USB surround headphones. The software that is demonstrated on the system recognizes people's faces near the wearer of 360EyeToEar, and determines the direction and distance. This measurement is done using the live streaming data from RICOH THETA S. The alarm sound is created based on the distance and direction, and fed into the surround headphones. The wearer can hear the sound in the direction of the bystander.
Using this system, one can learn (hear) the distance and direction of someone approaching you on a street or a city corner without depending on one's vision.
There is a large potential for application in this submission, "360EyeToEar", which uses audio for the visually-challenged. The ideas expressed in "360EyeToEar" and similar ones will become more important in the coming years. The potential of 360-degree audio is also investigated in many quarters. In this sense, this work reminds us of the future possibilities acutely.
Interview with the award winners
We came up with the rabbit-ear-like part of “360EyeToEar” when we put two RICOH THETAs over our heads by chance. We expect RICOH THETA to provide much more interesting use after it’s connected to the Internet to expand its system in the future. We’re planning to create another work for the contest next year.