GIBSON with PLATEAU

The GIBSON Project started from two frequently asked questions: what kind of framework do we need to reconnect people while the global pandemic of COVID-19 and how can we deliver a way of communication to people who visit places virtually?

This experimental project was opereated in collaboration with the PLEATEU project which aimes to build a open city 3D model and is sponsored by Japan's Ministry of Land, Infrastructure, Transport and Tourism. Through this project, we are trying to build a communication platform for city life in various fields, such as event, travel, and commerce. The research paper about the result of this experiment was accepted by SIGGRAPH ASIA 2022. 

GIBSON delivers city-scale cyber-physical communication experience that allows AR users and VR remote users can share the same spatial environments. We use the Shibuya/Jinnan area for this experiment. By synchronizing the spatial coordinates of AR/VR users, we enable them to enjoy sightseeing and windowshopping together.

Conventional VR experiences lacks the ability of updating in reflection to the real world once the digital twin world is created. On the other hand, GIBSON allows AR users to send the information of their sight in the form of real-time video streaming, which can create the sense of copresense for sightseeing and shopping.

UIs for cross-AR/VR communication experience GIBSON is designed to deliver spatial and comfortable interaction, and its 3D UI animations are created with 3D modeling tool Cinema 4D. The research scope of GIBSON covers the accuracy of city-scale spatial coordinate synch with VPS (Visual Positioning System) , the handling of loading spatially located contents, the quality of RGB video streaming from an AR user to a VR user through WebRTC and more.

The research on these topics aims to test the feasibility of delivering cross-AR/VR communication experience in city scale. The virtual Shibuya environment of GIBSON is build based on CityGML of Project PLATEAU, a project led by Japan's MLIT to develop and open data on 3D city models throughout Japan. We optimized the quality level of PLATEAU's CityGML for Oculus Quest 2 and add visual effects to its textures and meshs based on the concept and vision of GIBSON.

In the Jinnan area of Shibuya, we verified the added value and facilitation of communication by walking around and communicating as if sharing the same space across cyberspace and the local Shibuya area. We also verified the effect of discovering the attractiveness of the city by arranging information contents connected to the urban space.

Through the verification study, we found that using AR/VR technology to walk around a city while sharing the same space leads to the fostering of a sense of intimacy among users and an increase in discoveries about and attachment to the city. In addition, it was suggested that the sensation of being eye-to-eye with passersby from a distance and hearing the hustle and bustle of the city during a city tour contributed to a higher sense of realism than conventional VR experiences, and thus contributed to satisfaction.