Problem
ABSTRACT Emerging platforms such as Kinect, Epson Moverio, or Meta SpaceGlasses enable immersive experiences, where applica- tions display content on multiple walls and multiple devices, detect objects in the world, and display content near those objects. App stores for these platforms enable users to run applications from third parties. Unfortunately, to display content properly near objects and on room surfaces, these applications need highly sensitive information, such as video and depth streams from the room, thus creating a serious privacy problem for app users.
Approach
To solve this problem, we introduce two new abstractions enabling least privilege interactions of apps with the room. First, a room skeleton that provides least privilege for ren- dering, unlike previous approaches that focus on inputs alone. Second, a detection sandbox that allows registering content to show if an object is detected, but prevents the application from knowing if the object is present.
Results
To demonstrate our ideas, we have built SurroundWeb, a 3D browser that enables web applications to use object recognition and room display capabilities with our least priv- ilege abstractions. We used SurroundWeb to build appli- cations for immersive presentation experiences, karaoke, etc. To assess the privacy of our approach, we used user sur- veys to demonstrate that the information revealed by our abstractions is acceptable. SurroundWeb does not lead to unacceptable runtime overheads: after a one-time setup pro- cedure that scans a room for projectable surfaces in about a minute, our prototype can render immersive multi-display web rooms at greater than 30 frames per second with up to 25 screens and up to a 1,440×720 display.