SMigraPH: a perceptually retained method for passive haptics-based migration of MR indoor scenes

To enhance users’ immersion in the mixed reality (MR) cross-scene environment, it is imperative to make geometric modifications to arbitrary multi-scale virtual scenes, including adjustments to layout and size, based on the appearance of diverse real-world spaces. Numerous studies have been conducted on the layout arrangement of pure virtual scenes; however, they often neglect the issue of incongruity between virtual and real environments. Our objective is to mitigate the incongruity between virtual and real scenes in MR, establish a rational layout and size for any virtual scene within an enclosed indoor environment, and leverage tangible real objects to achieve multi-class passive haptic feedback. To achieve these goals, we propose SMigraPH, a perceptually retained indoor scene migration method with passive haptics in MR. Firstly, we propose a scene abstraction technique for constructing mathematical representations of both virtual and real scenes, capturing geometric information and topological relationships, while providing a mapping strategy from the virtual to the real domain. Subsequently, we develop an optimization framework called v2rSA that integrates rationality, relationship preservation, haptic reuse, and scale fitting constraints in order to iteratively generate final layouts for virtual scenes. Finally, we render scenarios on optical see-through MR head-mounted displays (HMDs) to enable users to engage in realistic scene exploration and interaction with haptic feedback. We have conducted experiments and a user study on our proposed method, which demonstrates significant improvements in surface registration accuracy, haptic interaction efficiency, and fidelity compared to the state-of-the-art indoor scene layout arrangement method MakeItHome as well as the random placement approach RandomIn. The results of our approach closely resemble those achieved through manual placement using the Human method.