TY - GEN
T1 - Interactive Panoramic Ray Tracing for Mixed 360° RGBD Videos
AU - Wu, Jian
AU - Wang, Lili
AU - Ke, Wei
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - This paper introduces an interactive panoramic ray tracing method for rendering real-time photo-realistic illumination and shadow effects when inserting virtual objects into 360° RGBD videos. First, we approximate the geometry of a real scene with a panoramic depth buffer and a screen space depth buffer. Then, a sparse sampling ray generation method is proposed to accelerate the tracing process by reducing the number of rays that need to be emitted in ray tracing. After this, an irradiance estimation pass is introduced to generate a noisy Monte-Carlo image. At last, the final result is smoothed by interpolation, spatial-temporal filtering, and differential rendering. We tested our method in some natural and synthetic scenes and compared the results of our approach with the ground truth and that of the Image-Based Lighting method. The results show that our method can generate visually photo-realistic frames for virtual objects in 360° RGBD videos in real-time, making the rendering results more natural and credible.
AB - This paper introduces an interactive panoramic ray tracing method for rendering real-time photo-realistic illumination and shadow effects when inserting virtual objects into 360° RGBD videos. First, we approximate the geometry of a real scene with a panoramic depth buffer and a screen space depth buffer. Then, a sparse sampling ray generation method is proposed to accelerate the tracing process by reducing the number of rays that need to be emitted in ray tracing. After this, an irradiance estimation pass is introduced to generate a noisy Monte-Carlo image. At last, the final result is smoothed by interpolation, spatial-temporal filtering, and differential rendering. We tested our method in some natural and synthetic scenes and compared the results of our approach with the ground truth and that of the Image-Based Lighting method. The results show that our method can generate visually photo-realistic frames for virtual objects in 360° RGBD videos in real-time, making the rendering results more natural and credible.
KW - Mixed reality-360° RGBD video-Real-time rendering-Panoramic ray tracing
UR - http://www.scopus.com/inward/record.url?scp=85159655022&partnerID=8YFLogxK
U2 - 10.1109/VRW58643.2023.00231
DO - 10.1109/VRW58643.2023.00231
M3 - Conference contribution
AN - SCOPUS:85159655022
T3 - Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023
SP - 777
EP - 778
BT - Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023
Y2 - 25 March 2023 through 29 March 2023
ER -