Mixed reality technologies provide real-time and immersive experiences, which bring tremendous opportunities in entertainment, education, and enriched experiences that are not directly accessible owing to safety or cost. The research in this field has been in the spotlight in the last few years as the metaverse went viral. The recently emerging omnidirectional video streams, i.e., 360° videos, provide an affordable way to capture and present dynamic real-world scenes. In the last decade, fueled by the rapid development of artificial intelligence and computational photography technologies, the research interests in mixed reality systems using 360° videos with richer and more realistic experiences are dramatically increased to unlock the true potential of the metaverse. In this survey, we cover recent research aimed at addressing the above issues in the 360° image and video processing technologies and applications for mixed reality. The survey summarizes the contributions of the recent research and describes potential future research directions about 360° media in the field of mixed reality.

混合现实技术提供实时和身临其境的体验，这为娱乐、教育和丰富的体验带来了巨大的机会，而这些体验由于安全或成本而无法直接获得。过去几年，随着虚拟宇宙的病毒式传播，这一领域的研究成为人们关注的焦点。最近出现的全向视频流，即 360° 视频，提供了一种经济实惠的方式来捕捉和呈现动态的现实世界场景。近十年来，在人工智能和计算摄影技术快速发展的推动下，人们对使用360°视频、具有更丰富、更真实体验的混合现实系统的研究兴趣急剧增加，以释放虚拟宇宙的真正潜力。在本次调查中，我们介绍了旨在解决 360° 图像和视频处理技术以及混合现实应用中上述问题的最新研究。该调查总结了近期研究的贡献，并描述了混合现实领域360°媒体未来潜在的研究方向。