MainConcept – Will Multiview HEVC pave the way for Extended Reality and the Metaverse?
Geoff Gordon, VP Global Marketing, MainConcept
The next frontier of entertainment lies in interactive and immersive experiences, where traditional TV and device screens give way to rich, 3D environments that surround the viewer. And while video compression technology might not grab the headlines, it’s the backbone of 3D video content creation and delivery, enabling the seamless, high-quality experiences that extended reality (XR) and metaverse applications demand. Interestingly, the codec, Multiview HEVC (MV-HEVC), which is designed to enable 3D video experiences, is currently enjoying a surge in market adoption. This increase is largely driven by Apple’s launch of the Apple Vision Pro headset, which uses the MV-HEVC codec. Since then, many more OEMs are adding in support for the codec to their devices. What impact will this have on the industry, and might it help facilitate widespread adoption of extended reality and acceleration of the metaverse? For clarity, extended reality and XR are used throughout as a catch-all term for virtual, augmented and mixed reality.
Immersive experiences
Viewers today are being presented with more interactive and immersive viewing experiences. VR headsets like the Meta Quest have grown in popularity among consumers, enabling users to experience things like travel to exotic destinations, rock climbing, riding a roller coaster, watch movies and TV shows, play multiplayer games, and socialize with friends from the comfort of their living room.
Unsurprisingly, the Apple Vision Pro has generated a huge amount of interest since its launch, merging virtual with augmented reality so the experience is not limited by the physical confines of the TV or device, but instead happens in the space all around the user. Starting with the iPhone 15 Pro, Apple added support for MV-HEVC, enabling users to capture spatial video to create 3D content that can be played back on the Apple Vision Pro. The MV-HEVC codec is instrumental in delivering the high-quality immersive experiences that the Apple Vision Pro will undoubtedly provide.
Understanding MV-HEVC
The volume of data needed to create 3D and VR experiences is vast and much greater than traditional 2D video. For VR, content creators can use stereoscopic lenses to capture scenes from ‘a left and right eye’ perspective to more closely resemble the way we view the real world. For 360° videos, cameras record the entire scene in every direction via multiple overlapping lenses. This use of multiple views for every frame creates a vast amount of data, which needs to be compressed efficiently so that it can be stored and then transmitted to VR headsets. The sheer amount of data involved in this process is often considered as one of the barriers to widespread VR adoption.
MV-HEVC is the multiview extension to the highly efficient HEVC (High Efficiency Video Coding) codec, extending the scope of HEVC to include 3D video. MV-HEVC is designed to reduce the file size by reducing the amount of redundant information stored from one view to another. It compares the similarities between different views, and by recognizing these similarities, it can avoid storing redundant data, effectively reducing the overall file size. Put simply, MV-HEVC makes the video files smaller by finding and reducing similar information across multiple views, storing less data. And importantly, the way that MV-HEVC efficiently encodes spatially related left eye and right eye image views, means that it is able to compress the video without losing noticeable quality, which can be an undesirable consequence of the compression process.
Potential impact on video industry
So, with all this in mind, what might the increased uptake of MV-HEVC mean for the video industry? Just as with all advanced codecs, by efficiently compressing files and reducing bitrate, MV-HEVC helps to reduce storage and distribution costs, and enables video to be transmitted quicker with lower bandwidth requirements. But its potential impact goes much, much further than these practical advantages. The compression efficiency that MV-HEVC provides is crucial for creating high-quality immersive experiences, because it allows for smoother interactions with less lag, making virtual environments feel more realistic and responsive. These factors are both critical components to delivering a great XR user experience.
MV-HEVC’s ability to efficiently compress multi-angle views would also enable the development of more intricate and detailed virtual worlds, allowing users to explore richer environments in real time. This capability could lead to more engaging and interactive experiences, whatever the application, whether in gaming, travel experiences, education, live streaming, or social interactions.
The increased use of MV-HEVC could help to accelerate the adoption of VR, AR and the metaverse as a whole by delivering high-quality immersive video with reduced data and bandwidth requirements. By lowering the data needed for high-quality content, these experiences would be more accessible to users with slower internet connections or in areas with less robust infrastructure, as well as a diversity of device-types. This wider accessibility would encourage greater adoption of these technologies, allowing more people to participate in immersive experiences and shared virtual environments in the metaverse.
The bottom line
Like HEVC, MV-HEVC is still relatively early on in its lifecycle, having been released in 2014 as part of the second version of the HEVC standard. And as such, there’s still a way to go before it is supported across the majority of platforms and devices. However, its increase in uptake and strong support from companies such as Apple does show a willingness and expectation within the industry that XR technology will become more mainstream over time. Ultimately, as MV-HEVC grows in popularity across the industry, it’s highly likely that it will drive innovation in immersive content, boost the growth of XR technologies, and set the stage for a more interactive and immersive video industry.
