The global pandemic has acted as a catalyst for the surging demand for virtual production (VP). Once the preserve of high-end films, this approach has made its way into TV series, such as the Mandalorian and the Lion King – both shot in LED/green screen studios – and even live television, as we saw with BBC Sport’s virtual studios for its coverage of the recent Summer and Winter Games.
While VP is associated with green/blue screens, the use of LED panels to display VR sets and backgrounds allows content producers to create dynamic set design that delivers an immersive effect. Pioneers such as Mo-Sys have helped to make VP technology more accessible to smaller projects as well as live television.
In a typical project involving green screen elements, directors and cinematographers will make continuous calculations to match real and virtual elements in post-production. But good post production does not come cheap, and when repetitive processes are involved, the costs soon add up.
Mo-Sy makes it easy to produce seamless, high-end productions. VP Pro XR combines the power and capability of professional systems with the easy operation of prosumer solutions and with affordable scalability. It offers seamless set extensions and confers minimal XR delay.
The first purpose-built Cinematic XR server solution, Mo-Sys’ VP Pro XR allows cinematic standards to be brought into on-set real-time virtual production. This innovative solution takes a radical new approach and combines the power and capability of scalable professional systems with the easy operation and affordability of prosumer solutions to enable traditional shooting techniques within an LED volume. Mo-Sys uses the strengths of both green screen and LED volume, to deliver on composite image quality.
Designed with LED stages – with or without set extensions – in mind, the system can be used with blue/green screens and enables cinematographers to use the traditional shooting techniques that they are familiar with within an LED volume. Mo-Sys VP Pro XR is designed to blend these tried and tested shooting techniques with new virtual production environments. The aim of the technology is to remove the limitations on content producers’ ability to tell powerful and engaging stories that are imposed by current XR stage designs.
VP Pro XR, which supports Epic Games’ Unreal Engine 4.27, delivers innovative capabilities, including:
Cinematic XR Focus: an industry first innovation for seamless interaction between virtual and real worlds. It turns an LED wall into more than just a backdrop, allowing it to integrate with the real stage. Cinematographers can now seamlessly rack focus deep into the virtual world and create layered images to enhance their visual storytelling. Mo-Sys achieves this by combining the same wireless lens control system commonly used in filmmaking with its StarTracker camera tracking technology to constantly track the distance between the camera and the LED wall.
The Cinematic XR Focus feature addresses a major creative limitation when using an LED volume for virtual production. Traditionally, the lens focal plane stops at the LED wall, so when cinematographer needed to pull focus between real foreground objects – such as actors – and virtual objects displayed on an LED wall – such as a deer located in the distance in the virtual world – they have been unable to do this; the deer would always remain out of focus. Now, cinematographers seamlessly rack focus deep into the virtual world for richer images
Near Time®, which picked up the HPA Engineering Excellence award, is a unique workflow for virtual production allows cast and crew to see the full effect of the shot on-set in real-time. It delivers a higher-quality version of the shot, completely automated, and in a timescale which matches the practical requirements of the production – ‘near-time’ without the overheads of real-time augmented reality.
The NearTime function, achieves results fast enough, without the complications and cost of real-time delivery. On set, the graphics elements are rendered in a good enough resolution in real time. This might be HD. Rendering parameters are selected for time optimization. The director and DP see the blend of real and virtual elements in real time, allowing for perfect positioning of actors, props and camera moves. This allows creative freedom.
The moment the shot is complete, a quality optimized rendering starts. Where the on-set resolution might be HD and good enough, this render might be 4k and perfect, taking between five and fifty times real time. The key to the success of NearTime is that the rendering takes place in the cloud. Mo-Sys has partnered with AWS for processing and content management, giving users effectively infinite parallel resources.
VP Pro XR also offers an Online Lens Library of cost-effective lens distortion calibration tools to tweak lenses on-set. The library answers the need to custom calibrate lens distortion quickly and easily on set for seamless integration between the virtual and live elements of a production.
In short, as a pioneer in virtual production Mo-Sys technology is aimed at helping to make VP technology more accessible to a wide variety of virtual production projects, regardless of size or budget. VP Pro XR and the Cinematic XR initiative, which are specifically designed to address the current on-set real-time quality and functionality challenges, are a perfect example. These innovations provide a platform for unique features (e.g. Cinematic XR Focus) and address known XR challenges, such as investment costs, expansion costs, and XR graphics delay, meting the pressing need for a ‘fit for purpose’ LED/XR technology for the broadcast and film industries.
