In-camera visual effects using LED screens have moved very rapidly from a pioneering specialty to a mainstream technique being widely used across film and television production. The fundamental idea is to do your visual effects “in camera” by filming the action against a display showing the background scene, rather than using green screen and adding the background in post. That idea is not new, but the latest fine-pitch LED has significant advantages over other display technologies and can create flexible LED volumes that typically also include ceiling and wrap-around elements to provide realistic lighting. Large volumes for virtual production are now being built all over the world and kept busy producing major motion pictures, commercials, and iconic sci-fi and fantasy series, such as the latest, highly-anticipated series of Emmy-winning Star Trek: Discovery, which has just premiered on Paramount+ and was shot on Pixomondo’s flagship virtual production stage in Toronto, Canada.

Alongside filmic production, many broadcasters have successfully embraced this new cutting-edge technology to bring major events to life in-studio, such as Fox Sports delivering the world’s first live to air xR broadcast at one of the most prestigious sports events on the National Rugby League calendar, the Dally M Awards. And the Serbian commercial television station, TV Nova, deployed the technology to give its viewers a unique view of the Perseverance rover landing on Mars.

From being able to put the viewers in the driver’s seat at the Formula One World Championship to bringing them into the eye of the storm, or even dropping them into Earth’s deepest, and possibly most alien realm, The Mariana Trench, these extended reality techniques can help build more informative and immersive broadcasts. The smart money is clearly betting that this is how much of our film and television content is going to be created in the future.

Early adopters working with LED screens didn’t always have a smooth ride. LED screens that were not designed with use on camera in mind can turn out to be unsuitable, and the wrong LED solution or the wrong pairing of screen and camera causes more problems than it solves. Even when working well, the LED screen was often a fixed point that every other department had to adapt around. But, despite those challenges, the technology’s multiple advantages mean the market for LED in virtual production has grown rapidly and manufacturers are responding with products tailored to its needs.

Brompton Technology is an established market-leader in video processing for LED walls for live events, film and broadcast, and started out developing products to meet the demanding requirements of the world’s largest tours, corporate events and award shows. LED video processing is the catch-all term for everything that happens between receiving source video and the actual LED driver chips within an LED panel. It typically consists of a processor that receives the video signal, does any necessary formatting and adjustments, splits the image into chunks that map to each individual LED panel and distributes that data to the panels – most commonly over Gigabit Ethernet. Within each panel is a receiver card that converts the video data into linear light control signals for the LED drivers.

The video processing is sometimes overlooked as just the mapping function of the LED panel, but in reality there are many technical challenges in ensuring perfect synchronisation, achieving the lowest possible latency, correcting for the many non-ideal behaviours of LEDs and LED drivers, and managing the many trade-offs that arise from trying to squeeze the maximum performance out of an LED panel. Having the right processing is every bit as important as the choice of LED panel itself, and if you don’t know what processing is inside an LED panel then you can’t know what behaviour to expect.

Brompton has been working hard to ensure that the LED screen becomes a collaborative partner in the creative process rather than an obstacle to work around. This is an ongoing process of working with users to understand the pain points – then developing new features or integrating with complementary technologies to create a more optimised workflow for virtual production.

For example, creatives are used to being able to adjust their shutter angle to achieve a particular creative effect or manage motion blur. But with an LED screen in the mix, they have often been restricted to selecting a specific shutter angle to avoid visual artefacts. Earlier this year Brompton introduced ShutterSync®, a feature that allows the refresh timing of the LED screen to be adjusted to match the chosen shutter angle of the camera – putting creative control back in the hands of the user.

It is a similar story with colour, where LED screens can vary wildly in their achievable colour gamut and accuracy of colour reproduction. Brompton is a pioneer of Dynamic Calibration, a technology for LED that ensures accurate reproduction of colours within the gamut and intelligent management of out-of-gamut colours. Dynamic Calibration also underpins its implementation of HDR video, which is increasingly becoming a requirement for virtual production – in part due to the ability to encode absolute colour and brightness information into HDR content, which can then be faithfully reproduced on the LED screen.

Where custom colour management or on-set grading is required, Brompton’s Tessera processors support import of 3D LUTs (Look Up Tables) in industry-standard .cube formats. The capability to manage those 3D LUTs is now integrated into leading tools such as Pomfort Livegrade, so that Digital Imaging Technicians can now grade an LED screen in the same way they would manage any other monitor.

There are now even workflows available to support multi-camera virtual production shoots using LED screens. These rely upon Brompton’s Frame Remapping technology – which allows multiple different content streams to be interleaved so that a frame from each stream is shown in turn. Each camera can run at a different genlock phase offset so they each see their own unique content on the one screen. The typical application in virtual production is to allow different camera positions to see a background with the correct perspective.

Every project has a different workflow so there is no ‘one-size-fits-all’ approach. The key requirement is product reliability and flexibility that allows them to produce high-quality, immersive content at a faster pace. From smaller LED setups to larger volumes like those used on The Mandalorian, LED screens can deliver new possibilities to teams of all sizes.

The last 24 months have seen rapid, widespread adoption of virtual production using LED, and equally rapid advances in the technology available to make the LED screen a full partner in the creative process for film and broadcast production. By strategically utilising virtual production technology, broadcasters can gain an edge over the competition to present the information in a unique and immersive way, in real time, and with captivating graphics. Brompton Technology is continuing to support its many users in the space and advance the state of the art for virtual production and extended reality using LED.