LCEVC – increasing compression efficiency, lowering costs and reducing carbon footprint

Guido Meardi

CEO and Co-Founder of V-Nova

We spoke with Guido Meardi, CEO and Co-Founder of V-Nova to understand the benefits of LCEVC video compression and what these could mean for users across Broadcast, Media & Entertainment.

1. Why LCEVC?

Video is the most engaging communication and entertainment medium and the appetite for better and more engaging video experiences is only growing. However, delivering higher resolutions and colour depths to more people and devices remains a challenge, regardless of the network used.

MPEG-5 LCEVC aims to address today several of the challenges increasingly felt in video streaming, broadcasting, gaming, immersive video, etc.

Improved video compression is the key to solving these challenges. More recent and advanced codecs have been designed with these in mind, but most services will have to wait for device support to reach a critical mass to make their deployment meaningful and this will take years. Every new generation of video codec also comes with substantially higher encoding compute costs that can impact both profitability and the environmental sustainability of video delivery.

To meet these challenges, MPEG-5 Part 2 LCEVC ‘Low Complexity Enhancement Video Coding’ adds an enhancement layer to any codec improving their compression efficiency (and therefore quality) at up to 40% lower bitrates whilst maintaining the same device compatibility as the codec that is being enhanced. Low complexity also results in 60-70% lower compute costs, as well as lower carbon emissions.

In short, the low-complexity enhancement layer approach of LCEVC can be immediately deployed with software updates and helps solve video delivery challenges. It does this by enabling higher quality over any given bandwidth and a reduction of content preparation and delivery costs, while being supported on the vast majority of devices available today.

2. Explain the licensing model

From the outset we wanted to understand what would deliver the best value to customers and to the industry. We consulted widely with services and other key industry players to define the licensing model. As a result, the fees follow these key principles:

1) Free for integration by device or chipset manufacturers, operating systems, browsers, for in-house development and encoder or player vendors.

2) Low-cost and capped for services, with overall cost based on service size.

3) Designed together with customers and key industry players in order to meet their needs and ensure swift adoption

The V-Nova LCEVC licensing model reflects the low complexity of the standard and the ability to be implemented efficiently also in software. As such, the licensing model does not require to follow traditional models charging devices and silicon deployment, but rather it can align with modern software-as-a-service (SaaS) models.  The functionality is made available to the ecosystem, and services only pay if and when using and benefitting from the increased user engagement, satisfaction and retention as well as the cost efficiencies that LCEVC offers.

3. Who gets the licensing revenues?

V-Nova licenses and receives revenues for the V-Nova LCEVC, a fully productised implementation of the LCEVC standard.

4. What is the future for LCEVC?

LCEVC has been standardised and recently MPEG has published the Verification Tests validating that the objective of enhancing the compression efficiency of another video compression standard while reducing its computational complexity has been achieved.

In parallel, we have developed implementations of LCEVC that we are making available to our customers. At this point, our SDK already supports a wide range of encoding and device environments. For encoding we have libraries optimised for CPUs (both Intel and ARM), GPUs and even FPGA. For decoding we have optimised libraries to cover the most popular platforms including Android, iOS, Windows, MAC and scripted decoding for HTML5-capable browsers to deploy LCEVC avoiding the need for any plugins.

To facilitate adoption, we also made available multiple reference integrations for a wide range of encoders and decoders including FFmpeg (with support for over 20 base encoders), ExoPlayer for Android, AVPlayer for iOS, Microsoft UWP for Windows and web players like HLS.js, Shaka Player and video.js. We also have integrations at the operating system level, such as a patch for AOSP (Android Open-Source Project) that makes LCEVC easily implementable in all devices with operating systems derived from that project. Importantly, those are not tech demos: they are deployable implementations, and while working on actual deployments with our Early Adopter Program, we have also engaged with professional testing companies to soak test and certify them across devices.

With the announcement of clear and simple licensing terms, we are now completing the package for anyone wanting to deploy LCEVC in any media and entertainment application.

Our focus is now on supporting the adoption and deployment of LCEVC through a range of active customer and partner projects. We see great interest and we’re confident that the future for LCEVC is wide adoption.

The evolution of a regional TV and advertising system

Peter Blatchford

CMO, Starfish Technologies Ltd

This narrative begins in 2006. One of the more forward-thinking European broadcasters began the search for a system to take their single national feed, that was funded through traditional ad break advertising, and produce just over 30 variants of this channel. Every channel (regional feed) would now include some regional advertising content, replacing the national advertising breaks at specific times of the day. Starfish Technologies was awarded the contract to design and supply this system. It was implemented using SDI based technology located at each of the regional distribution hubs, the majority of which were unmanned. This system worked well and generated a significant additional revenue stream, so the broadcaster requested an ‘upgrade’ to this system that would also enable local news bulletins and late changing schedules to be inserted into every regional feed. These requirements were best implemented by moving to a centralised architecture and building a complete regional channel system located at its main transmission centre. The first centralised Starfish Technologies system went live in 2009 and again proved reliable, commercially rewarding and with the significant benefit of providing viewers with locally relevant news content.

In 2012, the requirement specification expanded again, this time to supplying the 30+ regions with HD feeds and HD commercial/news inserts. It also included providing a range of encoded feeds to support different distribution platforms, with both MPEG and H264 streams at various bitrates. The system Starfish built to meet these requirements was incredibly sophisticated and included software-based encoding, stream multiplexing, auto failover redundancy and web-based system monitoring. This was a hybrid system where the input national feed was provided as high-definition SDI signal. This signal was replicated for each channel encoder (that also performed the content insertion/ replacement) and then transcoded to the required output format and bitrate, and then leaving the complete Starfish system as four encoded MPTS streams. See fig 1. At that time, the traditional approach to providing regional variants of a TV channel would have been to use multiple playout servers (typically ‘channel in a box’ systems), multiple channels of traditional broadcast automation and a large group of dedicated broadcast encoders, followed by stream multiplexers. The estimated physical size of this type of installation, power requirements (and subsequent air conditioning requirement) and not least the cost, would have been totally prohibitive.  

All the systems described above were designed and built by Starfish Technologies using generic enterprise grade server hardware, and primarily Starfish Technologies software products. The resultant reduction in physical size, power consumption and cost over a traditional approach was revolutionary at that time.

Figure 1 The System went live in 2014 producing over 60 channels (30 regional signals configured with full 1+1 redundancy) of HD content insertion and multi format out encoding.

When this HD system was scheduled for a hardware refresh, because of hardware warranty expiration, the increase in SDI card density and the improvement in Ad inserter/encoder performance resulting from improved CPU performance and software optimisation, meant the re-built system (using fundamentally the same software licences) had a physical size reduction of over 60%. This system installation is shown in figure 2.

The next logical move in building this type of system was to move away from the restrictions imposed by SDI infrastructure, such as SDI card density, SDI routing and most importantly the desire to move beyond HD resolutions. This would require implementing the content switching/ad insertion on high

bitrate (mezzanine quality) encoded streams. At that time, the products available to perform this function had a number of limitations, primarily they decoded the input stream back to base band video, inserted the replacement content, and then re-encoded the output stream. This resulted in concatenated encoding artifacts in complex systems and significant propagation delays through the processing channel - in some cases this was over 4 seconds.

Starfish Technologies decided to begin a very significant development project to design a next generation product that would avoid the limitations of the transport stream processing technology available at that time. The result of this development project is the Starfish TS splicer.  This product incorporates Starfish’s patented technology to produce technically clean switching of encoded streams (splicing) without re-encoding the input stream and results in a significantly reduced propagation delay (typically less than 1 second).

Using these sophisticated stream switching techniques, continued software optimisation and combined with the ever-growing improvements in CPU performance and network speed, Starfish can now offer a 30 channel HD TS Splicer system on a single 1U generic server.

Figure 2 Image of the system update in 2019 after a hardware refresh/update

With the system manager functions of advertisement schedule handling, data base management, automation command processing, channel redundancy switching and system monitoring (via a web-based GUI) running on another, relatively low performance server. For a fully redundant 1+1 configuration, this two-server installation can be replicated at a remote site where both systems are producing the same 30 channels of live content insertion/replacement on encoded mezzanine broadcast streams.

Figure 3 Two servers providing a managed, 30 channel HD system of connect replacement/ ad insertion.

Because all these systems are processing live TV channels and have a 24x7 requirement, all of our customers have chosen to deploy our software on their own, on-premise servers.

Moving forward

Applications of the TS Splicer technology include adding regional/local advertising, blocking advertising where distribution rights are not available, blocking content where there is cultural sensitivity and simply processing new OTT feeds to exploit the commercial benefits of advertising insertion.

There is also a wide range of requirements to process advertisement break signalling, including adding advertising break commands to enable downstream distributors to replace or insert advertising, and reformatting advertising break commands to meet the requirements of a new OTT system, including adding anti skip markers. Typically, these break signals are required to be compliant with the SCTE35 specification for signalling in encoded streams. The TS Splicer can add an SCTE35 signal to the channel stream using commands derived from local automation and system control systems, or even manually generated GPI’s.

The opportunities for commercially focused ‘broadcasters’ are obvious and the more forward-thinking media suppliers are implementing new operational models to exploit these opportunities. Starfish Technologies will continue to develop our products for these applications and enable these suppliers to maximise the opportunity by incorporating sophisticated, automated solutions.

Democratizing Livestreaming Around the World

Ren Egawa

CEO of Rexcel Nippon Co., Ltd. (Tokyo) and Rexcel Group (Philadelphia)

Sammi Cannold

Director at New York City Center

日本版

A case study within the performing arts industry

“I can’t enter the theater?”

In 2020, I traveled to Taiwan, to conduct the livestreaming of an opera production. But with a sudden surge of COVID cases abroad, foreigners were quickly barred from entering certain buildings in the country, including the very theater the opera was to be performed in. As I could no longer organize the livestreaming in person, I resorted to configuring the components for it in a hotel room, and passed them along to the opera production team’s camera crew. This allowed the crew to simply connect a few cables, and start the livestreaming with the push of a button. In this moment, the “SIMPLE LaaS” (Livestream-as-a-Service) was developed - enabling performers to livestream high quality content with four simple steps: 

Little did I know at the time, SIMPLE LaaS would assist in understanding the various concerns livestreaming can introduce for a performer. In this article, I will walk through each of these concerns, and discuss how livestreaming can serve performing artists to the best of its abilities.

Sammi Cannold, director at New York City Center, has directed a documentary about the survival of theater during the pandemic. She recently shared with me her thoughts on the potential of livestreaming technology:

“In the world of commercial musical theater, where I operate as a freelance director, conversations around livestreaming are incredibly active and lively at this moment. For years, proponents have been trying to win over the commercial theater community, but never have they seen more traction than in the wake of the global pandemic. Having lived through a near existential threat to our industry, many theatermakers are eager to ensure that our artform reaches as many people as possible and streaming has proven to be a primary--if not the main--vehicle for that.”

But its potential does not come without concerns:

“One of the issues we're contending with is in the distinction between streaming and capture/release. Several major productions--such as the Broadway productions of Diana and Come From Away--have recently pursued capture and release approaches (with Netflix and Apple respectively). Where there is more reticence is in actual live-streaming as artists--myself included--are nervous about the lack of control that one has when relaying the content of a live event live. Of course, we as theatermakers, give up that control in the theater as our artform is necessarily live, but we seem to have trouble getting over the hurdle when it comes to capturing and immediately releasing, perhaps in part because we aren't familiar with the methods of quality control in the world of streaming. So, it feels as if there's an artist education campaign to be waged in that regard.”

As a livestream professional who has worked in the broadcast and consumer electronics industries for more than thirty years, I echo Sammi’s points, especially regarding the concern on “quality control.”

When producing a performance that is simultaneously in person and online, it is essential to have a tight collaboration between the stage and video directors. Luckily, the broadcasting industry has produced experienced video directors who can work well with stage directors, and assist in producing a still effective performance for the remote audience. But with the introduction of livestreaming, new and unique challenges come into play for video directors, who are used to only worrying about TV viewers.

A livestream must cater to the possibility of different viewing devices, and artistry must still be well presented to an audience regardless of how they are watching. For example, even if a stage’s lighting looks good in person, it is challenging to capture it in the same way on camera for streaming. The only way to know if a livestream is doing justice to the artistry of a performance’s lighting, effects, etc., is through “practice, practice, practice.” But to practice means to rehearse and recreate multiple times, and it hasn’t been particularly easy to set up livestreams for concerts before, much less to recreate them for “practicing.” Fortunately, the easy setup of “SIMPLE LaaS'” I devised in Taiwan allows performers to effortlessly practice livestreaming. In Taiwan, opera rehearsals were “practice livestreamed'' as many times as the performers wished until everyone felt comfortable with the display and quality of artistry in this livestreamed video.

Aside from the concerns of portraying the artistry of a performance on video, we must be conscious of how engaging the livestream is as well. Boredom easily drives away online viewers. Considering how a portion of a remote audience may well be tuning into a performance on the small screens of their smartphones, a single, fixed angle livestream is practically inviting them to leave. But what makes them stay? A multiple-angle or XR? While I can’t say for sure, the camera angles used should regardlessly capture the soul of a performer, and deliver passion to online audiences. This 4HDs-in-one-4K livestream provides a possible solution to keeping online audiences engaged with a performance.

Multiple-angle cameras bring out more expressions, but also add more work and cost to a production. “Simple LaaS” pre-configures encoders to their respective cameras, allowing stage and video directors to focus on setting up different camera angles and other more important tasks. Once an optimal multi-view presentation is achieved, a few camera angles may be selected as a “premium channel”; these angles may be accessed by online audiences who chose to pay an extra fee.

“Quality control” is important. However, a different issue arises, as audiences are accustomed to watching online content for free. Extra steps are necessary to encourage them to pay for a livestreamed show. And as long as there exists a third party that controls and stands between performers and online audiences, it is unrealistic to expect the audiences to pay the performers. There are two underlying problems, and solutions to both point in the direction of democratizing livestreaming.

The first problem is when an audience assumes that watching intrusive advertisements and commercials pays for artists and the production of a show. However, it is possible to optimize this situation by providing performers with a financially viable method to control what commercials are shown, and when to show them. See diagram below, SIMPLE CM ADD, as an example as to how this may be achieved.

SIMPLE CM ADD:

  • Allows performers to decide when to insert the commercial, such as during an intermission (between time A and time B).
  • Allows performers to decide what commercials play by placing them in the cloud.
  • Needs a commercial cloud service provider such as AWS (unlike a free platform, such as YouTube) that incurs an additional cost. But this cost can be recouped by:
    • Comparing and choosing a low-cost CM ADD operation method

While conventional methods require costly equipment (e.g. SCTE-104) or cloud processes (e.g. transcoding), new methods such as SIMPLE CM ADD do not.

Covering the operation cost by increasing sponsorship revenues, by placing targeted commercials pertaining to different audiences of their respective regions.

E.g. Placing a Japanese commercial for Japanese audiences and a French commercial for French audiences creates the possibility of receiving sponsorships from companies that plan to enter either the Japanese or French market. These commercials can be deployed during the same timeslot as well. If used globally, the chance of having more and diverse sponsorships is increased.

The second problem that may arise is the idea that a pre-recorded production is finished, and that the people involved have been paid for it all already. This can be deflected by offering a “once-in-a-lifetime experience” to an audience. We are experimenting with an approach of disabling DVR, and livestreaming every show instead. If an audience member misses a show, they can buy or exchange for a ticket to watch the next livestream show. Each livestream should be ”live,” and the excitement it creates encourages audiences to pay to watch, rather than feeling as if they are paying to watch DVR content. Disabling DVR also helps lower the livestream operation cost by eliminating the cost of storing content in the cloud.

Every country’s livestream skill varies, and it is impractical to travel to all countries to support their livestream operations. SIMPLE LaaS allows us to support remotely, by requiring an encoder that:

  • Is light, small, and durable.
  • Is quiet (no-fan), as some traditional art forms (e.g. Noh) require absolute silence.
  • Concurrently supports various domestic and international CDNs so if one server is down, a backup keeps the show running.
  • Supports 4K resolution so performances are appreciated to their full extent, even in the details.

To fulfil these requirements, we came to rely on Videon’s “EdgeCaster.” It is powered by a Qualcomm SoC that was originally designed for portable devices, consumes little power (< 7w), and operates without a fan. It is small (38.1mm×127mm×107.5mm) and light (450g), delivers multiple HTTP and RTMP streams simultaneously, and supports 4K.

Summary

In this article, we share our mission of enabling performers to livestream shows at low cost and high quality, offering innovative technologies and tools to generate new revenue streams, and maintaining a profitable and sustainable livestream operation under the performers’ own control… all through democratizing livestreams by connecting performers directly to the audiences.

In Conversation with Never.no

We are joined by Never.No CEO Scott Davies to discuss whether the predictions he made in January 2021 (available to view here) are going as expected.

We also hear how Never.No’s customers have changed the way they work over the last 18 months and how the team have helped with this along with some notable projects they have worked on.

Finally, we hear how Never.No have helped companies enhance their virtual and hybrid events.

Access Never.no’s guide on Producing Virtual & Hybrid Events by clicking here.

Accelerating the Move to Remote & Virtualised Workflows

Rob Green

Senior Manager - Pro AV, Broadcast & Consumer - Xilinx

Introduction

Over the last three decades, we’ve seen a shift from specialised units for various video and audio functions, to high performance hardware platforms which can be repurposed on-the-fly to support any AV processing, and now to virtualized media applications and microservices which can be spun up and deployed (and paid for) only when you need them.

The industry shift from specialised to virtualised

Broadcasters have applied constant pressure to suppliers to enable them to move their capital expenses to operating expenses which can be more flexibly managed, as well as improve agility and scalability by providing services just for specific events or productions.

Two major technology advances have enabled this shift to occur: AV-over-IP and compression. Moving from point-to-point SDI connections to IP transport using open and interoperable standards such as SMPTE ST 2110, has meant that equipment can become addressable on the network, and that packets of audio, video and data can be transported over larger distances and to multiple places at once. This has many benefits, not least that the hardware can effectively be anywhere in the world, either on-premise or deployed in cloud data centres. Additionally, the emergence of lightweight, low-latency mezzanine compression schemes such as JPEG-XS are helping to move media around the network more easily, using existing Ethernet installations for AV-over-IP at 4K and beyond, and enabling efficient streaming in and out of the cloud. H.264 and H.265 encoding has also been more widely deployed for contribution and for live streaming.

The impact of a global pandemic

This move from on-premise, specialised platforms to distributed workflows was happening anyway, but the global COVID-19 pandemic has vastly accelerated the move from technology proof-of-concepts to technically and commercially viable deployments for live TV and production. Unsurprisingly, the IABM Special Report (Sep 2020) listed the three most important priorities in technology roadmaps as Cloud & Virtualization, OTT & Streaming Platforms and At-Home/Remote Production, and with proven deployments are likely to become the norm.

We’ve seen many parts of the industry suffer greatly with lockdowns forcing abandonment of major sports and live events, but some have thrived.

The demand for remote operations has increased significantly because of travel restrictions and social distancing. On the contribution side, technology has enabled live streaming content from broadcaster staff working from home (WFH) using low-cost production equipment - often a decent quality SLR camera with HDMI output, and a live H.264 encoder box or mini-switcher connected to the Internet is all that’s needed. This is one of the major reasons we saw a big increase in adoption of the Xilinx Zynq® UltraScale+™ MPSoC (figure 2) which integrates AV connectivity and H.264/H.265 codec into a low-cost, single-chip multimedia processor.

Xilinx Zynq UltraScale+ MPSoCs

On the acquisition side, ensuring social distancing on-set has been made possible with the same devices, offering compressed video through wireless camera transmitters to remote field monitors, or the use of remotely controlled PTZ (point, tilt & zoom) cameras. KVM (keyboard, video & mouse extenders) have also seen growing use for remote control and display, safely separating the operators from the source.

Travel restrictions can be somewhat overcome with the use of virtual sets, with exciting developments in the use of fine-pitch LED walls in place of traditional green screens and real-time CG with in-camera VFX offering a way to create virtually any location, real or imaginary. Driving such walls, particularly with large formats such as 8K, needs high-performance software processing and hardware acceleration, and the creativity that it unleashes is incredible.

In production, the emergence of ST 2110, cloud-based workflows, and reliable streaming with Secure Reliable Transport (SRT) for example, has also allowed distributed production teams to work on major live events from the comfort of their homes.

All these use cases were not driven by the pandemic itself, but rather accelerated their adoption. They showed that the technology works, and that new ways of working can be achieved much sooner than planned and likely to become more permanent in many cases.

Virtualisation

One of the major technology shifts in remote production and distributed workflows is abstraction away from hardware, or virtualisation. Understandably, we hear many broadcast equipment companies stating that they are now “software-only”, but software still needs something to run on. What they really mean is that they no longer care what hardware their systems run on, be it CPU, GPU or FPGA, or where it resides. They just need it to be easy to deploy and use, easy to integrate and provides the required performance for handling low latency, multichannel HD, 4K or more.

Deploying hardware such as the Xilinx Alveo™ PCIe accelerator cards either on-premise or in data centres has meant that production teams can benefit from high-density and high-quality encoding and transcoding for OTT streaming, or accelerated multi-channel video processing in production. Abstraction means that the software environment just needs a well-defined API. The key to agility in either data centre or on-premise embedded media systems is the use of complete software stacks that enable abstraction and portability using common, open frameworks and libraries such as FFmpeg and GStreamer. The addition of microservices deployment to distributed hardware using Docker containers and Kubernetes orchestration gives even greater flexibility, management and control. This all means that broadcasters can effectively fire up services only when needed, or where needed and still benefit from the performance required to handle 4K video processing.  

The New Normal

As the world hopefully returns to normal, it’s clear that attitudes will change. The pandemic forced innovation to speed up, with social distancing and travel restrictions requiring fast deployment of streaming-enabled technologies. Despite the downsides, remote working has shown itself to be a viable option for productivity. The technologies that have been used by remarkably innovative companies in the creation and distribution of media content and collaboration have proved that decentralized workflows can not only work successfully, but can be more efficient, more scalable and more agile. Enabled by AV-over-IP and compression, with abstraction using multimedia software frameworks, media companies can more effectively work with distributed content creation to keep consumers connected and entertained.

Vizrt Leverages Monarch EDGE for Remote & Cloud Production

The Monarch EDGE encoder and decoder pair powers remote and cloud-based productions by securely transporting high-quality live video across public Internet

VizrTV Goes Live on Location(s) with Matrox Monarch EDGE

The Matrox Monarch EDGE encoder and decoder pair powers seamless remote and cloud-based productions by securely transporting high-quality live video across public Internet for VizrTV

How do you make two people thousands of kilometers apart appear side-by-side in a virtual studio instantaneously? Teleportation technology. While it may sound like science fiction, Vizrt regularly brings people together from separate locations across the globe with help from a pair of Matrox® Monarch EDGE 4K/multi-HD encoder and decoder devices when producing its own VizrTV online series.

The Monarch EDGE encoder and decoder pair alongside Vizrt’s software-defined visual storytelling tools for broadcasters allowed Vizrt to ‘teleport’ resident experts Chris Black, Vizrt’s Norway-based Head of Brand and Content Team, and Gerhard Lang, Vizrt’s Austria-based Chief Technology Officer into a virtual studio in Vomp, Austria – along with guest experts from anywhere in the world. With the ability to seamlessly execute both remote and cloud-based productions during different episodes, the Monarch EDGE has become Vizrt’s go-to broadcast-quality encoding and decoding appliances for delivering low-latency, strikingly-realistic virtual interviews.

No travel? No problem.

Due to COVID-19 travel restrictions, what began as a means of internal communication to unite and inspire more than 700 Vizrt employees in 30 offices around the world quickly became VizrTV: an external platform focused on issues currently impacting the broadcast industry while demonstrating how Vizrt’s broadcasting tools can help solve them in real-time. While broadcasters around the world were unable to return to their broadcast studios and forced to report the news from their own homes, Vizrt’s in-house and guest experts found themselves in the exact same predicament.

To show the world how Vizrt’s software-defined tools can help broadcasters create a more realistic live reporting experience for their viewers, Lang did not want to simply talk about the tools. Instead, he wanted to demonstrate their capabilities during live interviews. “We wanted to create a talk show format that looks natural and resembles what viewers are accustomed to seeing on TV,” said Lang. “Zoom-style interviews may become the norm, but with our technology, we can do better. We wanted to inspire our customers by showing them what is possible. How do you bring two storytellers together in one place? This is the question we were trying to answer when we decided to use the Monarch EDGE encoder and decoder devices. The 4:2:2 10-bit video that the Monarch EDGE provides was critical to allow us to achieve the quality of keying we needed to make the production truly convincing.”

The Monarch EDGE encoder and decoder pair met all of Vizrt’s requirements for a high-performance solution that would allow them to create the image of multiple individuals having a seamless interview in one location despite their physical distance from one another. Vizrt’s ‘location’ for VizrTV is a virtual studio, exquisitely rendered by Vizrt’s Viz Engine 4.1 compositing, real-time 3D rendering, and video playout platform. An essential tool to realizing this illusion is the Monarch EDGE, which is able to deliver 4:2:2 10-bit H.264 streams that translate to flawless green screen compositions. Furthermore, SRT support on the Monarch EDGE encoder and decoder pair allows for ultra-low-latency video transport over public Internet – and ultimately – a smooth and realistic viewing experience for those watching the VizrTV live stream.

Bringing storytellers together

For VizrTV episodes featuring a one-on-one dialog between Black and Lang, cameras capturing each individual are stationed in separate studios in Bergen, Norway and Vomp, Austria, respectively. During these productions, the Monarch EDGE encoder was housed in the Norway studio, while the Monarch EDGE decoder was located in the Austria studio. SDI cameras send HD video along with embedded audio to the Monarch EDGE encoder – which encoded at 6 Mbps and streamed in SRT – with delivery latencies that allow for a natural flow of conversation. The Monarch EDGE decoder received the stream, then output an HD-SDI feed with audio to the Viz Engine 4.1 system housed in the Austria studio. A second SDI camera in studio sends SDI video with embedded audio to the same Viz Engine 4.1 system, which generates the final composition for delivery to a Viz Vectar switching system. This system provides cuts between clips and live and encoded program delivery to Vimeo, Facebook Live, and LinkedIn, as well as proxy feedback to Black for program monitoring on his laptop. Although the Monarch EDGE encoder and decoder pair does have an independent, bi-directional analog audio circuit available to users, Black and Lang opted to use Microsoft® Teams for their real-time audio communications during these broadcasts.

Productions involving three participants follow a similar setup with an encoder required at each remote location. A single Monarch EDGE decoder and take up to four streams. In an example of a three-participant VizrTV production, Dr. Andrew Cross, President of Global Research and Development for the Vizrt Group, appeared on screen while being filmed in San Antonio, United States, alongside Lang and Black. The Monarch EDGE encoder was used to capture and encode the feed captured by the camera in the U.S. studio and transport it to the Monarch EDGE decoder in Austria. There, the device was also used to decode feeds coming from the other Monarch EDGE encoder in Norway. The end result was three individuals appearing side-by-side-by-side in a realistic virtual studio environment rendered by the Viz Engine 4.1. Despite participants being located in three separate cities across the globe, VizrTV was able to deliver a realistic live interview.

More than just great technology

The Monarch EDGE encoder and decoder pair’s ability to encode multiple 4:2:2 10-bit video feeds and transport them using SRT over public Internet with ultra-low latency has made it easy for Vizrt to create a realistic virtual studio setting for VizrTV, in which multiple individuals can come together from separate locations and in real time. “We wanted the end result to be a seamless viewing experience for the people watching,” said Lang. “We didn’t want the viewers to say, ‘Wow, this is great technology.’ Instead, we wanted seamless interaction with the video so viewers can focus on the story. We could not have accomplished that without the Monarch EDGE encoder and decoder.”

Lang noted that Vizrt plans to leverage the Monarch EDGE encoder and decoder for its upcoming VizrTV productions. “We are looking forward to being able to use the Monarch EDGE devices alongside our own solutions in future VizrTV episodes and more,” he said. “We are eager to see what else we can accomplish with this dynamic combination.”

EMG diPloy: SDNsquare Case Study

The customer

EMG (Euro Media Group) are a leading global provider of broadcast services and media solutions for live sports, entertainment and events across the world. With 35 years of experience in the industry, and a team spanning ten countries, EMG’s expertise sits at the heart of many of the world’s biggest productions, including the Tour de France, Football World Cup, Formula One, Eurovision and X-factor.

The context

EMG’s core business proposition is the ability to deliver productions from events across the world, making use of remote and now, increasingly, distributed production setups. In 2017, it became clear that reliance on legacy SDI systems would be prohibitive in achieving the company’s goals, and that a move to IP constituted the next logical step; allowing EMG to yet further reduce the amount of equipment and personnel which needed to be transported to remote sites. Not only would this represent a key element in reducing associated production costs, but more crucially, it would also support EMG’s environment and sustainability goals – reducing the carbon footprint associated with the transport of kit and people: a factor of importance for both EMG and their clients.  

A further benefit of the distributed production abilities facilitated by IP relate to the past year’s need for socially distanced production – a concern which may prove to have ongoing relevance in the field of broadcast for some time to come. IP allows for production professionals to work in their own specialized environments, in real-time, without needing to coordinate within a single OB van or on-site studio.

In addition, a key element of EMG’s success is a focus on creative vision; delivering the excitement and emotion of live events to viewers across the world. This means that the technologies which they chose to implement would need to be able to deliver the highest level of performance; reliably and in a way that does not increase complexity or learning curve for those implementing and configuring the system on projects.  

At the heart of EMG’s growth would be the ability to create a solution that allowed them to implement IP-based production across the entirety of their operations in a modular, flexible, reliable and cost-effective manner.

The need

With the above factors in mind, EMG developed the diPloy system,  based on an ‘Anywhere, Anyway, Any size’ concept, which would sit at the heart of their live remote production and video editing offerings. The diPloy system – a modular, standardized, scalable and flexible IP platform, was imagined back in 2017; a progressive approach at the time considering the complexities that existed regarding the homogenization of standards.

Key to EMG’s ability to achieve success with diPloy would therefore be the ability of vendors who were contributing components to the system to integrate the relevant standards and protocols of each modular element, correctly and on time. In particular, EMG were determined to have the system fully functional for the major sporting events of 2020. However, with the inevitable postponements that the COVID crisis brought, it was the 2021 FIS Nordic World Ski Championship in Oberstdorf which provided the ideal opportunity for EMG to showcase diPloy’s abilities.

The Solution

In identifying a suitable strategic partner, EMG looked to SDNsquare to coordinate their IP system through deployment of the GRID Network Controller; a Software Defined Network controller which guides the path for all data-streams and creates predictable, low latency flow.

The key outcome facilitated by GRID is the predictable and optimal management of the network, providing for reliable, real-time production operations which are not disrupted by data transfer issues or blockage – using intelligent management to deliver predictable and reliable data flow.

Moreover, GRID works towards facilitating quick and automated configuration of an IP infrastructure in accordance with the needs of each of EMG’s projects, accommodating modular and scalable setups, and using an interface and approach to configuration that is intuitive, easy to learn and based on WYSIWYG representations of network entities.

The specifics

Within EMG, all traditional OB units are divided into individual modules – each with different functions:  Dedicated Audio & Comms-Modules, Video Switcher-modules, Replay-modules and Basestation-Modules. Tailboard Modules guarantee connectivity with the outside world. All modules are combined according to the Networked Media Open Specifications (NMOS) of the project and connected using SMPTE ST2110.

diPloy’s control software is designed around a Northbound and Southbound system. The Northbound Control is based on familiar broadcast control software, interfacing with the Southbound Controller that runs hidden in the background, communicating with switches and routers. This is powered by GRID; guiding the path for all data-streams to facilitate the above-mentioned predictable low latency data-streams which are crucial to the ability to engage in distributed live production.

The key advantage facilitated by GRID is that it allows for this to occur even in a many-to-many environment, using a redundant MESH network rather than the traditional IT-based spine-leaf approach. This allows for an unrivalled level of operational flexibility; accommodating more than 250 switches and close to 2000 end devices, whilst monitoring, detecting and rectifying cabling and address allocation issues, all through the use of a fully location-based DHCP server.

The benefits

SDNsquare’s GRID has been involved in helping EMG to realise their diPloy vision, and thus working towards the benefits associated with IP-based production. More than simply facilitating a ‘straightforward’ IP setup though (a challenging enough undertaking in its own right), SDNsquare have provided a solution which will allow for a straightforward, frequently automated approach to network setup and facilitates complex, scalable and flexible configurations. In this way, EMG can adopt a modular approach their operations, setting up diPloy in accordance with the specifications of each given production project.

The result is a production environment which gains all of the benefits of IP – including future-proofing for UHD, HDR and beyond, which can be accessed and engaged with from anywhere in the world, in real-time. Even within a traditional production environment, EMG are able to benefit from the fact that an IP setup results in a significantly lower number of cables. This reduces what the German’s have helpfully named ‘Kabelsalat’; the tangling of numerous wires, since with the GRID powered diPloy, each production area requires only one cable core. This makes for a more streamlined, reliable, lightweight and safe working environment.

The central advantage for EMG has been the ability to more closely manage the costs associated with production; the modular approach meaning that technology deployment can be scaled to meet the size of the project, and does not need to be transported half way across the world.

In addition to the financial benefits though, EMG have also been able to meet their sustainability goals – both through the reduction of transport-associated carbon footprint, but also because when implemented with ToR switches, GRID can reduce the overall power consumption of a standard network by up to 45%.

Ultimately, the use of GRID has allowed EMG to deliver a better, more cost-effective, more reliable and higher value level of service to their own clients. This was proven in the field during deployment of the system at the Nordic World Ski Championships 2021 in Oberstdorf, where the client commented on the fact that EMG’s systems ran smoothly and without disruption, delivering the professionalism of production expected, whilst facilitating crew safety (through distancing), and operating in a challenging environment (namely, the snowy Bavarian Alps).

Client observations

Speaking for EMG, CTO Bevan Gibson observed: “The key benefit of IP technology is that it allows the creation scalable, flexible systems which are easily capable of remote and distributed production workflows. The idea with the modular nature is that it’s fast to install, fast to prepare, very flexible in what it can do — and because the connectivity between modules is IP it’s far simpler than legacy infrastructure based on SDI. Whether it’s distributed locally across a couple of kilometres, or distributed to 5,000 kilometres away, the diPloy system utilising GRID, is designed to handle both operations.

Moreover, aside from the technical benefits that the implementation has realised for us, increasingly, our clients demand a solution for their event that is not only modular and scalable, but also environmentally sustainable. We strongly believe that nowadays, real innovation means reducing our ecological footprint – an idea that is echoed by SDNsquare”.

Commenting on their own role within the collaboration, Henry Alexander said for SDNsquare: “diPloy with GRID allows sports federations and sports event organisers greater opportunity to optimise the on-site resources necessary to deliver the highest quality of broadcast coverage, whilst also promising a more sustainable approach to broadcast operations”.