Managing the Unpredictability of Live Cricket During the Pandemic

By Chris Clarke, CEO, Cerberus Tech

Cricket, like many other sports, has been seriously impacted by the social distancing restrictions imposed by the pandemic. The postponement and cancellation of matches at the height of lockdown, through to the more recent challenges of positive Covid tests, all caused a huge amount of disruption.

In May 2021, The England & Wales Cricket Board (ECB) announced a loss of 16.1 million pounds ($22.78 million) for the past year, with revenue dropping while Covid cases rose. Despite the worst-case scenario for the sport being avoided with a full programme of successful international cricket during summer 2020, teams, organisers and broadcasters have endured their fair share of upheaval over the last year and a half.

Predicting the Unpredictable

One of the challenges which goes along with all live sports during the pandemic, is the inability to effectively plan ahead. Alongside the logistics involved when managing short notice changes to matches and subsequent booking alterations, international teams have been subject to a variety of last-minute shifts, which have left content owners and broadcasters scrambling to adjust.

That said, cricket broadcasters are no strangers to some of the uncertainties that regularly accompany live matches. Unlike fixed-duration sports, the length of cricket cannot always be reliably predicted, so the need for some flexibility is to be expected. Unfortunately, this isn’t always possible with satellite and fibre delivery. Often broadcasting capacity needs to be overbooked, in order to safeguard against matches running longer than expected.

The Importance of Quick Implementation

The challenges facing international cricket were highlighted recently when our team was tasked with quickly turning around a live delivery request for six separate events from the Sri Lanka vs England Cricket tour.

Our team was approached by the national television network, The Sri Lanka Rūpavāhinī Corporation (SLRC), following referrals from Sri Lanka Telecom (SLT) and  Dialog TV (DTV), a direct broadcast satellite pay TV service provider based in Sri Lanka. SLRC’s initial request was to pick up and transport a feed from BT Tower and our team advised delivery of a main and backup h.264 encoded linear Zixi feed to the broadcast partner.

Even though the live event deadline was just three days after our initial conversations with SLRC, we were able to turn things around without any issues. BT Tower pick-up details were provided the day before each match, and the nature of IP delivery meant that the team could remain responsive throughout the entire process. As SLRC is well-established, the necessary hardware set-up was already in place to enable all content to be received in a Zixi format. The Infrastructure as a Service (IaaS) workflow to transport the feeds took less than an hour to deploy and could be connected at the start of each match. By using a protocol agnostic approach to delivery infrastructure, IP engineers can adapt to any broadcast requirement.

Despite working across different time zones, there was no need for our team to be on-site, and the entire deployment was undertaken remotely. Conversations on any adjustments to the infrastructure were straight-forward and the language barrier did not prove to be a challenge. The engineers were able to send a few brief messages to organise and run the set-up, and IP proved to be an international language of its own, which made managing the feeds very simple.

A Scalable Set-up

In total, 40 hours of coverage was delivered, across five days within a 2-week period, featuring a mixture of T20 matches between Sri Lanka and England, as well as some one-day internationals. The initial request was for 36 hours of content, but this was then scaled as the live event progressed and matches over-ran.

By using a cloud-based expand-on-demand environment, we could ensure that SLRC only needed to pay for the infrastructure they required, and this could be adjusted at a moment’s notice. A key benefit of delivering live events via IP is that capacity can be allocated at an extremely granular level. In one case, when a match was rained off, we were able to move the IP infrastructure into a stasis mode, so there would be no need for the customer to pay for that coverage. With scheduling changes happening for events all the time, this sort of responsiveness will prove crucial for live sports over the coming months.

Looking Ahead

This isn’t the first project we’ve undertaken in Sri Lanka and the team has found the response to IP in the region very encouraging. The cost-effectiveness and technical reliability of IP delivery is extremely appealing for live sports broadcasters, who require a quick turnaround but don’t want to compromise on the quality of feeds. We anticipate that there will be an increase in IP adoption, as more live sports events make use of this agile delivery method.

Palitha Gallage, Deputy Director General (Engineering), The Sri Lanka Rūpavāhinī Corporation, commented on the project: "We are very satisfied with both the IP delivery of live feeds and coordination provided. We consider Cerberus Tech a reliable technical service provider and look forward to working with them for such future events."

It is clear that the sports industry is continuing to shift in response to Covid. However, the real question is, where will these changes lead over the long-term? The pandemic has highlighted the restrictive nature of traditional broadcasting infrastructure. Pre-booking physical requirements such as OB trucks or planning for fixed capacity satellite delivery, has proven to be extremely difficult to manage during periods of uncertainty.

Now that next-generation IP solutions have matured within the market, they are poised to tackle the problems facing the sports sector. By maximising the operational efficiency of IP, sports broadcasters and content owners can transport feeds cost-effectively around the world. This allows these organisations to remain responsive for years to come, by changing the way that content is broadcast and future-proofing their delivery infrastructure.

LED for virtual production in film and broadcast applications

Choosing the right product for in-camera VFX starts with choosing the right partner

Virtual set technology has experienced a coming of age for both film, broadcast, and events applications. The last two years have resulted in a steep learning curve for virtual production technology, forever changing the way content is made.

How to find your way in this new area that so heavily relies on technology?

The use of image output from real-time engines to a live LED wall in combination with camera tracking to produce final-pixel imagery, completely in-camera, represents the state-of-the-art for virtual production, but what application best works for you?

Contrary to what you might think, this starts with looking for the right partner, not the right product.

The right partner will consider your requirements and test the combination of products selected before making the final choice. The use of image output from real-time engines to a live LED wall combined with camera tracking to produce final-pixel imagery, completely in-camera, represents the state-of-the-art for virtual production and asks for considerate testing, syncing, and fine-tuning of the products on set. The best results can only be obtained if the LED screen, LED processor, camera, and media server are meticulously aligned. ROE Visual strives to optimize every aspect of the technology for the creatives behind each project.

ROE Visual won’t just give you a box and wish you good luck, their support ranges further. Partnering with all the leading players in the field, including ARRI, disguise, and Epic Games (Unreal Engine), and through combining knowledge, endless testing, and syncing the equipment used, optimal results are achieved. Synchronizing input sources to the camera and playback on-screen is critical to the success of any production using virtual production technology

“Building an LED panel is not that difficult; building one with the quality and reliability demanded by media and film producers is an order of magnitude more challenging.”

Consistency and quality are a hallmark of ROE Visual’s LED technology.

Replaced panels need to have the same quality ingrained and must fit with the existing screen setup. ROE Visual supports install and setup with 1-1 training sessions and with technicians on-site as a standard procedure. Issues to be worked through upfront include Pixel Pitch, genlock, refresh rate, and color accuracy. Don’t go by the numbers provided by any manufacturer; it’s of paramount importance to ask how these are working out in your complete setup and the type of shots you require.

“The production should not have to worry about the quality and reliability of the LED, but just concentrate on creating their vision on set.”

ROE Visual has considerable pedigree in both broadcast as well as film applications.

As a designer and manufacturer of LED screens for many years, ROE Visual’s technology helps rental companies in film and TV production as well as permanent installs at the world’s most prestigious studios. Implementing the latest technologies, such as GhostFrame™ and supporting its renowned client base with the best engineering and support. They could be the right choice for you.

Democratization of Business Improvement

Emilio L. Zapata
Founder Tedial

In a recent report “The Evolution of Production Workflows”*, MovieLabs asks "Would it be nice if software-defined workflows could be assembled as interconnecting children’s blocks, where integration is as simple as connecting the pieces in the desired configuration?”  To make workflows in the Media & Entertainment (M&E) market more flexible, it proposes defining a minimum set of standards and practices for workflow interactions, thus promoting interoperability and minimising the work required to quickly create a custom workflow. In this way, "the creatives decide what must be done and which workflow components are interconnected to them".

In the M&E market there are media processes that can be complex because media files reside in different storage spaces, are in different formats, must be processed with specialised tools and are the fundamental component of application-to-application integration. In addition, there is a need to minimise media file movements between systems, ensuring that it is the applications that go to the media and not the media to the applications.

A software-defined workflow uses a reconfigurable set of tools/applications and processes to facilitate creative tasks by connecting them through collaboration and automation via software. We can easily deduce that the flexibility demanded in the previous question cannot be achieved by classical point-to-point integrations due to their complexity and lack of flexibility, among other risks. Consequently, the integration paradigm has to be changed.

When we analyse the processes that exist within an M&E organisation, we tend to focus on those that stand out in complexity. On the other hand, we neglect the more elementary and less visible processes that tend to be simple, informal and ad-hoc in nature. The graph below shows a rough distribution of processes in an M&E organisation. There is a large majority of simple processes that should be implemented without the need to go to the IT department. In other words, users should be able to design all processes that do not require programming knowledge.

We are transitioning to a broader and more diverse range of software, IP, cloud, and cross-platform technologies for the M&E market. There are more tools than ever for every part of the workflow. More and more software applications are needed and companies have to turn to a greater number of providers. Trying to make everything work and stay connected as software is updated or new tools are added is becoming a tiring task. It is the responsibility of technology suppliers to provide an enhanced ability to create software that generates tangible business results and accelerates fundamental cultural change for companies.

Media Integration Platform

Often people don't know what they want until they see it. When this happens, software projects take too long. Then come the change requests and the problems associated with possible breaches, because too often users include everything they can think of in the technical requirements specification (RFP), assuming that everything is possible.

The question is, how do you get users and the software team to work together in the initial phase of the automation project using software-defined workflows? The best way to start designing a solution is to think of simple processes (prototypes) and start implementing them, then make adjustments to the prototype. A prototype verifies that business ideas work, analyses how to improve the prototype and can define additional functionalities as needed. In this way, ideas can be expressed quickly and it would not take months for the development team to understand the RFP and transfer the ideas to the application.

A Media Integration Platform represents a new generation of applications that allow the time it takes to develop software projects to be drastically shortened. This is because the platform facilitates communication between users and development teams, creating prototypes that visualise the needs of the business. We are talking about the difference between many months to two or three weeks in the development of a project.

A Media Integration Platform allows sophisticated business processes to be visually composed by dragging and dropping components (applications and user tasks) into the design area (canvas) and then configuring their functionality, speeding up process development. A Media Integration Platform should allow users to design at least 70% of the software-defined workflows in the graph above without knowing how to program.

A Media Integration Platform is ideal for implementing the development of solutions based on software-defined workflows in the M&E market, because it integrates the different applications and the people involved in each process at the metadata and media file processing level, as well as organising the work of the users.

In fact, a Media Integration Platform allows both the exchange of metadata between applications and the efficient development of workflows for receiving, indexing, archiving, exchanging, transforming, producing and distributing content in multiple formats (codecs, components, segmented) and qualities. Examples are the preparation of content for multiplatform distribution, content localization or the automatic indexing of content using artificial intelligence tools.

Thanks to Media Integration Platform technology, M&E organisations no longer have to start from scratch or wait for IT to build, upgrade or enable the digital transformation of legacy applications. A Media Integration Platform allows both "technical" and "business" users to create any type of process, from simple to complex, without writing code. With a Media Integration Platform-based solution, technology development becomes more agile, collaborative, dynamic and responsive to customer needs including:

  • Ability to translate business requirements and outcomes into technology solutions.
  • Unprecedented flexibility, speed and agility to adapt to changing customer conditions.
  • Maximises efficiency and reduces costs, improves profitability and accelerates growth.
  • Dramatically reduces delivery times.

Democratise is a strong word, Media Integration Platform technology democratises solution development and accelerates innovation in the M&E market. By combining different skills, "technical" and "business" users, they break down functional silos and hierarchies and help drive innovation and business agility.  We believe automation should be easy, because everyone deserves to work smarter, not harder. Our mission is to democratise process automation in the M&E market, where innovation and agility are needed and business processes need to be rethought or refined quite frequently.

Market and consumer behaviours are changing faster than ever before. We are at the forefront of this technology transition and perfectly positioned to enable next generation user experiences, creating solutions that leverage the potential of the cloud, maximise interoperability and enable users to define their processes autonomously and create workflows in a flexible and agile manner. Without a doubt, we are getting closer every day to providing a solid and practical answer to the question posed by MovieLabs.

*Reference: White paper “The Evolution of Production Workflows” https://movielabs.com/news/6334/

Meaningful metadata – the international standard

Chris Wood
CTO, Spicy Mango

In the last 36 months, the way in which machine learning technologies have advanced is incredible. As the world moves to more automated ways of working, I’m going to dive into how media supply chains are shaped, driven and limited by data.

AI and ML are always billed as the saviour. As buzzwords on an array of the latest product datasheets, occasionally there is a reason to see why. The ability we now have to analyse video, whether this be clips or entire programmes, and generate meaningful metadata is second to none. From a short clip of some relatively mundane motoring content, we can identify people, places, and objects and not just at a high level. An implementation we’ve looked at recently can not only extract the colour, year and model of a Ford motorcar, but also tell us that there’s a wheel, a headlamp, a mirror. Additionally - the metadata produced can be timed – so we now have a system that can tell us not only that there’s a car in the video, but at what point in time the car arrives in the scene.

Moving forward, what do we do with this information? Can we make search more insightful for our users? What about counting occurrences of objects to influence recommendations? Linking to parts catalogues or online stores? How about generating caption details on screen or content categorisation and tagging? All incredibly valid use cases we couldn’t have dreamt of five years ago. Great benefits to the end consumer, but what about the precursor – and making this content available in the first place?  

When media supply chains are built, they rely on a few elements: primarily, essences of video and audio, and a metadata payload to be able to identify them. Through AI/ML, our ability to generate and augment that metadata to include more useful information about the contents of the video is hugely useful. Downstream systems can make decisions in real time during ingest and processing around what to do with that asset such as where it goes in the catalogue, how to categorise it into the correct price point or tier and so on. In the case of sports content, identification of a goal can generate a clip from a highlights programme – or even the reverse, and automatically publish this to your OTT platform of choice.

On the technical front, our ability to analyse a piece of content and understand its makeup have been with us a while longer. Generation of a file size, length, codec, resolution, aspect ratio data are all known entities now. The smarts here relate to the way in which we use this information to drive downstream decision making. In OTT, we commonly leverage this data to make meaningful transcode choices, formatting for the correct devices and platforms.  

There is no doubt that the use of automation and analytics technologies will help a great deal – but there’s still a gap to fill that the technology isn’t yet able to bridge. To make best use of automation, simplify our content chains and delivery ecosystems, driving home the need for an international standard for high quality metadata at source is still the key.

The modern supply chain

Having explored what these innovations mean for the consumer, how do we start to think about what they look like for the industry as a whole?

In order to make this useful we need to understand what supply chains look like in today’s world (if you’d have asked me this question 15 years ago – you’d get a very different answer). The major brands we love and know are content businesses. Sure, they build and own technology and products, but fundamentally what is being delivered to the consumer is content. It’s the ‘product’ we’re all buying.

The notion that content starts and ends life within the same four walls doesn’t apply anymore. In fact rarely is it even the same organisation! Assets are now transferred around the globe between content producers and service providers on a many to many basis. Our world is pillared by licence deals and syndication agreements, so the need to move assets in a supply chain is no longer as limited as it was when television had four channels and everything was taken care of under one roof.

Supply chains are now more complex than ever. Every organisation moving content and metadata operates with its own standard. Many of these standards are driven by either what is required to support processing or driven by years of platform development and integration with a variety of tools and systems. Change is often slow, and it’s very hard to adjust a system and workflow that hasn’t been designed from the ground up to be modular and support dynamic change without affecting everything around it.

Efforts like the DPP initiative in the UK have worked hard to try to introduce standards and simplify the asset logistic challenges. In looking at the member roster, and having worked with a number of these partners over the years, many are still some way away from a seamless unified approach to logistics - highlighting how complex and fragmented the delivery ecosystem is. 

Take this problem and multiply it for every syndication partner that’s using a different format and delivery method, rights and license rule variants, and we start to see how big the challenge is with many custom integrations, content and metadata transformations all needed. Despite efforts, no one is singing from the same hymn sheet.

Lastly, there’s one other challenge that we haven’t yet touched upon. This is the availability of metadata (and I mean good metadata!) from source. Having been in this industry for a long time, I’m still surprised at how many supply chains are driven by Excel, PDF and Word documents; assets arriving via FTP with a basic document (sometimes even forwarded as an email) that incorporates no more than a title, description, series name, season identifier and episode number. Do we have a supply chain? Sure, but it’s pillared by large teams of people manually inputting data, often consolidated from disparate third party sources that are yet to be integrated.

AI/ML technologies haven’t yet evolved enough to analyse a piece of content and tell us who the director, the producer, or wardrobe assistant was, or even who the file should be syndicated to and when. What are the license rules and distribution parameters?

Despite everything we can do with technology, the ability to create structured data that we can use downstream is still limited if what is received from source is either incomplete or inaccurate.

So what is a data driven supply chain? I’d argue it’s the process of using data and information to make decisions on media asset logistics. Does this need AI and ML to get there? Not at all.

How to attract, acquire and retain subscribers to your OTT service

Silvia Werd Elías
Marketing Director, Jump Data-Driven Video

One of the constant challenges of OTT services is how to attract and acquire new subscribers while holding on to existing subscribers. Thanks to improvements in technology, user behavior can now be better forecasted and anticipated with help of predictive analytics, using advancements in ML and AI.

The world of OTT services is increasingly competitive, as more and more  companies adopt OTT to deliver their product. Platforms such as Disney + and HBO Max have recently taken this step to compete with big names like Netflix, Amazon Prime and Hulu, platforms that have dominated the video-on-demand subscription market for years.

How does one compete in the OTT services market?

There are now greater opportunities to contend with giants like Netflix and Amazon Prime than ever before. To a large extent this phenomenon has been  brought on by recent events such as the closure of movie theaters and performance venues or the suspension of sporting events and even concerts, creating a gap in both challenges and opportunities for OTT services.

One of the keys to being successful in OTT services is the ability to deliver engaging and relevant content while fully understanding your audience. For an OTT service to take root, it must be able to accurately predict user behavior, generate personalized content, reduce acquisition costs and increase CLTV or customer lifetime value by 100 %. These are all goals that can be achieved through AI applications and smart data optimization.

At Jump Data Driven Video, our expertise is in the use and management of intelligent data to improve your understanding of your user base. We can provide you with a set of specially-designed data management tools to optimize your OTT service through personalization, increased user engagement, customer retention and acquisition, along with other strategies to shape the key differentiators that will propel you to the next level.

Acquire and retain subscribers to your OTT service

The main success of OTT services is in the ability to offer content that maintains user interest, thereby improving their participation and interaction with the platform, and to increase the number of new subscribers while keeping the churn rate down.

Jump helps you understand more accurately the status of your video service by giving you a view into the journey your customers take. This will allow you to improve your marketing strategies and offer personalized and relevant content to generate an impact on the specific groups of users you want to target at a specific time with a personalized offer.

What follows are the main performance indicators to quantify and measure the effectiveness of actions and strategies  to compete in the OTT services market.

Acquisition

This process is basically focused on meeting customer expectations. In the past, OTT services lived by the rule that it costs five times more to acquire a new customer than to maintain an existing one, but this rule no longer holds true because of predictive analytics that determine future behavior through the application of AI and ML.

A fundamental acquisition strategy  is to connect with the customer and give it a value in both the present and the future, known as Customer Lifetime Value (CLTV). Video industry OTT services have recently seen a notable increase in the number of new subscriptions, and this trend is expected to continue over the next few years.

According to the Parks association, one of the main factors that drives new subscriptions is to offer content that the customer wants to see, be it a variety of programs or a specific program. Another factor to success is to offer a free trial month. Customers tend to keep a service because they like the content it offers.

Here are some more factors to evaluate in the acquisition phase of the OTT service.

Acquisition of services

Quantifying the number of new subscribers is important, but other factors must also be taken into account, such as the acquisition of services, knowing how and when  new customers sign up, as well as the type of service or package subscribed. This will help you forecast your customer’s value, revenue, and future plans.

Service discovery

This phase is key to knowing how your subscribers reach your service, to help you determine and implement the most viable channels and methods you should invest in to get more subscribers.

UAP

When we talk about UAP, we refer to the performance of the user´s attribution, the capacity and amount of subscription that will allow us to determine if the acquisition channels are paying off, and thus implement specific campaigns, managing to personalize the acquisition.

Engagement or commitment

This evaluates new customer behavior and how they may commit to the service for as long as possible. This can only be achieved by enhancing their experience of interaction with the OTT services by offering relevant, personalized content. One way to achieve this is with content recommendations that make it easy to find titles and allow the customer to discover new programs.

Tracking this behavior will help you to predict the customer’s future actions and offer an individualized experience. The implementation of AI is really useful in this area because it allows you to segment your users based on their preferences and level of participation, facilitating the creation of groups of users based on the type of content they have in common, and creating specific and personalized campaigns aimed at any target audience.

Jump’s prediction system will help you identify which of your potential customers may make a long-term commitment and become a regular customer of the service, and at the same time predict which users are most likely to convert, offering you an effective, solid conversion strategy with immediate and concrete actions.

Retention

There are many reasons why someone may abandon an OTT service. With Jump’s strategic prediction system you can better evaluate retention performance indicators and other KPIs  that will help you create the strategies to improve retention. Knowing your user makes it easier for you to accurately predict the most likely moment they may decide to leave the service, allowing you to generate a quick response that reverses the situation, which in turn will increase CLTV.

The more committed a user is, the greater the CLTV. This can produce a notable impact on ROI, which is why user retention is one of the main factors of success for OTT services.

Taking on the growing challenge of rising content volumes and a spiking carbon footprint

Sergio Grce
CEO, iSIZE

The amount of video content being distributed is only going in one direction – and that is up. Over half the global IP video traffic (56.8%) will be HD and around a quarter (22.3%) will be Ultra HD by 2022 according to Cisco. This demand for high resolution video inevitably requires a trade-off somewhere along the line - either in terms of bandwidth or to the end user experience. Higher resolution video, which consumers increasingly expect as standard, also typically means higher bitrates, which can result in slow starts, video buffering and high content delivery network (CDN) and storage costs. This is bad news for the viewer and bad news for the content provider.

The continuing surge in online media consumption means our industry faces two pressing challenges. First, there is unprecedented stress on network infrastructures worldwide, which not only creates content delivery bottlenecks, but also affects how content can be distributed efficiently to the ever-growing numbers of viewers. Second, this rapid rise in content consumption and delivery also has a huge impact on the industry’s environmental footprint.

In the perennial drive to balance efficiency and capacity, interest in the perceptual optimisation of video – in other words, the processing of digital video streams to deliver the uncompromising quality that users expect without a simultaneous uptick in bandwidth – is rising. Traditionally, the world of digital video has looked to compression technology to address these issues, working to increase the efficiency and sophistication of the codecs it uses – but this brings much higher levels of complexity and is highly processor-intensive.

We are now facing a situation where the increase in video encoding complexity is outpacing Moore’s Law. Even with more GPUs and CPUs capacity to encode video content, the sheer volume of content being produced - and watched - means we will very quickly outstrip the compute cycles available. In parallel, the carbon footprint of the internet is estimated to be greater than that of the aviation industry and is something we need to address.

As a company, we believe that the only way we will meet the growing demand for online video, reducing processing, energy, and storage requirements is through disruptive innovation for video streaming. For us this takes the form of new deep perceptual pre- and post-processing, encoding and delivery tools that are device-aware and cross-codec compatible.

We are laser focused on helping customers solve the challenges they face, and we are working – through our own R&D efforts, as well as through projects such as the SEQUOIA R&D project partnership – towards this aim. Deep perceptual optimization of video streams is a key focus for us as a way of reducing the bandwidth required for equal quality, and iSIZE has built up extensive expertise in this domain.

A unique approach for an urgent challenge

ISize believes that the increasingly urgent challenge of finding trade-offs between the various metrics, between bitrate and perception and between more content and the need to lower the environmental impact – all while managing processing and encoding complexity - requires a unique approach. Instead of relying on more complex codecs and greater GPU/CPU capacity, we believe the more sensible route is to reduce the bandwidth needed for high-quality video streaming. We have directed our patent-pending artificial intelligence (AI) features and machine learning, combined with the latest advances in perceptual quality metrics to this aim. By reducing the bits required for elements of the image that perceptual metrics tell us are not important to human viewers, our technology innovation can deliver perceptual quality that is optimally balanced against encoding bitrate.

If we are to make real headway as an industry, the most effective and efficient approach is to implement a server-side deep perceptual pre-processing enhancement that enhances details of the areas of each frame that affect the perceptual quality score of the content after encoding. In this way, we do not change the encoding, packaging, transport or decoding mechanisms – unlike solutions such as LCEVC. Furthermore, we can be fully compatible with any encoding, streaming and playout device with zero modifications. By using a method that is cross-codec applicable, codec-agnostic, and optimizes legacy encoders like AVC, but also HEVC, AV1 and VVC, we no longer need to know the encoding specifics of each encoder – and so can remove an added layer of complexity.

Most pre-processing solutions use sharpening techniques to deliver perceptual optimisation. iSIZE comes at the problem from a different angle; we maintain the perceptual characteristics of the source and eliminate the need for multi-pass encoding and in-the-loop integration used by many other optimization tools. We have created a single-pass, pre-processing solution that needs no metadata or integration with the subsequent encoding engine(s) and delivers significant gains in quality.

Deep learning for optimized results

iSIZE challenges the accepted norms within the video delivery industry by placing our technology before the encoder. We also ensure that our solution does not depend on a specific codec, and it optimises both for low-level metrics like SSIM (structural similarity index metric), as well as for higher-level (and more perceptually oriented) metrics like VMAF, Apple’s AVQT metric or AI-based perceptual quality metrics like LPIPS. In fact, we are able to deliver average bitrate savings – compared to the same encoder and codec - in excess of 20%. On top of this, our technology has been designed in a way that does not break coding standards; this means it can easily be used in existing distribution chains and with existing client devices without causing disruption to customers’ workflows. Thanks to the single-pass approach and agnosticism to coding standards, we are also able to ensure easy deployment on custom hardware or high-performance CPU/GPU clusters.

In a nutshell, we have created a methodology that delivers significant savings in two key areas. First, by reducing the bitrate required from a standard codec to deliver a certain quality level.  And second, if bitrate saving is not the only goal, our technology can be used to make the actual encoding much faster - up to 500%, or even faster in the case of VP9, AV1 or VVC encoding.

Leveraging our knowledge and expertise in AI and deep neural networks, we have elegantly answered one of the growing challenges faced by the industry: sustainable distribution of Ultra High Definition content, while limiting the impact of video on internet traffic and reducing distribution costs.

At iSIZE we believe that by proactively reducing energy consumption at all stages within the media value chain, this type of innovation can make a difference to every aspect of media distribution, delivering benefits for the whole sector. With efficiency a key buzzword and environmental ramifications a rising concern, the need to reduce energy consumption and eliminate complexity is front of mind for anyone who delivers content. We are already working with customers to roll our technology out in several vertical sectors, including gaming, social media, and entertainment streaming and will be making announcements in the months ahead.

3 Reasons Why Automated AI/ML-Based QC is Critical for the Future of Media Content Delivery

Penny Westlake
Director, Europe at Interra Systems

Media content delivery is changing. During the COVID-19 pandemic, OTT video consumption soared to new heights, and that trend will continue into the future. According to Acumen Research and Consulting, the global video streaming market is expected to grow at a CAGR of around 12.2% from 2020 to 2027, reaching a market value of over $843.1 billion by 2027.

Viewer expectations are not the only change happening in the video world. Workflow enhancements and technology innovations are transforming the way broadcasters and media companies create, deliver, and inspect the quality of content.

As broadcasters and service providers look to improve content quality and deliver the highest-quality viewing experiences to audiences around the globe, they will increasingly rely on AI and ML technologies, combined with computer vision techniques. Here are three reasons why.

Consumer Tastes are Evolving

Diversity and choice are abundant in the OTT environment. Today, consumers can watch media content on a variety of different screens, including TVs, smartphones, tablets, and PCs. There are also many different services consumers can choose from, including Netflix, Hulu, HBO Max, NBCU Peacock, Disney+, Amazon Prime Video, and more. Having so many options available has set the bar extremely high for quality of viewing experience.

Consumers expect exceptional quality content on every screen. Research firm Sensum found that a viewer’s negative emotions increase 16% while engagement decreases nearly 20% as a result of poor-quality streaming experiences. The survey also found that 76% of participants would stop using a service if issues occurred several times. With AI/ML-based quality control (QC) technologies, broadcasters and service providers can ensure that the experience they are delivering is of the highest quality, while also ensuring that processing time is kept to a minimum.

Content Volume is Growing

OTT technology has globalized content delivery. Through an OTT service, broadcasters and service providers can seek out new audiences and expand their reach. This means service providers are preparing video content in a wide range of different languages, which requires them to account for national and regional regulations, dubbing, and captions. Content must also be prepared to support the multitude of devices that exist today. Each device has a different screen size and supports different formats. Since there are so many variations to maintain and a massive amount of content to manage, broadcasters and media companies need more efficient methods for QC. ML/AI-based QC solutions speed up the QC process, allowing broadcasters to achieve higher levels of productivity, greater operational efficiency, and improved accuracy.

Modern Media Content Workflows are Complex

OTT video consumption has skyrocketed, and as a result video creation, preparation, and delivery processes have become much more complex, yet there is still a requirement to make the process as speedy as possible.

Traditionally, broadcasters and media companies relied on visual inspection methods to detect issues with audio and video streams. Now broadcasters and service providers are handling a higher volume of content, with multiple output requirements, and manual methods are too time-consuming and inconsistent. Using automated AI/ML-based QC solutions, broadcasters and media companies can prepare content faster, taking into account the need for multiple encoding formats, resolutions, audio, and captions in multiple languages, with audio suited for the different fidelities of end devices, and with multiple delivery mechanisms.

Applying AI/ML QC in the Real World

AI/ML used within automated QC processes enables broadcasters and service providers to operate faster and more efficiently, bringing increased consistency and reliability to certain media tasks, such as content quality checks, compliance, classification, content categorization, lip sync checks, and more.

Using a fully comprehensive, automated QC system, broadcasters can rapidly check the quality of myriad video and audio formats, as well as checking the quality of closed captions and subtitles. As broadcasters expand their reach into new countries, AI/ML-based QC solutions will help them to ensure that content complies with all industry and government regulations and address the various OTT and on-demand delivery ecosystem requirements.

AI/ML technologies are expected to play an increasingly crucial role in enabling broadcasters and media providers to generate metadata for content classification purposes. Content classification is important for VOD and OTT delivery, enabling broadcasters to censor certain types of content, identify celebrities, and detect the presence of brands or objects within content. Previously, this kind of analysis would have required human decision making at every level.

In addition, AL/ML auto QC solutions leveraging image processing, ML technology and deep neural networks can aid in quick, precise identification of lip sync issues and facial recognition, optimizing the quality of experience for viewers. Lip sync issues have long been one of the most noticeable errors that consumers find irritating, and can lead to consumer churn. The latest AI/ML models, designed for broadcaster and media company use, can speed up the identification of such errors across multiple content formats. With automated and AI models for data analytics, broadcasters can gain unique insights into viewing behavior and further enhance QoE.

Finally, good captioning, subtitling, and audio description have become an increasingly key requirement for media content today, and ML is effective at checking for the presence and accuracy of these services.

Conclusion

Given the rapidly evolving broadcast landscape today, changing viewer expectations, and massive volume of content that is distributed to global audiences, it is more important than ever for broadcasters and service providers to deploy auto QC solutions that offer more than basic audio and video checks. Broadcasters need solutions that offer increased efficiency, speed, and accuracy for classifying and categorizing content, performing lip sync checks, and adding captions. Recent innovations in AI and ML technology are paving the path toward higher-quality video experiences.

Live frame rate conversion in the cloud, on a pay-per-use basis

Paola Hobson
Managing Director, InSync Technology

Introduction

Broadcasters and media companies engaged in live international content distribution are familiar with the need for standards conversion. Multiple broadcast frame rates and formats are in use throughout the world; and with an ever-growing number of standards to support in mobile and streaming services, high quality, live, standards conversion is an essential part of many businesses.

OTT streaming service providers are no different. These delivery workflows provide localised broadcast grade streams to a myriad of devices globally. Typically, these workflows run fully in the public cloud, receiving mezzanine quality transport streams from playout; and inserting dynamic content to produce localized versions, where reformatting, transcoding, packaging, encrypting and delivering to CDN are entirely located in the cloud.

Moving the workflow to the cloud

M2A Media are innovators in cloud broadcast; they work with some of the biggest names in the industry by helping them to connect with new audiences, realise greater commercial benefit and reduce operational overhead. In 2020, M2A media delivered hundreds of thousands of live events to over 80 countries, streaming more than 1 billion hours to millions of concurrent viewers.

Typically, an OTT customer’s acquisition workflow might take live source content from events produced around the world and feed it into their playout service using IP transport in a public cloud. The customer will then want to deliver localized, broadcast grade streams to OTT devices globally. They would use a head end workflow running fully in the public cloud to receive the mezzanine quality transport streams, insert dynamic content to produce localized versions, reformat, transcode, package, encrypt and deliver to CDN with integration to a live ad insertion service. Both the acquisition and the head end workflows require frame rate conversion between 50 and 59Hz. 

Many of these services include live streaming of sporting events of international importance. Frame rate conversion (standards conversion) is therefore needed in order to manage the multiple delivery formats and frame rates for each region. For example, US Football acquired at 59Hz needs frame rate conversion to 50Hz for European viewers. When streaming live sports, very high quality frame rate conversion is essential to ensure all viewers obtain the best possible experience.

Event-based services

When provisioning services or investing in equipment, it's important to consider that sporting events might take place weekly for a specific season, or just for a two week period annually, or even for a short period every four years, etc. For such events, on-premise proprietary hardware represents a costly asset on the balance sheet which requires considerable effort and resources to be configured into the acquisition or head end workflows. Once in place it is costly to reconfigure hardware workflows so responding to last minute demands is typically not possible.

Broadcasters and media companies have shown a lot of interest in the integration of InSync Technology's FrameFormer motion compensated standards converter into M2A's cloud based services. This has created a unique service: where customers can access live frame rate conversion in a pay-per-use scenario and the service provider is orchestrating and gathering the end-to-end cloud resources needed to support thousands of live events each week.

The service also runs fully redundantly, with on demand capacity management, scaling and monitoring.  This is very beneficial for unexpected situations e.g. event starts which get delayed by bad weather or over-runs such as extra time in a football match.

This type of pay-per-use standards conversion service isn't exclusively aimed at sports providers. It's a great cost saver for any broadcaster or content owner that has occasional conversion requirements. In this case, they don't want to have the trouble and cost of buying their own dedicated converter, and having staff available to run and maintain it. It's also available globally so there's no need to transport physical assets to events.

If you're paying to have a converter available 24/7 but only use it at the weekends, you save a lot of money using a conversion service and will see a much higher return on investment. The unique part of M2A Connect’s FrameFormer integration is its status as the only live, cloud-based, pay-per-use conversion service currently available in the world.

For all of this to work, it has to be pure software, CPU-only.  It cannot be reliant on GPUs as M2A need to provision an instance to run on and take down afterwards, hundreds of times a day. Therefore, it has to be consistent.

The standards conversion must not add to the latency of the live stream, it has to be easy to deploy in a standard container, and start and stop quickly and cleanly. The service also has to run consistently and have monitoring hooks to ensure reliability and quality of service.

Customer feedback

M2A's customers address global markets so there's been a lot of curiosity about how they can use event-based frame rate conversion in the cloud, on a pay-per-use basis. 

In a typical sports production environment, international content is delivered to a local studio, where graphics, commentary and captions are added. The live source then continues its journey through playout where regionalisation, branding, accessibility and break signalling is added onto head ends for transcoding, encryption and packaging before final distribution through CDNs. Increasingly, broadcasters and sports rights holders are taking advantage of cloud-based pay-per-use services to support their operational needs and converging the playout and head end functions. With hundreds of events taking place around the world on a daily basis, there's an ever increasing need for frame rate conversion in the a cloud hosted workflow.

As a leading OTT sports provider, handling over 30,000 live events a year, DAZN was interested to learn more.  Being an early adopter of new technologies, their continuous innovation enables them to stay ahead in live and on-demand streaming, with a focus on delivering the best user experience.

DAZN’s channels reach millions of monthly users in multiple languages, so a finished program may be delivered to a single end point, or multiple destinations, where different formats and frame rates may be needed. For example, the UEFA Women's Champions League matches will be produced in 50Hz since it originates in Europe, but viewers in Japan or USA will need a 59Hz version.

With this array of variables in mind, DAZN agreed to evaluate frame rate conversion in the cloud. Since the majority of their workflows are cloud-based, FrameFormer by InSync Technology Ltd was an obvious choice for experimentation with a new workflow.

To exercise FrameFormer's temporal rate conversion, DAZN chose a variety of complex and fast moving content. Conversions between 50Hz and 59Hz were tested, on compressed material.  Motion compensated frame rate conversion, such as that provided by FrameFormer, is needed for the highest possible picture quality when sports content is converted.  Alternatives such as frame repeat/duplication or linear conversion lead to blurring, judder, and loss of resolution.

Conclusions

Pay-per-use standards conversion in the cloud brings important benefits to OTT broadcasters, especially those with event-driven workflows.

The future for metadata

Darby Marriott
Product Manager - Production Playout Solutions at Imagine Communications

The Merriam Webster dictionary defines metadata as “data that provides information about other data”. Some call it “bits about bits”.

In the media industry, it is the information which tells you about the video or audio content. And, as such, metadata has been around a lot longer than the computer. If you still have film in your archive, you will expect to find shot lists on a sheet of paper inside each can. And, at the very least, archivists would have a card index to help find things on the shelves.

These manual processes broke down when we moved into storing content on video servers. When the essence is just ones and zeroes spread across a number of disk drives, then you need a database to track files and find them when you need them.

Asset management went hand in hand with broadcast automation, ensuring the programmes and commercials in the schedule were correctly identified for the playlist, and those files were loaded into the playout servers and cued at the right moment.

Today, we demand more from metadata. Schemas have grown ever bigger, with a huge amount of information on each piece of content. Some of this is descriptive, telling you what you have and who was involved; some of it is technical – everything from codec and aspect ratio to camera and lens serial numbers.

In a modern, connected production and delivery center, a large number of people will care about parts of the metadata, but – system administrator aside – no-one needs to know all of it. So the well-designed system will manage the data and the user interfaces to ensure everyone is perfectly informed without being overloaded with information.

To explain what I mean, think about the marketing department of a broadcaster. A great new series is being delivered, and you want to ensure it attracts the biggest possible audience. One of the ways that you do this is by creating a suite of trailers and promos.

The editor charged with the task of creating these promos does not care about any of the technical information – he or she will reasonably assume that the content is fine if it has made it to the live servers. All the editor has is a work order, saying “take our standard trailer format and make a set for this new program.”

The editor may well be using Adobe Premiere and After Effects. Adobe has its own, excellent metadata format, XMP. A smart system design will integrate the overall asset management and playout system with Adobe so that, when the editor opens the work order, the content – from the video server and from the Adobe templates server – is already loaded into Premiere and is ready to go.

When the trailer is completed, the editor or graphics artist initiates the export process. The relevant XMP metadata is wrapped in the file where it is then exposed to the asset management system. This will also handle the generation of house-standard content IDs, and add them to schedules and playlists as needed.

A really smart system will integrate even more data: when to trigger coordinated live events. So rather than the graphic in the trailer just saying “coming soon”, it will say “launches 1 September at 9:00pm”, dynamically linked to the schedule information along with coordinated live DVE sqeezeback upon playback.

This metadata connectivity is all real and happening now. Promo teams use it; and newsroom editors use exactly the same workflow when a news package needs a craft edit with content from the archive.

Any news broadcaster will tell you that the archive is the most valuable asset they possess. Reporting a story is one thing: putting it into the context of what has happened before, what people said in the past, what the consequences of an action are likely to be are all vital.

It is probably in the newsroom, then, that we will see the next big steps in the development of metadata. This will build on the intelligent automation and communication we have today, and augment it with machine learning and artificial intelligence to enhance metadata, with the aim of creating better content by knowing more about what is in the archive.

The pandemic is a good example. Since February 2020, politicians, medical experts, statisticians and citizens have all said a very great deal on the subject of Covid-19. A journalist reporting on a new development today will create a better story if it reflects back on what the parties involved have said in the past.

AI tools are now already available which can create very detailed metadata, completely automatically. The soundtrack can be automatically transcribed, so you can search for what someone said at a given time.

AI can now be applied to video analysis, creating shot lists and detailed information on the content of the clip or programme, with each incident tied to a timecode. This has obvious benefits in the newsroom, where journalists can quickly find exactly the clip they need. But it has wider benefits, too: it is a powerful tool in discovery, whether that is in programme syndication or direct to consumers.

AI can, of course, be run on premises. But it is an obvious cloud use case. The major cloud providers, like Amazon Web Services (AWS) and Microsoft Azure, have their own powerful AI tools which can be trained to your specific requirements. Video and audio analysis is also inherently “peaky”, with long periods of inactivity interspersed with high processor demand when there is new material to be indexed.

Wherever you are in the broadcast and media production and delivery chain, the  future of metadata is that it will become ever richer. Automated tools and machine learning will contribute greatly, as will the integration of multiple application-specific databases like Adobe XMP.

Critical to the success in benefitting from all this information is an over-arching integration and management layer. The right tools and the right information must be presented to each individual in the organisation, as they need it, with planned transfer of large blocks of content to minimize the risk of bottlenecks.

The future of metadata – in production and in delivery – will be the enabling technology for orchestrating workflows across every aspect of the operation.

How AI Will Simplify Your Broadcast News Workflow

Large broadcast news operations ingest huge volumes of content every day, all of which must be logged and reviewed to determine which assets, illustrated by which video clips, will be presented to viewers. It’s a tedious and time-consuming task, performed by dedicated teams who manually review incoming content and then describe or notate that content, which producers later access to produce finished news segments.

This process is compounded by the wide variety of delivery platforms – linear broadcast, OTT, various Multichannel Video Programming Distributor (MVPD) services, social media platforms and others – and each with its own formatting rules and other requirements. And of course, with news, time is in short supply and every second counts when turning around content and getting news to air. 

Applying AI and ML technologies to news workflows can auto-generate highly accurate metadata of video content more quickly and less expensively than traditional methods, which yield big cost and time benefits for fast-paced news operations.

The key to delivering these benefits is in the ability to harness the power of these next-generation technologies. By leveraging high-performance AI capabilities in the cloud (speech-to-text, facial recognition, object identification, content classification, etc.), combined with powerful knowledge management orchestration tools, these metadata solutions deliver actionable intelligence for media operators and accelerate the search, retrieval, production and delivery of news content.

The solution described above is what one major broadcaster is using to enhance its existing news production workflow and incorporates several AI engines and three specific toolsets as follows.

AI engines include:

  • Automatic speech recognition (ASR)
  • Natural Language Processing (NLP)
  • Text Translation
  • Facial recognition
  • Emotion recognition
  • Logo detection
  • Object identification
  • Optical character recognition (OCR)

Toolsets include:

  • Automated metadata creation for raw feeds and post-production content residing in PAM/MAM that applies advanced AI- and ML-based content analysis for automatic generation of better-structured, more detailed, and more accurate metadata. Media operations benefit in two key ways — first, with tremendous time savings in up-front metadata generation, and then by giving producers the ability to zero in on the assets they need right away.
  • Closed caption/subtitle generation platform, which unites cutting-edge STT technology and other AI-driven processes with cloud-based architecture. This toolset radically reduces the time and cost of delivering accurate, compliant captions for publishing worldwide.
  • Broadcast monitoring and compliance logging that allows operators to record, store, monitor, analyze, and repurpose content quickly and efficiently with a minimum of clicks. Integration with AI microservices enables powerful insights and video intelligence applications, including:

○           The ability to record, store, and retrieve content for compliance, quality of service, and insights into broadcast content.

○           Automatic transcription of content from live broadcasts and commercials.    

○           Automated detection of logos, objects, faces, and shots.

○           Automatic extraction of on-screen text.

○           The ability to identify ad breaks in logged content.

○           The ability to identify restricted words or topics in recorded/logged content, as well as the classification of incoming ad material for restricted content. 

○           Generation of automated reports for loudness compliance, QoE, SCTE inserts, and ad detection and identification.

○           Automatic content classification.

○           The ability to assess the quality and conformance of captions.

The solution uses a variety of AI and ML technologies to process numerous, 24x7 live news feeds simultaneously from a wide array of sources.  Concurrent, real-time processing of this quantity of content, is something no amount of personnel could ever hope to accomplish.

As feeds are received and ingested, they pass through APIs to all the applicable AI engines and on to an orchestration layer, where the solution automatically harmonizes and synthesizes the metadata and doles it out to the software tools mentioned above. Those high-value applications — the transcription, video metadata generation, and monitoring tools — take over, working in unison or individually as directed by the operator to provide intelligence and actionable insights that make the metadata usable. These capabilities enable intelligent and immediate logging and feedback of content quality and compliance, better positioning broadcasters to meet regulatory, compliance, and licensing requirements for closed captioning, decency, and advertising monitoring.

Importantly for news producers, the solution generates text in the form of transcripts of the spoken word.  Transcript metadata essentially becomes a script of the content, which is time indexed back to the media within the edit and production environment.

In this enhanced state, producers and editors can quickly search for and retrieve relevant media assets needed to create their news stories, thereby accelerating the entire production process.  This all happens automatically in real-time, with results being transferred back into the editing environment. Everyone within the editing ecosystem can get their work done faster, easier, and more effectively than with using traditional methods. Editors and content producers are now only limited by how fast they can make decisions.

While this solution is deployed for an ever-growing feed of live content, all of these capabilities could just as easily be applied to existing recorded content.

Using powerful AI capabilities available through an intuitive and functional software layer, easily accessible across the media enterprise, media professionals can expedite critical processes, reduce mundane tasks and accelerate the creative process – all while saving critical time and expense.