In this IABM TV interview, Gerald Wong (Senior Vice President, Caton Technology) discusses the CatonNet Video Platform and how easy it is for broadcasters to utilise the platform if they have cloud hosted services. Gerald also explains how Caton is working to build on device development and applications that will enable broadcasters out in the field to become more agile in delivering content with speed and efficiency.
In this IABM TV interview, Ivan Verbesselt (Chief Product and Marketing Officer, MEDIAGENIX) discusses collaborative workflows and Lean Content Supply Chains.
In this IABM TV interview, Benjamin Lardinoit (CEO & Co-Founder, On-Hertz) discusses virtualization & production workflows.
Esen Bayar
CEO at ETL Systems
Awarded with a PhD following his study at King's College, University of London, Dr Esen Bayar has over 30 years’ experience in satellite communication systems and hardware design and development. Esen joined ETL via a management buy-in in June 2003. He is engaged in developing ETL's operational procedures and technologies, which have played a pivotal role in transforming ETL into a global leader in RF engineering.
Prior to ETL, Esen took on engineering and engineering management roles working for Marconi Space and Defence Systems (now known as Airbus), Com Dev (now part of Honeywell) and Radiant Broadband Wireless Systems. He has a number of technical publications in the fields of RF and microwave design, as well as satellite and broadband wireless communications, and holds many patents.
Hi Esen, could you please tell us about ETL’s history, growth and current operations?
ETL’s heritage is in designing satellite signal routing solutions for broadcasters, who need extremely high levels of RF performance, as well as redundancy and resilience. Over time, our range has expanded to cover applications for government, defence, marine, and private VSAT networks, as well as global satellite operators.
Set up in 1984, ETL has a long and rich RF engineering history, but a key turning point for the business was in 2003, when my business partner and I acquired the company. Our vision was to make ETL a truly international company.
We introduced innovative, high-tech automated manufacturing, and elevated our product to the highest quality. We formed partnerships with our customers, which enabled us to understand the challenges they have and steer our growth along those lines. We also grew our engineering capabilities with significant investment in manufacturing, which has given us a great competitive advantage over the years, enabling us to get to market fast, retain greater control and allow innovation to flourish. We have also invested in our people throughout, which has been integral to our growth.
Fast-forward to now and ETL has 155 employees and, as well as our headquarters in Herefordshire, we have an R&D centre just north of London and sales offices in Dubai and Washington DC.
Could you give us an outline of ETL’s current range of products and services?
We design a broad range of RF equipment to serve the satellite communication sector, which is a multi-billion-dollar industry that continues to see year-on-year growth. Broadcasters rely on it, and us, to deliver high-quality content to billions of people around the world.
The beauty of satellite communication is that it doesn’t matter where you are, we can get a signal to you – while also integrating seamlessly with the terrestrial network.
What broadcasters want and need to transport has of course evolved over the years; our equipment used to send and receive lots of audio and video content, whereas now the vast majority is data.
Regardless of what is being transported, the use of satellites by broadcasters continues to be an integral piece of the jigsaw, enabling the reliable high-quality glitch-free coverage of sports, news and events the consumer has come to expect.
ETL’s main function is to develop and provide the necessary equipment and technology for transporting data from broadcast end-users to the satellite dishes, and then on to the satellites themselves. On the return, we pick up the data and transport it for the end user/operator.
In order to achieve this, the signal frequency needs to be increased and modulated - and with the satellite networks themselves becoming smarter and capable of greater throughput, technology in the ground segment has advanced and moved forward. Over time, our innovations are enabling us to help broadcasters to transport data via satellite networks at a lower cost without compromising on redundancy or performance.
What new products are we seeing coming to the market?
Research and development is central to our success and we often supersede our own products ourselves. We are always aiming to fulfil new requirements. I'm a strong believer in the power of teamwork and having a multidisciplinary skill-set in-house. At ETL, we’ve brought together expertise in RF and electronics, mechanical & thermal design, software, RF over fibre and all aspects of high-tech product design & manufacturing.
A major milestone this year has been the development of a new high-density large matrix called Havoc, which enables routing for up to 256 x 256 input and output feeds – the world’s first single chassis RF router of this capacity. These products are over a metre tall and can carry an amazing amount of data. This is the largest and most advanced device of its kind and enables the operator to cut down on modems and modulators and significantly reduce capital expenditure.
We have also launched a series of products called Genus, which is a completely new concept. We are not an “us too” company and this range demonstrates that. It is a flexible and intelligent habitat, great for combining lots of functional building blocks - power supplies, amplifiers, matrices, frequency converters, RF over fibre – in one unit. It’s not the conventional way of thinking about things. We’ve redesigned and reengineered it from the ground up to enable much more flexibility.
For example, the power supply normally has a five-year lifespan, while an amplifier has 15 years. Before the power supply goes down, Genus would send a notification to the end-user to remind them to replace it. The aim with all of this is to future-proof the equipment, add flexibility, and all while continue to deliver the best performance and redundancy.
Are there any projects you are particularly proud of?
There are many, including the two I have already mentioned. I am tremendously proud of the high-density, high-tech H-series matrices we’ve developed, including Havoc. Also developing Genus, a completely novel technology, from scratch.
Aside from these, we also introduced a new set of products for RF over fibre applications about five years ago. These are now responsible for about a fifth of our total sales, which has been absolutely incredible growth.
More recently, we launched a range of frequency converters, a first for us coming up against a number of established competitors. In a short timeframe, many large global operators have already begun adopting our technology. We are serving both indoor and outdoor applications.
Why do your customers continue to choose you over your competitors?
I think I can distill this down to three basic points.
In no specific order, the first is the quality and performance of our products; the second is our passion and customer empathy; and the third is our partnership with customers from concept to delivery, from beginning to end.
These are the three fundamental points that ensure our customers are satisfied when they choose us.
What’s next for ETL Systems?
Our product development is never ending. Our mission is always to come up with better, more intelligent and highly reliable products.
Perhaps the most promising new technology we are working on, which we expect to realise and begin commercial manufacture of in 2022, is digital IF for the satellite ground segment. This new technology, which we are working on alongside the European Space Agency, will enable us to convert ground infrastructure transmissions from analogue to digital, with many very interesting benefits for end-users.
This is highly exciting, but equally challenging. We have been working on this since 2019, and at times we were left scratching our heads and had to go back to the drawing board. But we thrive on these challenges. We’ve got a winning formula now and I believe in time it will have tremendous applications and implications for broadcasters. Watch this space!
Find out more here: https://www.etlsystems.com/.
In the past five years, media technology providers have developed innovative solutions to solve significant challenges that come with cloud-based video production.
There were advances in processing, editing, networking, storage, media transfer, and now, more recently, AI and automation that have enabled more video production workflows to move into the cloud.
More recently, and amplified by the current times, studios, rental houses, and enterprise production companies are turning to remote production activities, but still, the overall production and distribution workflows continue to present major challenges and inherent inefficiencies due to legacy media technology, workload management, and infrastructural costs.
It’s no news that the industry is on the precipice of significant disruption to those legacy media workflows: they kind of have to. However, to empower creative talent and compete in a market that’s producing content faster each day, studios must increase their ability to keep up with this demand.
Even before the past two years, cloud-based video production technology has enabled remote work for IT efforts. On the postproduction side, reviewing daily footage, color grading, audio editing, and VFX are increasingly becoming the norm - just think of suppliers like Frame.io or Blackbird. However, there are several key issues that are preventing some types of production work from being done remotely, especially when you look into the whole media supply chain.
So, where are we now?
We are all aware of the advantages of cloud production; this topic has been covered a lot over time. However, today’s demands have put strains on the legacy video workflows due to the need to support a higher volume of production and post-production resources - not only on the technology side but also talent. Current times have amplified the demand for direct-to-consumer content - think of the OTT explosion - and also put additional strain on those same resources to support remote access.
Supporting this migration can result in a massive increase in costs. Large studios might have budgets and dedicated IT capable of working with newer technologies, but smaller creators —mid-sized studios or even in-house brand managers — simply can’t access that same toolset.
Understanding Remote Production’s Excessive Costs
The advantages of going remote are many. We can have our equipment safely in one place and we can serve lots of different filming without having to transport everything. Of course, it’s not just the dedicated media machines that get transported somewhere else; we can also have our production team access the production from different locations, without compromising the workflow.
And whilst remote production is great, it’s very expensive. Ask anyone in these last two years that decided to move to remote, and they’ll tell you that the jump in costs is substantial. There are three main factors driving costs up when working with data centers and companies supplying remote solutions for video.
The first is dedicated equipment. In a remote workflow, dedicated hardware is placed in a specific data center. This means that:
a) if you want to work off of another datacentre that might be closer to you, bad luck, you can’t; and
b) supplying and housing dedicated units in data centers is expensive.
The second factor driving up costs is the necessity for dedicated personnel to maintain, repair, and interface with those machines. Whenever the production team wants to use them, they must contact whoever prepared the units to turn them on. So, let’s say you have an issue with the units you are using. Now you need dedicated support to go to that specific data center, to that dedicated unit, to fix the issue.
And even though that’s already a lot, there’s still the third reason why costs are so prohibitive: machine uptime.
On a standard remote production infrastructure, the dedicated machines you are connecting to are not usually optimized for remote. If you want to film for, let’s say, two weeks, you’d be paying for the two whole weeks where the machine is turned on. And you might think, fair enough, but the reality is that, during those two weeks, there are a lot of downtimes. The cameras are not rolling continuously to justify every hour of every day to match the cost. That leaves a lot of time where you’re just paying for nothing.
Summing it up, the three big reasons why Remote Production workflows in use today are so expensive are:
Native Cloud take Hold
To start, when we say native cloud, we mean that this is no longer a dedicated hardware solution, but a dedicated production software that can run on any hardware, on any data center.
Native Cloud is a virtualization technology that allows you to package and separate individual micro-services designed to be deployed on agnostic hardware, that fit multiple environments. Suddenly, we don’t have to pay extra costs for bringing dedicated machines to data centers, because we’re now running all the operations virtually, on the powerful units that the data center is already equipped with.
This means that no dedicated machines are needed. It also means that we don’t need specific personnel on that data center: all the operations are set up via your internet browser, with ease.
And finally, in the case that anything happens to a specific unit, another one next to it can take its place. Poor machine uptimes are no longer an issue, as production instances can be launched only when needed, thus dramatically reducing media processing costs.
In a nutshell, native cloud infrastructures offer many benefits for media organizations. For example, the micro-services provide the flexibility to efficiently manage new remote workflows and infrastructures consistently across all applications, such as ingest, encoding, transcoding, streaming, and video transport.
Looking into the future: Container-based deployments
We already covered that the endgame here is native cloud. When compared to remote production infrastructures in use today, native cloud massively reduces the cost of resources, talent, and technology.
Those factors are what are prompting more and more media businesses to work with native-cloud providers, like MOG, to evolve their hybrid or public cloud resources and workflows.
The shift to native cloud technologies is also driving interest in container-based deployments. Platforms such as Kubernetes offer high-available infrastructures that allow media companies to move their workflows to, or from any public, private or hybrid cloud, without having to completely change their entire infrastructure.
Containers will offer a consistent way to ensure high-level availability while removing the complexity associated with traditional approaches.
There’s no doubt for us that containers will become the main deployment solution for the remote and hybrid world. Native cloud technologies are essential to empower media businesses with the right tools and make remote production financially accessible to all. And we sure want to be a part of that change.
The Background
ATP Media is the global sales, broadcast production & distribution arm of the ATP Tour, the global governing body of men’s professional tennis. Formed in 1999, ATP Media provides the centralised exploitation and host broadcast production for the worldwide television and digital broadcast rights of 64 global tournaments.
ATP Media has unrivalled access and expertise in the broadcast production of men’s professional tennis. This unique service incorporates rights sales, multi-platform production, content management and worldwide delivery into a comprehensive end-to-end broadcast-focused solution.
Tennis TV is ATP Media’s direct-to-consumer, OTT streaming service, delivering both live and on demand content to all major streaming devices. It also features over 10,000 hours of full matches dating back to 1990.
The Customer Challenge
ATP Media already has a B2B commercial archive site in the cloud that hosts the ATP Tour Masters 1000 and Nitto ATP Finals match content for commercial exploitation. There was however, a wider asset management challenge - the aim to migrate and preserve all other match and video content across the Tour and present a globally accessible Production Asset Management service.
ATP Media initially approached Object Matrix with the requirement of preserving this high volume of media. ATP Media created a Preservation Project to enable the migration and long-term storage of existing heritage video content from legacy formats and infrastructure, such as LTO, to a secure location. As a by-product of this investment, ATP media was keen to consult with Object Matrix to also add the capability for the platform to serve as a media Production Asset Management system (PAM). This would enable the Production and Digital teams to manage their edit archive effectively, and to link ATP Media’s content delivery network to thousands of hours of footage.
As a globally moving property, ATP Media had previously been tied to drives that it needed to physically move around between events. This caused an issue for PAM and brought its own set of challenges and risks, being very cumbersome to manage and not always secure as the drives could get lost or damaged in transit between locations. Global production teams needed a scalable, flexible, and highly available digital environment to handle active media. With footage collated from approximately three thousand matches per year, it was becoming increasingly difficult to responsibly manage media footage.
Functionality was key but scope was also important, ATP Media was keen to ensure that the business could evolve over the long-term. The team required a solution that would be agile enough to integrate into relevant workflows and systems. As well as handle all work in progress, including all match content, synced live using global connective networks. Any solution also needed to take into account the evolution of the business, enabling the team to implement feature requests that might become important in the future.
The Solution and Workflow
ATP Media’s initial investment was half a petabyte of storage as a managed service (MaaS) so Object Matrix deployed a private instance of MatrixStore Cloud within a data centre in London, alongside a replication cluster in a secondary data centre. Through this bespoke cluster, ATP Media was allocated internet connectivity, as well as a dedicated fibre connection to ensure both global access over internet and local file system access to the facility using the MatrixStore File System (MXFS).
Scalability and security were key requirements, along with the need to avoid the potential egress costs of a public cloud solution, given the high demand that would be placed on moving content in and out. Having the ability to share content with conditional access between teams and freelancers means that ATP Media is always in control of its content through auditing capabilities and access permissions.
MatrixStore’s Vision interface brought the team additional operational benefits through ease of use, either by freelancers or internal production staff. ATP Media wanted a system that didn’t require a lot of training and support, to locate and share content. Vision allows users to quickly integrate, and global connectivity offers the option to work with staff in any location or on the move.
The Future
Investment in object-based storage provided the key to future flexibility for the team. Offering them the ability to present media for any environment and workflow, including Ai, ML services as well as both cloud and on-prem solutions.
ATP Media have already added another replicated half a Petabyte as the adoption of the system has been exponential given the challenges presented by the COVID pandemic and the need for even greater remote access of content.
About Object Matrix
Object Matrix is the award winning software company that pioneered object storage and the modernisation of media archives. It exists to enable global collaboration, increase operational efficiencies and empower creativity through deployment of MatrixStore, the on-prem and hybrid cloud storage platform. Their focus on the media industry gives them a deep understanding of the challenges organisations face when protecting, processing and sharing video content. Customers include: BBC, Orange, France Televisions, BT, HBO, TV Globo, MSG-N and NBC Universal.
Rob Oldfield
Co-Founder and CEO, Salsa Sound
Live sport is one of the most watched content genres and in an era where on-demand and non-linear viewing is edging out linear, it continues to buck the trend. Fans at home want a live experience that is better than being at the stadium, with multiple camera angles and immersive sound. Clubs and sports leagues also want their content to reach as many fans around the globe as possible – online, on linear and on social media.
What this means for broadcasters is creating flawless audio and video, making multiple language versions available and delivering live content as it happens to social media. The challenge facing the industry as a whole is that budgets and staffing numbers are not changing; everyone is trying to do more for less.
The job of a sound engineer is becoming harder rather than easier as new technologies and formats come to the fore, involving more time-intensive, manual processes. The average production can already involve creating over 16 mixes; anything more puts stress on an already stretched workflow – or requires an increased workforce.
Fans’ expectations are not getting any lower either – quite the opposite, in fact. So, how do broadcasters continue to meet fans’ expectations for live content experiences that continually push the envelope forward?
Getting immersive with AI
Enter AI. The technology enables sound engineers to automatically create bespoke, enhanced and 360 immersive audio experiences, with multiple simultaneous mixes that give fans the ultimate listening experience whatever their device or preferences.
By automating the manual-heavy processes, AI-based solutions free up sound engineers to focus on becoming creatives; they can craft rather than chase a mix. This means a far richer audio experience for the end viewer.
Bringing the type of intelligent automation into the workflow that AI makes possible, gives broadcasters a cost-effective way to meet the growing demand for coverage of even the most niche sports, and still deliver the high-end results that align with their brand.
In a traditional live production environment, audio is a job on its own, requiring a level of experience and expertise that is outside that of other members of the production team. When it comes to niche productions, everything – including cost – has to be pared right down. Here, leveraging automated mixes that can be set up ahead of time, means a small production crew – with one person handling audio, video, and graphics – can take care of the job. It is all about making the job easier and giving broadcasters a way to meet the demand for niche content (e.g., reserve or development team matches), that audiences are showing a growing appetite for.
AI can automatically render to multiple formats and mix multiple language versions and even create different crowd flavours (for home and away matches for example). Intelligent automation means each mix is made compliant to meet the loudness standards and parameters required by social media platforms, linear broadcast, VOD, or OTT.
Working with what you have
Doing more for less – without adding complexity – is top priority for any broadcaster today. What sets Salsa Sound apart is that we have developed our AI technology to integrate with and make use of existing infrastructure. We deliver all the automation capability and the ability to create stunning, immersive sound using a standard microphone set-up.
By taking audio feeds from existing broadcast microphones at a stadium, we can use AI algorithms that automatically detect, mix in and enhance the on-pitch/court/ring sounds. As a result, the sound engineer can create engaging real-time mixes without the need for additional kit requirements.
The beauty of this approach is that you do not have to be a topflight club or Tier 1 broadcaster to give viewers amazing, immersive sound. By making use of what is already in place, we are opening up the power of AI to smaller clubs, niche sports and even applications outside premium live sports broadcasting.
By adding automation to the workflow in an intelligent way that actually adds value, we can open up a world of possibility.
A new world of sound
Making effective use of AI is all about leveraging the data; if we can start to see microphones as data capture devices rather than just sound recorders, it opens up the possibility of mining that vast amount of data for sounds that give more meaning to video.
One key area where this approach comes into its own is in highlights creation. AI technology can be trained, for example, to automatically select clips based on what is happening in the audio feed and entering it as part of the metadata; picking up sections where the referee/umpire has been active or vocal, have changes in crowd noise. Commentary is another area where you can use machine learning to generate speech to text and in turn create powerful metadata describing the content. This can help content, tagging, searching or segmenting and provide a better service for content creators and viewers.
While sports, football in particular, is a natural starting point for AI-based audio mixing, this is by no means where the possibilities end. Audio recognition is relevant to everything from game shows to other live entertainment genres.
We are still just scratching the surface of what AI can do to take audio in live production to a whole new level.
Sandy Macintyre
VP News, Associated Press
Maarten Verwaest
CEO, Limecraft
How the Associated Press applied AI to deliver automatic descriptions of live and non-live content
In 2019, the Associated Press (AP) – the world’s largest international news agency – instigated a project to leverage AI technology to shorten its production process so that its customers could receive content more quickly, while significantly reducing manual input, freeing its own staff for more creative purposes. AP turned to Limecraft to enable the transformation, using Vidrovr for scene description, facial recognition and gesture detection alongside Trint for audio description, which together deliver a single, coherent and frame-accurate description of every individual shot.
At a recent BaM live!™ event, Sandy Macintyre, VP News at Associated Press, and Maarten Verwaest, CEO of Limecraft, gave a wide-ranging and honest description of the unfolding of the project. Far from being a ‘fit and forget’ job, it proved to be a complex but ultimately very worthwhile project. What follows is an edited transcript of the session, providing a brilliant insight for everyone who is considering embarking on tapping up AI technology to enhance their processes: in short, the more you put in, the more you get out.
What drove Associated Press to investigate the role AI could play in its production process?
Sandy: “Our starting point was: can AI help us reduce the amount of time that we're spending on the very manual tasks, which of themselves are important, but are not part of the creative storytelling journalistic process? What we were trying to do achieve was to remove what we might call the ‘grunt work’ from a workflow - the really manual stuff. That took us towards AI because one of the things that all news companies in the broadcast space spend an inordinate amount of time doing is transcription of interviews and shot lists in terms of depicting frame-by-frame what is filmed and what's going into an edit. If AI could help us with this, we thought we could save literally tens of thousands of hours of manual time.
“To put this into context, as the world’s largest, and indeed oldest international news agency, we transmit to hundreds of broadcasters and digital publishers around the world; it all adds up to about 20,000 hours of live content or 100-150,000 edited items every year. That's literally hundreds of thousands of minutes of content that have to be manually transcribed and shot listed so the scale is vast, and therefore the time saving could be huge. But where we were also coming at this from was, if we can solve a problem for ourselves, we can also solve a problem for everybody who is a subscriber or customer of AP, which is around 700 broadcasters and probably twice that number of digital publishers; they get their content more quickly.”
What can you do with the ‘grunt’ time you’ve liberated?
Sandy: “If you go back to the days of film, you had a couple of hours between the time you shot something and the time something aired; when we got to tape, yes there was the editing process, but it was then running away someplace across town to the feed point to feed the video. This is just another big, polar change of how you spend that time. But crucially at this time in the era of fake news, fact checking - getting it right - has never been more important. So if you think about most news organizations that are about both being first with the news and about being right and accurate, winning this time by using AI allows for speed and accuracy to come to the fore – and of course, more creativity in the editing process.”
Quality and accuracy are paramount
Maarten: “The quality of the output of artificial intelligence is a critical success factor for acceptance. If the word error rate is four or five percent, that’s not acceptable – the rate needs to be pushed below 2% and that’s a challenge from an engineering point of view. We excel at pushing technology to a level where it becomes enjoyable for journalists to use. And what we found is that there is a huge gap between what a typical engineer seems to find acceptable and what the journalist will accept. There is nothing more frustrating than having to correct artificial intelligence again and again because it's been recognizing the wrong person, or the wrong word. And that's where technology has evolved a lot the last 24 months; I would say that's where massive training comes in. People looking to engage with artificial intelligence should look for a proper man in the middle interface - a proofreading interface: you have to accept that artificial intelligence will make mistakes from time to time, and that it needs correction, and quality control, and that needs to go as smoothly as possible. With these three conditions: a good engine, proper training and a good user interface, you increase your chances of an acceptable solution.”
What process did you adopt to roll out the AI?
Sandy: “I think we've got to be very honest and say that at the start of this process - at the start of all new technology processes - there is an element of fear: fear that people might lose their jobs, fear of change, fear that ‘I won't get the new ways’. So what we had to do was be very open and honest with people about what we were trying to achieve, which was to liberate quality time back to the folks who had to do this work. But we also had to recognize that because real time video is now such a big and growing part of the news ecosystem, that wasn't going away. And therefore, liberating that time was doubly important, because the volume has gone up. The terms of success on which we are judged is live real time, and therefore we had to make these changes. So we effectively put together a coalition of the willing. And within that, we deliberately also asked people who were potential naysayers and doubters to come to the table because their input was super-important. And we knew that over time, they would probably become some of our biggest and most evangelical supporters, if we could get this right.
“But crucially as well, it had to be people who were actually doing this work, so that they could know the difference between how it was before, and how it will be. And also, so we could get to the tipping point between when it is faster to do it all manually, versus when AI wins the race - and also recognizing the point where, while AI still makes some mistakes, it's actually quicker to catch and correct those mistakes. So what you have ended up with is a shift to where I describe AI as the best Assistant Producer – the best intern you ever had - right now. But in time with the training, we're constantly giving knowledge back to it, creating that learning loop, so that Assistant Producer goes up through the ranks and potentially becomes the Executive Producer of this whole workflow process. That’s very much the journey that we’re on.”
All-in or phased process?
Sandy: “The first thing we quite quickly realized was that a ‘boil the ocean’ approach of throwing all our content into the AI bot and trying to get AI to recognize it just wasn't sustainable; the technology is not that smart yet. We quickly realized that a huge amount of content that appears on television screens across digital publishers every day is what you might call governmental, political, diplomatic content - Joe Biden getting off a plane, getting in a car, getting on a stage, making a speech, having a bilateral meeting etc. All of these things you could also apply to any world leader, foreign minister or celebrity. So we knew that if we could teach it recognize these kinds of actions, we could take possibly 20 to 25% of the news content that flows through the AP system every day, and gain understanding of that – and that indeed is what we've done. We've been able to teach it say the top 300 names that appear most frequently on the screen in the world, and the actions that those people might take within a range of known domains.
“The second is that we absolutely did this offline - we got into the digital sandpit and we played there, so that it wasn't polluting the everyday workflows that continued. It's been a fast, iterative process which hasn't got too hung up early on about how it integrates with legacy systems. We got something that works in a beta phase in the sandpit, and then figured out how we would integrate it into the technical and editorial workflows - and let the people who've done that work in the sandpit be the ones who are leading the conversation about integration.
“This process also allows us to take a very hard line - drive it or park it. Is this going to work, how big a help is it going to be, is it worth persisting with? We thought we would get this done inside a year; perhaps more accurately, I naively thought we would get this done inside a year. And the truth is it’s probably taken us two and it will take us another one to get the amount of learnings in there for us to really drive change. It goes back to that analogy of this being as clever, sophisticated and experienced as your assistant producer. But when you create that learning loop of knowledge and experience in the real world, in 12 months’ time with the amount that will go through that learning loop, that machine will be way more experienced, way more helpful - way more like a senior member of the editorial team.”
What have been the catalysts to propelling AI into the demonstrably useful tool AP has now?
Maarten: “We’ve seen plenty of technologies like speech recognition, facial recognition maturing as standalone singular point solutions over recent years, and there are many AI companies out there. But users like news desks have realized that there was a serious effort required to integrate these point technologies into workable solution. What's changed? From 2018 to 2020, Limecraft has been involved in Memad, a €3.5m R&D effort funded by the European Commission. That’s enabled us to combine all the different aspects - speech recognition, scene detection, gesture detection, face detection, optical character recognition, emotional tonality detection - into a single, coherent and phrase-accurate description as if it were produced by a journalist. And it’s made it searchable, effectively automating the shot-listing process in a quantum leap from technology into solution that can be adopted by real-world users.”
Where will AI take AP in the coming years?
Sandy: “We need to figure out how or if AI can help us with the tonality and the accidental bias of any content. AP prides itself on fair, accurate, impartial news, and therefore we need to ensure that our reporting both reflects fairness and the world we live in. I think there will definitely be an AI component to understanding whether you get the tone right, are you biased in favour of one side or the other or one gender or the other, one ethnic group or the other. I think you can get an awful lot of information back from AI in this regard, and there is a project already underway which we are associated with, under the IBC Accelerator umbrella. It is beginning to take baby steps into doing this – and Hollywood and the movie industry is already running scripts through just this kind of tonality index; look out for this at IBC.”
Future direction of AI
Maarten: “Artificial intelligence is only as intelligent as it has been trained to be and training with accurate, normalized data is a critical success factor, as well as a scalability inhibitor if not done correctly. What we’ve done with AP is set up the feedback loop from a journalist into the ‘brain’, continuously updating the data model going forward. We hope this corpus of data will be exposed to third parties, and in the future, it's my hope that other customers like local news desks will also provide feedback to the overall data model. So we have this joint co-creation effort, and my hope is that AI evolves in that direction.”
You can watch the original video of this session at BaM Live!™ here.
Roberto Musso
NDI® Senior Product Manager, Vizrt Group
NDI (Network Device Interface) from Vizrt Group is the world's most widely adopted, software-defined IP video standard. The free-to-download platform allows everyone to easily share high-quality, low-latency video with anyone — live and in real-time — using billions of available internet-connected devices. At its core, NDI 5 moves video, audio, and data across any global, wireless, mobile or local network; between cameras; mobile devices; computers; and other production equipment.
The NDI platform has evolved for years since its initial launch in 2016, and the 2021 launch of NDI 5 provides users with a more straightforward, efficient, and flexible workflow with many key developments. Briefly stated, NDI 5 gives control to the users who can now specify which network interface will be used, reducing the potential for network resource conflicts with other protocols or applications. With NDI 5, it’s easier and faster than ever to set up private NDI networks to ensure NDI streams or inputs can only be viewed by your selected production team. NDI 5 also supercharges post-production workflows and makes collaboration between sites easy and secure.
NDI 5 is used worldwide to add connected devices to networks at television stations, webcasting facilities, houses of worship, schools and universities, offices, or even out on the street from simple discovery. From school classrooms to network broadcasting, NDI supports the software and hardware in use today from a wide variety of NDI partners.
Why NDI?
Unlike other systems, NDI makes video a standard part of every computer network, from common IT implementations to mobile environments. This gives everyone access to easy-to-use, high-quality content creation capabilities with the equipment they already have. NDI 5 elevates video storytelling for broadcasters and producers of all sizes by turning the whole world into a studio.
NDI 5 breaks the limits of the local area network so that production content contribution can come from virtually anywhere. Remote video production effectively becomes local cloud-based studio production. This is because NDI 5 is about more than bringing in a few guests, like one might do on Zoom or Microsoft Teams, but instead it’s about bringing in all the assets from one area and transporting them directly into a live or recorded production.
Keep in mind that NDI is free to download and use. While some solutions – such as hardware with NDI built-in or specific software and applications – may come at a cost, the ability to access NDI is absolutely free. Users can access NDI and its features immediately using tools they may already own such as a smartphone or tablet.
Challenging the Status Quo
NDI 5 offers an array of all-new features that break down the boundaries of AV-over-IP technology. Billions of devices are now NDI-enabled through ARM support. With a new Reliable UDP transmission protocol and a new network interface management, it's easy to set up a private NDI network. In addition, NDI 5 now offers significant Adobe Creative Cloud plugin improvements, an all-new Final Cut Pro plugin, and a new NDI Tools quick launcher with step-by-step setup guides.
A huge game-changing benefit of NDI is its ability to empower creative teams of any size to achieve big show results as there are no limitations from specialized upgrades, networks, or hardware. NDI enables multiple video sources to work simultaneously on Gigabit Ethernet networks and grows to 10Gbit and beyond for massive scaling to almost unlimited streams and resolutions. NDI can even use Wi-Fi networks as another complementary path to utilize smartphones, tablets, and other compatible products as content sources.
Why Now?
NDI 5 and the latest NDI Tools are particularly relevant today because NDI lives where people work— in their existing workflows and creative environments even if they are remote. The new Adobe Creative Cloud plugin from NDI 5, for example, allows audio output both to the soundcard and the NDI output, meaning users can hear the very same audio as is going out to NDI, providing a full editor workflow for Adobe Video Production. The system's new Final Cut Pro plugin enables NDI output from the timeline playout, with real-time audio and video frame buffer output – making working and collaborating with Final Cut Pro easier than ever.
And of course, NDI is more than just a means of transport. It allows for control of devices like PTZ cameras, capturing video feeds directly from the network to use in editing, and defines a standard for encoding and decoding. While all NDI is compressed, there is also a high-efficiency option that includes an uncomplicated way for devices to find each other on a network.
In addition to near universal acceptance with broadcast studio productions, NDI's capabilities have also spread to the pro installation space. A substantial number of houses of worship, schools, live sporting, and music venues, and enterprises have added NDI feeds to their workflows to improve communication. NDI is widely used in live productions where switching is done in real-time to save on post-production efforts.
These live production systems were once found only in the realm of broadcast but are now more readily affordable by small-to-medium-sized businesses and individual content creators. For example, a few PTZ cameras can be positioned in a remote area connected to the network while the live production system can be in a completely different location— eliminating the need for two large production sites to be created.
What the future of NDI holds
NDI 5 is designed to harness the massive creative potential of software and networks, allowing anyone, anywhere to work with video. Because NDI is free-to-use and extremely flexible and scalable, we will continue to see its uses grow rapidly and become even more diverse.
This flexible, scalable environment is excellent for video interviews, videogame streaming, tabletop roleplaying and board game streaming, on-site musical or stage performances, the possibilities are limitless. People have gotten extremely creative with these tools— everything from cooking shows to exercise classes are being produced with NDI.
While there are many other specific use cases, it would be impossible to list them all. The reality is that NDI enables more video capabilities to exist. For in-studio, remote or live events, NDI provides an elegant solution that merges the many elements of a production to accomplish any creative vision. With unrivaled ease of use, efficiency, and cost-effectiveness, NDI means almost any space – work, schools, churches, homes, venues, or even sports fields – can become a location for live video, and this is a very liberating industry development, both for creativity and for business.
Watch our video on NDI 5 here.
Neal Michie
Verimatrix
A digital rights management (DRM) solution alone simply doesn’t do the trick when it comes to premium playback apps. Cyberattacks are becoming increasingly advanced and although the DRM has delivered secure viewing experiences, the user’s device still presents risks.
Apps are often thought of as an extension of the business thus leaving security for them on the back burner. In fact, more often than not developers and security officers have very different understandings of the word “protection” and security mandates are not always understood.
Verimatrix performed an assessment of 14 popular Android media applications to better understand the state of streaming app security. The full results are published in the ebook, “Media App Vulnerabilities Exposed,” But we wanted to share some of the highlights and insights about the story behind the results.
A: It’s easy to say “naivety,” but I think that would be unfair to the very bright people working in security at media organizations. I think the reality is that mobile app security falls into a gap. Traditional risk/security teams are focused on back-end security, while mobile development teams often believe that their DRM solution is enough to protect the content – and it is. But content isn’t the only asset that needs protecting in a streaming app, although it may be the most obvious.
The trouble is that with no outside factor pushing the media app owners to look at app security, it often gets missed until it is too late.
A: The biggest misconception developers have about streaming app security is that DRM is enough. It’s not. DRM is more secure if it can’t be isolated from the rest of the app.
Any attack will start with reverse engineering (understand the app to be attacked). Reverse engineering is a lot easier if you can quickly identify and isolate interesting parts of the software. An attacker can then focus on the code that is of interest and ignore the rest.
It’s also important to realize that a lot of data and valuable intellectual property exists in these apps beyond the content stream. Streaming apps also house payment information, personal data, code language and company secrets. Protecting all of these assets is critical to safeguarding revenue and maintaining customer trust.
A: Content owners are very keen that their content isn’t pirated—understandably so, since they spend a lot of money creating it. Content producers have well-resourced security and risk teams that analyze their ecosystem in its entirety. They are trained to spot gaps and vulnerabilities, and they tighten their mandates when they see a risk.
So far, mandates typically come from individual studios rather than MovieLabs or other regulatory organizations; and content owners seem to view all platforms as equal risk.
These mandates typically take the form of “Robustness Rules,” which are technical conditions that a licensee (e.g., app developer or service provider) must satisfy. Robustness Rules typically require implementations that make it difficult to crack layers of security within the system. This takes the shape of commercial obfuscation and environmental checks, two security methods that protect code, APIs, data, and other valuable assets within the app.
In a perfect world, it would be possible to reference an exact and unchanging set of requirements for different terms (e.g., release window, content quality level, network type, client device type, usage rules). Unfortunately, this isn’t the case. Ambiguities and subtleties about security technologies abound, and they change over time.
What we do know is that studios’ release windows are shrinking, or in some cases completely removed, due to various market pressures and current events (such as COVID-19 and the shutdown of many theaters), while playback quality and bandwidth are increasing. This has led to a general tightening of security mandates. The earlier the release, the more valuable the content and the more stringent the security requirements.
I’m an optimist, so, yes, I do. We’ve seen in other industries that when security standards are well defined and there is a consistent requirement to follow them, then they get near universal adoption.
This has proven good for these industries. Everyone’s responsibilities are clearly defined, there is a level playing field for all participants, and one poor implementation doesn’t damage the industry’s reputation for everyone.
A: Short answer: no. And if developers are aware of the mandates and the tools available, it is only a superficial awareness. In fact, if you ask many developers whether they protect their apps, their understanding of what constitutes “protection” is much different than that of a security officer, CTO, or CISO.
When a security professional asks whether an app is protected, what they really want to know is whether an app is safe from reverse-engineering. When an app developer says that an application is protected, they often mean that they have employed the free tools that come with Android Studio. However, these tools are described in the Android community as “optimizers” rather than “protectors.” What’s more, these tools do little to prevent a hacker attempting to reverse-engineer app code—they merely present a small hurdle.
A: What surprised me most was that many apps aren’t even employing the free tools (such as Proguard and R8) that come with the development kits. The usage rate for these free security tools was below mobile development norms! While the protection offered is minimal, it is better than nothing; and given that the cost to enable these tools is zero, it seems negligent not to turn them on. The time and effort it takes to configure these tools is negligible – typically this task would take about half a day for most apps – so there is really no excuse to not use them.
A: It is often difficult to quantify cybersecurity since it is a complex issue consisting of many layers, factors, and possible attack vectors. To help businesses assess their security measures, Verimatrix initially developed the app security grading scale as part of an investigation into the state of mobile banking security (you can view the eBook here).
During our research, we found that the best standards for app security were the ones put forth by Visa and Mastercard for mobile payment security. Their standards are high, yet practical and well-defined, which means that the implementor isn’t required to dedicate excessive time and resources to unpacking each regulation. We used their standards as an example of good practice, which roughly equates to a B grade on the Verimatrix /UL scale.
For the complete assessment findings and tactical solutions to ensure security in premium playback apps download the eBook: “Media App Vulnerabilities Exposed.” As media apps become increasingly popular and cyberattacks become more sophisticated, it is imperative to assess app security.