VisualOn – Harnessing the power of content-adaptive encoding: a revolution in video streaming
As digital content consumption reaches unprecedented levels, the demand for efficient video streaming and storage solutions is more critical than ever. Content-Adaptive Encoding (CAE) emerges as a game-changer, revolutionizing video delivery by dynamically adjusting encoding parameters based on the unique characteristics of each video. This innovative approach leads to significant improvements in video quality, bandwidth efficiency, and storage optimization, all without disrupting existing workflows.
What is content-adaptive encoding?
CAE builds upon previous innovations such as Netflix’s per-title, per-chunk, and per-shot encoding strategies, pushing video compression even further. Unlike traditional encoding methods that apply uniform settings across an entire video, CAE analyzes factors such as motion, texture, and complexity within the video to optimize encoding settings dynamically. This approach delivers high-quality video while reducing file size and bandwidth needs.
Key benefits of content-adaptive encoding:
- Enhanced quality: CAE dynamically allocates bitrate, ensuring high-motion scenes maintain visual fidelity, while simpler scenes use lower bitrates, resulting in consistently high-quality playback.
- Bandwidth efficiency: By optimizing bitrate for less complex segments, CAE reduces overall bandwidth, offering smoother streaming, especially for users with limited internet speeds.
- Cost savings: Lower bandwidth usage directly translates into cost savings for streaming platforms and CDNs, benefiting both providers and users.
- Storage optimization: CAE minimizes storage needs by producing smaller file sizes, allowing more content to fit within existing storage capacities.
- Scalability: As demand for 4K and 8K content grows, CAE offers a scalable solution, enabling high-resolution streaming without significantly increasing bandwidth or storage requirements.
Applications of content-adaptive encoding
- Video streaming services: Enhances user experience by delivering high-quality videos with minimal buffering on platforms like Netflix and YouTube.
- Social media platforms: Handles diverse video uploads efficiently, ensuring optimal quality on Facebook, Instagram, and TikTok, without overburdening the platform’s infrastructure.
- Online education: Provides high-quality instructional videos accessible to students with varying internet capabilities.
- Corporate communications: Produces and distributes high-quality videos for communication, training, and marketing.
- Broadcast and media companies: Optimizes content delivery for live broadcasts and on-demand content on digital platforms.
Introducing VisualOn Optimizer
VisualOn’s AI-enhanced Universal CAE Optimizer, unveiled at IBC 2023 and awarded Product of the Year at NAB 2024, is a single-pass transcoding technology that analyzes content frame-by-frame, dynamically adjusting encoders to achieve target quality with minimal bitrate. It supports H.264, HEVC, AV1, and integrates with any CPU, GPU, or ASIC-based encoders.
Optimizer revolutionizes media streaming by integrating dynamic coding with scene recognition and image enhancement. It reduces bitrates while maintaining or improving video quality, as shown by VMAF scores, using machine learning to predict optimal encoding parameters for efficient compression.
Optimizer maintains high video quality at minimal bitrates by adjusting encoder settings based on real-time scene analysis and quality evaluations (PSNR, SSIM, VMAF). It also offers optional preprocessing like sharpening and noise reduction. Live streaming tests with FFmpeg confirm its efficiency, stability, and industrial readiness.
As shown in Figure 1, Optimizer integrates seamlessly into any streaming workflow via a simple API call, without disrupting other modules.
Figure 1: Optimizer workflow illustration
Optimizer is integrated within the FFmpeg ecosystem and can be easily used with video encoders via FFmpeg’s APIs. It has several variants for different use cases:
- Optimizer Live: For streaming workflows with real-time transcoding. Its efficient implementation allows it to achieve zero additional latency with reducing both average and peak bitrates without compromising visual quality, ideal for large events.
- Optimizer VOD: For VOD workflows, using FFmpeg’s filter-complex to transcode the entire ABR ladder in a single command.
- Optimizer Fidelity: For visually lossless file-to-file video transcoding to reduce the storage requirements of massive mezzanine video files.
- Optimizer: For general purpose file-to-file transcoding to reduce size of video files.
Key benefits of Optimizer include:
- Universal compatibility: Not bound by any encoder implementation, making it suitable for various use cases and workflows.
- Efficiency: Significantly reduces average video bitrate while maintaining or improving video quality, as demonstrated in Figure 2 below, leading to reduced bandwidth and storage costs, better visual quality, improved KPIs (startup time, buffering ratio), and lower energy consumption.
- Visual quality: Drastically improves visual quality without increasing video bitrate, as illustrated in Figure 3 below.
- Easy integration: Can be integrated into any streaming workflow without disrupting existing operations or requiring additional hardware.
Figure 2: bitrate comparison
Figure 3.1: The quality improvement – left x264, right x264 with Optimizer
Figure 3.2: VMAF score comparison per frame
The following sessions show some benchmark results with Optimizer in action.
Production results:
VisualOn Optimizer is a production proven solution that has been successfully deployed by multiple customers with dramatically improved results, not just in terms of bandwidth and storage reductions, but also in improved user experience KPIs, such as startup time and buffering ratio. Table 1 below shows the comparison of Integral operation results before and after adopting Optimizer [1].
| Before Optimizer | After Optimizer | Improvement | |
| Average bitrate | 3.0mbps | 1.36mbps | 54.67% |
| Startup time | 1.84s | 1.51s | 17.93% |
| Buffering ratio | 0.195% | 0.185% | 5.13% |
Table 1: x264 vs. Optimizer with x264
Another customer, EiTV in Brazil, was able to integrate Optimizer with their production workflow on their own within one week, and achieved over 40% average bitrate reduction [2].
Optimizing open-source encoders
In our study, we evaluated VisualOn’s AI-Enhanced Optimizer across various codecs and hardware/software environments, including X264, HEVC, AV1, Nvidia NVENC, Intel QSV, Qualcomm ARM encoders, and ASIC hardware encoders. Benchmarks showed significant bitrate reductions and VMAF score improvements with the Optimizer, all while maintaining or reducing encoding times. For detailed results, please see the full tables in the original study which will be published soon.
Evaluate your encoding processes and explore CAE’s potential with VisualOn. Integrate CAE into your workflow for smarter, more efficient video encoding. Visit www.visualon.com to download your Optimizer evaluation copy and experience the transformation.
