Al Bovik, Director
Perceptual Picture Quality Algorithms and Databases for Streaming and Social Media
Researchers in UT-Austin’s LIVE (UT-LIVE), Directed by Professor Al Bovik, have singularly pioneered the use of visual neuroscience to create picture and video quality measurement and monitoring tools that control the quality and bandwidths of a large percentage of all streaming videos, television, and social media. Their breakthrough inventions include the iconic Structural Similarity (SSIM), Multi-scale SSIM (MS-SSIM), and Visual Information Fidelity (VIF) “reference” visual quality tools, which delivered dramatic leaps in performance when introduced, and are still dominant today. These tools are used today to control the quality of most streaming and social media pictures and videos in the US and beyond. Bovik and his team also disrupted the field by inventing the first accurate and practical “blind” visual quality models (BRISQUE and NIQE), using models of neuro-statistical distances, at the neural level, between distorted and distortion-free visual signals. These tools are also globally marketed and used in numerous industry applications, including inspection of streaming and social video uploads, control of cameras, and remote video transcoding in the Cloud.
As examples of the impact of their inventions, UT-LIVE’s VIF is the core of the VMAF system controlling the quality of every video streamed by Netflix globally (as much as 35% of downstream US bandwidth), affecting tens of millions of viewers daily and >200 million worldwide. VIF/VMAF has been adopted by most of the video industry (e.g., Apple iTunes / AppleTV, Meta, etc). Likewise, SSIM and MS-SSIM are used in the workflows of the entire media industry, including controlling the compressed quality of dozens of British Telecom HD video streams for real-time streaming/broadcast. These algorithms, along with BRISQUE/NIQE, are marketed worldwide Video Clarity, The Mathworks, Elecard, MSU, FFMPEG, etc.
The experimental science work at UT-LIVE creating perceptual quality databases is just as significant. These huge data resources underlie all advances in the field: for decades UT-LIVE has conducted the most widely used, essential, large-scale psychometric studies of perceptual quality, collecting millions of judgments from tens of thousands of human subjects. These databases are required, de facto global standard tools supporting all quality-related work in streaming and social media, encompassing every aspect of picture and video quality (both cinematic and “user-generated), including 3D, variable and high frame rate, high dynamic range (HDR), audio-video combinations, and virtual reality.
All the algorithms and databases created by UT-LIVE are immediately available free to the world towards elevating the media industry, and always have been.
