How streaming ultra-HD video is about to change
Opinion
Innovations in content-aware encoding are transforming video streaming.
Over the last few years, video streaming consumption has accelerated rapidly.
Watching video anytime, anywhere, on any screen has become the preferred way to consume entertainment content in many households. As a result, bandwidth usage and costs have skyrocketed for video service providers. To boost their video profitability, service providers are deploying advanced technologies such as content-aware encoding (CAE).
CAE has seen many incarnations over the years. Designed to reduce bitrates and improve video quality, CAE is an essential tool for video service providers.
This article will briefly examine the history behind CAE, highlight the latest innovations in content-aware encoding and forecast the evolutions we can expect in the near future.
The origins of content-aware encoding
CAE was introduced in 2015 with Netflix announcing a per-title encoding technology that looked at the entire file and determined the best video file parameters to use, such as resolution, frame rate, and encoding settings (pictured, below).
Figure 1. Per-title bitrate ladder, per-title encoding blog. Courtesy of Netflix, December 2015.
Per-title encoding was followed by the launch of per-segment encoding where decisions are made at the segment level as opposed to the entire file. Per-segment encoding provides a gain of about 40% to 50% compared with traditional CBR encoding with a fixed ladder.
CAE for VOD has been deployed by all major OTT service providers, including Netflix, YouTube, Disney+, HBO, Discovery, and Paramount+. During the start of the COVID-19 global health crisis, the EU asked OTT service providers to drop ultra-HD (UHD) transmissions and to save 25% on the encoding ladder.
Overall, viewers did not complain about the impact on video quality.
But some were frustrated to no longer have UHD available despite paying for it.
CAE was initiated by Beamr on the file side, but Harmonic is known to have commercially deployed the first CAE technology for live video with its EyeQ technology. With EyeQ, the processing is done in line at the encoder level, and the encoded output is a VBR packaged in either HLS or DASH.
This technique can also be applied to enable ultra-fast file transcoding, which is critical when video service providers need a quick turnaround of 4K video-on-demand (VOD) content. The performance of such an approach is roughly 40% average bandwidth savings with a limited CPU overhead and no additional delay vs. CBR encoding.
Other companies took a similar approach, with CAE being a feature of their encoding platforms, whether as an appliance or in the cloud. The drawback of this type of CAE technology is that it’s tied to the encoder and can only work with the encoder’s vendor. The technology cannot be used by streaming companies who have developed their own encoding platform.
The next evolution of CAE
The next innovation for CAE is content-adaptive streaming (CAS), which involves end-to-end streaming orchestration, as opposed to CAE which is based on content characteristics. CAS takes into account any feedback from the network (i.e., client, CDN, physical network) during the content preparation stage.
The industry is aiming to reduce the bitrate while keeping the same quality and increasing the bitrate for less watched channels. This is a topic of interest for streaming at scale for millions of concurrent viewers, especially for live sports events.
The video industry has made significant progress since the introduction of per-title encoding, evolving from a per-segment approach to CAE schemes, universal content-adaptive encoding, and finally to a system where the network traffic guides the bitrate savings in real time.
All of those techniques have been instrumental to extending the life of AVC.
Now we are seeing new codecs like HEVC and AV1 replace AVC, with the possibility of CAE and CAS techniques being adopted by video service providers to further boost bandwidth savings and improve video quality for streaming applications.
Thierry Fautier is strategic consultant at connected TV software company VisualOn