Netflix started streaming with AV1 on compatible TVs . This is communicated by the service provider himself through a full-bodied article posted on his Tech Blog. AV1 is the high-efficiency codec developed by the Alliance for Open Media (AOMedia) and royalty-free. The specifications were published in 2018 and by 2020 Netflix already uses it for streaming via the application for Android devices.
The landing on televisions took longer as it was necessary to wait for a greater diffusion of SoCs capable of supporting hardware decoding, the only feasible solution since relying software alone would require computing capabilities not available on TVs.
Thanks to the collaboration of YouTube, an open source AV1 decoder has also been developed for game consoles. According to Netflix, using AV1 will improve the user experience for users. In the next paragraphs we will illustrate the benefits of the new codec and the details regarding the changes introduced, reporting all the information communicated by the streaming giant.
- THE AV1 IMPLEMENTATION
- HOW IT IMPROVES THE USE EXPERIENCE
- SUPPORTED DEVICES AND NEXT DEVELOPMENTS
THE IMPLEMENTATION OF AV1
The first point made explicit on the blog concerns the challenges to be faced to implement the new format. The work carried out is described as not easy as there are many critical issues to be overcome. First of all, we must understand what is the best way to make the most of the possibilities offered by AV1, tailor-made them for streaming movies and TV series, the contents around which Netflix’s business and platform are built. The decisions reached by the managers of Enconding Technologies are as follows:
- Content encoding is always done with the highest resolution and frame rate among those available for the source material . To better understand this passage, an example is cited “ Formula 1: Drive to Survive “: flows have been produced for this series AV1 at Ultra HD resolution and HFR (High Frame Rate), which is the quality necessary to preserve the creative intent of filmmakers on devices that support these technologies.
- All flows in AV1 are coded to 10 – bit , even if the AV1 Main Profile also offers 8-bit color depth. Almost all movies and TV series are now being delivered to Netflix (for encoding) with a color depth of 10 – bit or higher. Recourse to 10 – bit it serves to better preserve the creative intent and to reduce the incidence of artifacts such as “banding”, that effect that manifests itself with bands of color that mark a too sharp transition between shades of color.
- Dynamic optimization is used (we talked about it in this article) to intelligently assign a bitrate to content. This is basically the approach that Netflix has been using for years to balance image quality and bandwidth requirements in relation to the scenes to be played on screen. More complex video sequences are assigned a higher bitrate, i.e. better quality, while simpler scenes, such as those contained in some animated series, can instead maintain the same quality but with a lower bitrate.
Once the ideal balance for content encoding has been reached, move on to next step: ensure smooth playback on TVs . To ensure smooth playback for all users, it is necessary to test the capabilities of the hardware decoders built into the devices. The team responsible for Encoding Technologies has prepared a series of streaming streams specifically designed for the certification of televisions. The aforementioned flows use the same settings that we have listed for the encoding and are therefore perfectly representative of what will then be displayed on the screens.
C ‘ however, it is an additional level consisting of flows designed to subject the AV1 decoders to stress tests . Some of these have a peak bitrate approaching the limit allowed by the new codec specification. On this basis, the Client and UI Engineering teams have created a certification test capable of analyzing both the TV logs and the images displayed on the screen. All eventual problems are pointed out by creating a performance report; if a gap is detected in the ability to decode the signal, Netflix contacts the manufacturer to intervene on the decoder and make it compliant with the required specifications.
Encoding the entire Netflix catalog obviously takes a long time : despite all the optimizations made to the workflow, AV1 requires huge resources that weigh on the CPU and result in a greater amount of time spent than previously used encodings. In perspective we will obviously come to offer every title in AV1 but for the moment we have chosen to start with the most successful contents , such as “La Casa di Carta”. In this way, the advantages of AV1 can be offered to a wider audience of users.
Netflix has finally developed a series of useful tools to monitor streaming in AV1: we talk about a wide range of data, from the so-called “Quality of Experience” (QoE) to the performance of the devices. The Data Science and Engineering team has prepared an alert system capable of detecting the first signs of incorrect operation within the key parameters, in order to automatically send a report for further technical information. These tools are especially essential at this early stage for the introduction of AV1: the ultimate goal is to avoid any negative user experience.
HOW IT IMPROVES THE USE EXPERIENCE
Netflix has comparative among thousands of titles to evaluate the quality of AV1. The results highlighted a significant increase in efficiency compared to the other encodings . However, what matters most are not the analyzes carried out “offline” but the improvements that the end user can perceive. To obtain an evaluation before the launch, A / B tests were organized concerning two groups of users, each of which was equipped with TVs compatible with AV1.
The “treatment” group received the signal in AV1 while the one called “control” viewed the same contents but with the same coding used previously. To evaluate the quality of the streaming, the Video Multimethod Assessment Fusion (
VMAF ), an objective video quality metric developed by Netflix in collaboration with the University of Southern California, the IPI / LS2N lab University of Nantes and the Laboratory of University of Texas image and video engineering at Austin.
The tests reported results in line with expectations: since AV1 is more efficient, it is possible to obtain an improved image quality using the same bitrate and therefore with the same bandwidth consumption. The perceived improvement is particularly significant during sessions with severe network congestion and lower visual quality .
The more efficient compression also reduces the necessary bandwidth: for this reason it is easier to reach the highest resolution for that particular session . For sessions eligible to reach Ultra HD resolution, the duration of the streaming at maximum definition is increased by approximately 5% on average. The same principle also applies to the opposite: given that with the same video quality the bandwidth consumption decreases, with AV1 it is possible to keep the same profile for longer without going to level lower with a lower bitrate . The tests carried out have shown that on some TVs the drops in quality have decreased by 38%.
The latest advantage offered by AV1 is a faster time to start playback . The reason is always linked to the reduced bandwidth: the reader integrated in the applications manages to shorten the necessary time by about 2%.
SUPPORTED DEVICES AND NEXT DEVELOPMENTS
In these early stages of launch streaming in AV1 is available on various AV1 compatible TVs and on any screen connected to a console PlayStation 4 Pro . We do not currently have a complete list with all products; thanks to The Verge however we know that AV1 is already enabled on the following devices:
- A selection of Samsung Ultra HD Smart TVs 2020
- A selection of Samsung QLED Smart TVs 2020
- A selection of Samsung 8K QLED Smart TVs 2020 8K
- Smart TV Samsung The Frame 2020
- Smart TV Samsung The Serif 2020
- Smart TV Samsung The Terrace 2020
- PS4 Pro
- Amazon Fire TV Max
- A selection of Android TV devices running Android OS 10 and subsequent versions and
Netflix is working with various partners to expand the range of compatible products and is currently evaluating the potential offered by the combination of AV1 with HDR.