“VVC is expected to offer significant technical advantages over its predecessors and competitors. However, for the industry to cash in on these technical benefits, a clearer licensing structure for the standard will be important.”
Versatile video coding (VVC), also known as Future Video Coding (FVC) or H.266, is a video compression standard that was released on July 6 and is being positioned as the successor to High Efficiency Video Coding (HEVC or H.265). CPA Global has reviewed the key aspects of the standard and factors which could impact its licensing and adoption and we have identified and analyzed the key patented technologies that have a possibility of being relevant to the VVC standard. The standard was in the process of being finalized at the time we conducted our analysis and we have therefore referred to Draft 8 of VVC, which was approved by JVET in the Brussels meeting from January 7-17, 2020.
VVC has been developed by the Joint Video Experts Team (JVET), which is a partnership of the MPEG working group of ISO/IEC JTC 1 and the VCEG working group of ITU-T. The key aim has been to achieve coding efficiency gains of up to 50% over HEVC at HD/ UHD/ 8K resolutions; while being applicable to a wider range of applications. Just like HEVC, VVC also uses a block-based hybrid video coding structure which combines inter-picture and intra-picture prediction and transform coding with entropy coding. VVC’s major improvements over HEVC are on the fronts of partitioning, prediction and entropy coding. The algorithm is typically not mathematically lossless, as the exact source sample values are typically not preserved through the encoding and decoding processes, although some modes are included that provide lossless coding capability.
Based on studies and tests conducted, VVC seems to have shown a better coding efficiency compared to other competing standards such as AOMedia Video 1 (AV1).
Current Licensing Scenario for VVC
A significant bottleneck which hampered the adoption of HEVC has been the lack of a reliable and consolidated licensing structure for the implementors. Unfortunately, VVC also seems to be destined for a similar lack of clarity on licensing. As such, if the licensing framework is not streamlined, there is a possibility that VVC may struggle to find sufficient adoption despite the significant technical advantage it provides over its predecessor standards and other alternatives.
Anticipating this challenge, the Media Coding Industry Forum (MC-IF) was formed in 2018 to promote the VVC standard and make it well suited for widespread deployment and use. The goal of MC-IF is to work on the nontechnical aspects of deployment of media standards, notably including licensing, by facilitating cross-industry discussion. It is also intended to be a hub for information related to implementations, tools, and guidelines for usage of VVC and related standards. Its members include technology players, telecommunications providers, research institutes, patent pools, device manufacturers and media services providers.
The big question, however, is whether MC-IF would be able to make a measurable impact on streamlining the licensing framework for VVC. Jonatan Samuelsson, a board director of MC-IF and also, CEO of Divideon (a video compression company and one of the MC-IF members) has stated: “I don’t think we intend to structure licensing or create a patent pool; it will be more like an agreement for how a healthy ecosystem would function without mandating how to implement it”. Source
Analysis of VVC-Related Patent Data
To gain preliminary insights on patenting trends relevant to VVC technologies (which will be useful to both patent holders and implementors), CPA Global used the following approach:
- Step 1: Based on review of the draft version of the standard (Draft 8, which was approved by JVET in the Brussels meeting from January 7 – 17, 2020), CPA Global has created a hierarchical technical taxonomy which covers all the key technical aspects of VVC (Figure 1). The taxonomy becomes a very important element of CPA Global’s analysis since owners of IP pertaining to these specific technologies would be expected to be in a driver’s seat when it comes to VVC licensing. Particularly, the following technologies are of greater interest since they would be expected to play a pivotal role in the overall licensing scenario for the standard.
- Niche technologies – technologies which have not been used in previous multimedia coding standards and have been newly introduced in VVC
- Refinement technologies – technologies which have been borrowed from predecessor standards such as HEVC with further refinements/ improvements
These niche/ refinement technologies have been marked in ORANGE in the taxonomy of Figure 1
- Step 2: CPA Global has formulated search strategies to comprehensively capture patent data relevant to VVC. Independent review of each retrieved patent record to check for technical relevance has been omitted from the scope of our analysis.
Also, the Exploration phase of VVC was initiated in October 2015. Further, the first approved version of the predecessor standard (HEVC) was rolled out in April, 2013. In light of this, the dataset retrieved from our search strategies was limited to patent applications with priority date on or after January 1, 2013; so as to ensure that data used for subsequent analysis had a higher relevance to technologies implemented in VVC
- Step 3: Focused samples of the retrieved patent data were checked to identify sections of noise in the retrieved dataset and the search strategies were refined iteratively. The final search strategies retrieved a universe of 7,388 patent applications which were collated into 2,187 patent families. The family representation is useful for our analysis since it gives a fair indication of the number of innovations; assuming one innovation per family
- Step 4: The distribution and trends of IP specifically relevant to each of the niche technologies/ refinement technologies in VVC were deemed to be of particular interest from a licensing perspective. Therefore, using focused combinations of keywords and classifications, CPA Global mined the retrieved universe to identify patent data relevant to each niche technology and refinement technology
- Step 5: Finally, CPA Global performed statistical analysis, which when combined with inputs from its technical experts, resulted in the insights which have been presented in the subsequent section
Figure 1: CPA Global’s Taxonomy for VVC
CPA Global’s approach has also been depicted pictorially in Figure 2.
Figure 2: Description of CPA Global’s Approach for Analysis
It is important to note here that CPA Global’s analysis did not include identifying patents essential to VVC or its draft version. Instead, the analysis was more focused on identifying patents with potential to be relevant to the draft version of the standard.
CPA Global has analyzed the retrieved 2,187 patent families from multiple perspectives. The objective here has been to look at key trends within the patent data relevant to VVC; which should help in providing an overview on the licensing scenario for this standard. The entire analysis is based on patent data published through April 13, 2020. As such, data for 2018 and 2019 may not be comprehensive because of the inherent delay between filing and publication.
Figure 3: Innovation Trends
Based on the priority year trends within the dataset, it can be seen that innovation activity pertaining to VVC peaked in 2018. This coincides with the joint call for proposals for VVC (October 2017) and start of the actual development phase of VVC (April 2018). Source
Figure 4: Innovation Hotspots
Figure 4 shows the key jurisdictions from which the patent data has originated (based on priority country data). For each key jurisdiction, the count of patent families originating from it, the % of granted patents and the average family breadth (number of jurisdictions in which family patent applications have been filed) have been depicted. The analysis is based on priority country for each family and does not factor the Patent Cooperation Treaty (PCT) applications originating from each jurisdiction (PCT has been shown as a separate jurisdiction in the above analysis). Clearly, the United States is the key region from which VVC-related innovations have originated.
Figure 5: Technology Deep-Dive
Figure 5 shows the innovation activity for each of the individual technical concepts identified as niche technologies or refinement technologies in VVC (marked in ORANGE in Figure 1). Overall, maximum focus has been on technologies such as 360-degree video coding, block-based affine transform motion compensation prediction, adaptive loop filter and MMVD.
Figure 6: Technology-Wise Innovation Trends
Looking at the priority year trends for each technology in figure 6, it is apparent that innovation activity for most of the VVC technologies peaked in 2018, which is in line with the overall trend. Specifically, 2018 witnessed a significant acceleration in filing activity on block-based affine transform motion compensation prediction. Filing for 360-degree video coding peaked in 2017.
VVC is expected to offer significant technical advantages over its predecessors and competitors. However, for the industry to cash in on these technical benefits, a clearer licensing structure for the standard will be important. Creation of the MC-IF was one step in this direction. Meanwhile, patent holders (prospective licensors) as well as potential users (adopters/implementors) of the standard will benefit from proactive measures to identify patented technologies potentially essential to the standard.