High Efficiency Video Coding: How the video ecosystem is evolving

We recently completed a high-level analysis of High Efficiency Video Coding (HEVC) technologies. To understand the patenting and licensing scenarios around HEVC technology, we identified all patents related to HEVC and patents declared in various patent pools of HEVC. A total of 6410 patent families were identified and manually reviewed by technical experts to identify the major players in the industry and offer insight into licensing costs, market trends and future challenges and solutions for large corporations exploring HEVC.

__________

HEVC (also known as H.265) is a video compression standard originally developed to provide high quality video coding using half the bandwidth. The HEVC standard describes complex algorithms for motion estimation, frame prediction and the partitioning of coding block units, resulting in doubled compression ratios of coded content compared to the previous standard AVC (H.264).

HEVC delivers an average bit rate reduction of 50% compared to the previously adopted standard of H.264, transmitting the smallest amount of information necessary for a specified level of video quality.  This saves bandwidth cost, but also enables higher quality television delivery over the internet.

The latest television technology (4K) contains four times the number of pixels as 1080p (full HD).  Without HEVC, broadcasters wanting to transmit programs in 4K quality face the challenge of needing high quality broadband reception to make 4K broadcasts a reality.  A benefit of HEVC is that it makes broadcasting 4K more feasible – reducing both the cost and time it takes to deliver high quality programming.

While the technology is anticipated to be used in almost all video processors and display devices in the future, adoption remains slow because of a complex licensing scenario.

A developing technology

Development of HEVC started immediately after the preceding video coding standard (AVC) was finalized in 2003. AVC (Advanced Video Coding) is a block-oriented motion-compensation-based video compression standard that remains one of the most commonly used formats for the recording, compression, and distribution of video content.

The brainchild of several technology giants, HEVC was developed as a joint venture between Samsung, Qualcomm, LG, Mediatek, Microsoft, SK Telecom, Huawei and a number of other companies.

The main profile (‘8-bit’, or Profile 8 of HEVC) was finalized in the first version of the standard in 2013. The below Fig. 1 shows the timeline of development of various video coding standards:

Figure 1: Video Coding Timeline, Source: SlideShare

The techniques specified in the first version of the HEVC standard described the main 8-bit coding profiles and the technology has since been implemented in software and hardware decoders now found in smartphones, computers, televisions and set top boxes (STBs). It is predicted that HEVC will be adopted in satellite, cable, broadcast, gaming and streaming by the end of the decade.

Taking ownership

Of the 993 relevant HEVC patent families – identified through our analytics team’s analysis of 6500 patent families – most are still owned by the companies and institutes involved in the first stages of development. These companies include Qualcomm, Samsung, LG, Mediatek and Huawei. However, there are approximately 75 other companies and institutes that now own patents defining the HEVC standard. Fig. 2 illustrates the share of patent owners in this technology.

Figure 2: Patent Ownership Distribution

Patent families help identify the scale and scope of a patent’s protection – giving insight into a set of patents taken in various countries to protect a single invention. Looking at the most prolific HEVC technology patent owners, the number of patent families are provided in the table below:

Table 1: Top Patent Owners

Interestingly, the ownership of several patents has changed in recent years. Analysis of all 993 relevant patent families found approximately 12% had been reassigned or transacted as part of a patent acquisition deal.

Table 2: Patent transactions

Panasonic has transferred the largest number of patents to other companies. Initially, when there was only one patent pool body (i.e. MPEG LA), Panasonic assigned five of its patent families to Tagivan II LLC (subsidiary of MPEG LA) in 2014. However it later sold 34 of its patent families relevant to HEVC to Sun Patent Trust.

GE Video Compression bought all (seven patent families) of its HEVC related patents from German research organisation Fraunhofer, while almost 75% of Dolby’s HEVC related patent families (nine families out of 12) have been bought from entities such as Sharp, France Telecom and Broadcom.

Patent pools: taking over

Although the development of the HEVC standard was a collaborative effort of several different companies, the licensing scene of HEVC technologies cannot be termed as similalry collaborative or unified. At least three patent pools have been formed by groups of companies claiming to own the necessary technology. The notable patent licensors in these three patent pools are MPEG LA, HEVC Advance and Velos Media.

Table 3: Notable Patent Licensors

The above patent pools cover more than 4,200 patent documents –  grouped in approximately 460 patent families. MPEG LA has the most declarations with 360 patent families, while HEVC Advance enlists 73[4] and Velos Media has 27 patent families assigned to itself.

Of these 460 patent families, 321 were found to be relevant to the 8-Bit (Main) profile of HEVC standard. These 321 patent families constitute almost one-third of all (993) patent families that were identified as relevant to HEVC in the complete study.

Built on a legacy

The development of the HEVC (H.265) standard started after the first version of the AVC (H.264) standard was finalized in 2003. Since then, there has been continued research in developing future standards and further enhancing the H.264 techniques. HEVC is an advancement of AVC and it has adopted several techniques that were used in developing the preceding standard. The pie-chart in Fig. 4 shows the portion of patents relevant to HEVC that are also found applicable to the AVC standards.

Figure 3: Legacy from AVC

Almost a third of the HEVC-relevant patent families were found to have broad coverage and applicability to AVC standards. The major companies that own patents applicable to both the video coding technologies are Fujitsu and, NTT and Panasonic.

Licensing Scenario of HEVC

It is important to note that HEVC license fees are not uniform across different patent pools. Every patent pool has its own patent licensing strategy which varies for different types of patent licensor and device type.

By studying various licensing structures and comparing them with the licensing costs for previous coding technology (AVC), we identified the complex licensing fee structures for AVC (MPEG LA), HEVC – MPEG LA and HEVC Advance.

Table 4: License Fee Structure of AVC (MPEG LA)

Table 5: Licensing Fee Structure of HEVC (MPEG LA) and HEVC Advance

After detailed analysis, we identified the major points of difference between the licensing deals:

• AVC has lower licensing costs for the sale of more than five million units of licensed product
• AVC has a lower maximum annual royalty of $9.75 million. In contrast, the maximum annual royalty for MPEG LA is $25 million and for HEVC Advance is $40 million
• Paid content distributors and Live Rebroadcasting are charged under AVC, while this would not be due under MPEG LA or HEVC Advance for HEVC

Licensing structures for Velos Media and non-pool licensors (such as Technicolor) are still not available and this non-transparency could be further delaying the adoption of this technology.

HEVC is already several times costlier than AVC for any device manufacturer selling above five million units. The fact that a licensee may need to purchase licenses from different pools and from companies that are unaffiliated with these pools (since the patent pools cover only one-third of the patents relevant to the main profile of HEVC) will further increase costs and complexity.

A look to the future

HEVC technologies are being adopted at a slower pace than expected, largely as result of uncertain licensing structures and competition from technologies such as AV1 and AVS2. AVC technology, meanwhile, remains popular and is still used extensively.

The emergence of different patent pools has hindered HEVC patent owners from clearly defining the coverage and scope of licenses. Several of the major technology owners – including LG and Huawei – have not joined a patent pool, while companies such as Samsung and Mediatek have declared just a portion of their relevant patent portfolio in one or more of the patent pools. In fact, only one third of relevant patents have been declared in patent pools.

Despite partial declarations, the licensing costs for HEVC are significantly higher than AVC. A device manufacturer may need to purchase multiple licenses from different patent pools – increasing costs by several times – and still not be ensured complete patent protection with some companies still failing to disclose their licensing structures.

The industry may find it difficult to adopt the HEVC standard with significantly different and complex cost structures surrounding patent pools. With almost two thirds of relevant patents not covered by patent pools and many pools not transparent about licensing structures, this suggests that adoption rates of this technology may continue to be slow until the situation is clearer.

 

Image Source: Deposit Photos.