HEVC as a standard vs the alliance of open media
Introduction
This dissertation will show the workings of HEVC and what sets it apart and makes it the choice for the next generation codec. Having said that, there are are a lot of questions that are raised. The technology always needs to be monetized. The use, royalties and payments have led to an alternative. In a world where money rules the world, there is parallel world where there is open source software. Where currency is not of value. The Motion Group of Pictures,which we will address as MPEG from now have held the world hostage on the use usage of HEVC. This led to a delay of its usage.
Best services for writing your paper according to Trustpilot
C c d2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 d 3 3 3 3 3 3 3 3 3 d 3 d 3 d 3 d 3 d 3 d 3 d 3 d 3 d 3 d 3 d 3 d 3 d 3 d 3 d 333 3 e 3 e 3 e 3 e e 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
With the introduction of the alliance of open media this shifted the attention away from MPEG’s HEVC.i picked the alliance of open media because of the members of the organisation. This does not mean it has been HEVC is relegated to a lesser quality. It rather means that stakeholders of the industry can and will not be held to ransom, the can actually come together and produce a codec that will not reduce their quality in output and will not interfere with current software and hardware. Alliance of open media is one of the handful organisations that have come together.
List of competitors of HEVC:
• VP9- googles answer to HEVC
• Daala-mozilla foundation and Xiph.org Foundation collaberation
• Dirac VC-2- developed by the BBC R and D team
• Thor- tech company Cisco systems also released a competing codec
• X.265- free open source by MUltiCoreWare INC
The quality of HEVC at the moment is currently the most impressive and effective compression technique. The open source world will use the HEVC as a template and try to emulate it as close as as possible and making it free to be used under the General Public License or any of the other free usage licensces like MIT license that allows you to use it in any way you want to. This licensing debacle has led to a choice of codecs companies and smaller companies can utilise without incurring the cost of HEVC. s
What I will propose in my conclusion is that monetizing HEVC has led to a revolution of the codecs. Because MPEG was the most innovative and progressive and first choice for the professional film and broadcasting industry, the key word is monetisation, it inadvertently led to alternatives. In the conclusion I will demonstrate the how the alliance for open media will be spearheading the codec distribution in the world. Escpecially with the introduction of UHD. This is the reason HEVC was developed in the first place, the higher resolution wasn’t compatible enough with .H264, resulting in more visible higher accuracy when HEVC was applied. Improving overall quality whilst reducing bandwidth.
w a s d r e d f d s w s c f r t 5 4 e 3 e 4 5 t r 4 3 e w q a s d c x z a k j f r 4 e r 5 4 3 w s d f 5 4 r t g f c x s d 4 e w 2 3 9 o p o I k j l 9 8 u h n b j k m j h u I o l p I k j u h n m j u I o l k m j u I o l p o 9 I 8 u h j k I o 0 o 9 o p 0 o 9 I o 9 0 o 0 9 I f d s w s 3 e r f 4 r e d 3 e d c x s d f g t 5 r e d f v c x s w 2 w s d e 3 e d f r 4 e w 2 we 3 4 e r 4 e w 2 3 e r 4 e d s w 2 we 3 4 r e w 2 3 e r 4 5 t r e 3 w 2 w s x c f g h t g f c d x a z c f d e o I u j h j u I o l k I u j h n j m 8 u h g t r f g b n j h u 8 u I o 9 o 8 u 7 u 8 I 9 o 9 I 8 j 8 l o 9 8 I o 9 9 I 8 u 6 t 5 7 u 8 I 8 9 I 9 8 7 8 u 8 9 I 8 u 9 I 9 o 0 9 I 8 I 9 o 9 o 0 l 9 I 8
b n j h u 8 u I o 9 o 8 u 7 u 8 I 9 o 9 I 8 j 8 l o 9 8 I o 9 9 I 8 u 6 t 5 7 u 8 I 8 9 I 9 8 7 8 u 8 9 I 8 u 9 I 9 o 0 9 I 8 I 9 o 9 o 0 l 9 I 8
I will present the inner workings of HEVC. The open source codecs use this as a blueprint like I said. And ffrom there I will present where the open alliance for open media stands in the development of HEVC. the dissertation will be sectioned of into two parts. And from there my conclusion will leave questions answered. So when being read, it shouldn’t be read conventiently. The conclusion will answer questions that might be raised.
streaming a process in where the end user uses the internet to view content. the means of delivering content varies, a new generation of streaming has been developed: mobile streaming. Mobile streaming basically means that the internet is being used to delivering content to an end user and viewing it in instantaneous . Mobile streaming has become more popular trough web sites like netflix youtube and amazon, the increase in devices that can stream media has led to a new rise in compression techniques that allow for people to enjoy their viewing experience . the most common types of streaming are; Live, on demand and time- shifted
w a s d r e d f d s w s c f r t 5 4 e 3 e 4 5 t r 4 3 e w q a s d c x z a k j f r 4 e r 5 4 3 w s d f 5 4 r t g f c x s d 4 e w 2 3 9 o p o I k j l 9 8 u h n b j k m j h u I o l p I k j u h n m j u I o l k m j u I o l p o 9 I 8 u h j k I o 0 o 9 o p 0 o 9 I o 9 0 o 0 9 I f d s w s 3 e r f 4 r e d 3 e d c x s d f g t 5 r e d f v c x s w 2 w s d e 3 e d f r 4 e w 2 we 3 4 e r 4 e w 2 3 e r 4 e d s w 2 we 3 4 r e w 2 3 e r 4 5 t r e 3 w 2 w s x c f g h t g f c d x a z c f d e o I u j h j u I o l k I u j h n j m 8 u h g t r f g b n j h u 8 u I o 9 o 8 u 7 u 8 I 9 o 9 I 8 j 8 l o 9 8 I o 9 9 I 8 u 6 t 5 7 u 8 I 8 9 I 9 8 7 8 u 8 9 I 8 u 9 I 9 o 0 9 I 8 I 9 o 9 o 0 l 9 I 8
4 4 4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 111v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v
the questions surrounding codecs is an important one. the development of technology and in particular how the internet comes in to play with that has expanded in large increments, the adoption of mobile devices for consumption and the increase of its users has led to a increase in reasearch and development,on how to maximise the viewing experience of content, wethere in real time or stored.?? Background
.?During the late 1990’s and early 2000’s, Internet users realized that the Internet was going to be one of the most powerful tools in history, but it needed to be more developed.
Adobe Flash prompted the development of video streaming format through flash, which is the format used today by sites like youtube. but having moved on from flash to HTML.
??With the introduction of smartphones (2007) and tablets (2010), many people adopted this new devices since they were very useful and more convenient that a laptop computer. Content was very scarce, to maintain this new portable devices in business companies had to converge the vast majority of their content digitally and make it available for this mobile devices. Users wanted a device that was convenient, portable and easy to use. Mobile Applications has become a significant mobile content market since the release of the first iPhone in 2007.?
Once the content was created it became very popular by many of the smartphones and tablets users, they were able to watch movies, listen to music broadcast from stations, watch their favorite television programs and more. These advances in computer networking, made streaming practical and affordable for ordinary consumers. Additionally, this has changed how people use their conventional television and radio. Television was introduced in late 1920’s with few channels available and radio broadcasting was limited as well, with this new innovation of mobile streaming newer generations are going to adapt to this and change the form they watch television and listen to radio, all over the internet.?w a s d r e d f d s w s c f r t 5 4 e 3 e 4 5 t r 4 3 e w q a s d c x z a k j f r 4 e r 5 4 3 w s d f 5 4 r t g f c x s d 4 e w 2 3 9 o p o I k j l 9 8 u h n b j k m j h u I o l p I k j u h n m j u I o l k m j u I o l p o 9 I 8 u h j k I o 0 o 9 o p 0 o 9 I o 9 0 o 0 9 I f d s w s 3 e r f 4 r e d 3 e d c x s d f g t 5 r e d f v c x s w 2 w s d e 3 e d f r 4 e w 2 we 3 4 e r 4 e w 2 3 e r 4 e d s w 2 we 3 4 r e w 2 3 e r 4 5 t r e 3 w 2 w s x c f g h t g f c d x a z c f d e o I u j h j u I o l k I u j h n j m 8 u h g t r f g b n j h u 8 u I o 9 o 8 u 7 u 8 I 9 o 9 I 8 j 8 l o 9 8 I o 9 9 I 8 u 6 t 5 7 u 8 I 8 9 I 9 8 7 8 u 8 9 I 8 u 9 I 9 o 0 9 I 8 I 9 o 9 o 0 l 9 I 8
In order to stream to mobile devices, there are some standards and theories that need to be followed in order to stream properly with no loss of signal or quality. Mobile streaming includes common principles such as IP streaming, video and audio compression and streaming protocols
COMPRESSION TECHNIQUES
HEVC is the latest video compression standard , which evolved from the .H264 standard.
every new standard seems to promise the same thing, identical quality to the previous one but half the bitrate
so how does h.265 achieve less well there’s two main methods used in video compression one Is called inter frame which means that the previous and future frames are compared to the current one and we only encode what’s changed to do this we start on an I frame which is stored as a full image much like a JPEG then we divide it into small 16 by 16 pixel areas which in the previous standard were called macroblocks.
now we’re going to advance to the next frame and compare its macro block with that of the i frame and if there are several blocks which are roughly the same then we give the new frame the status of an P frame, that is a predicted frame this means that what we can easily do is inter code a block so because it’s identical, we bring the pixel values directly across and then we can compress only the movement, in other words give it entirely unique pixel values that are only for this frame thus making the frame size smaller.
w a s d r e d f d s w s c f r t 5 4 e 3 e 4 5 t r 4 3 e w q a s d c x z a k j f r 4 e r 5 4 3 w s d f 5 4 r t g f c x s d 4 e w 2 3 9 o p o I k j l 9 8 u h n b j k m j h u I o l p I k j u h n m j u I o l k m j u I o l p o 9 I 8 u h j k I o 0 o 9 o p 0 o 9 I o 9 0 o 0 9 I f d s w s 3 e r f 4 r e d 3 e d c x s d f g t 5 r e d f v c x s w 2 w s d e 3 e d f r 4 e w 2 we 3 4 e r 4 e w 2 3 e r 4 e d s w 2 we 3 4 r e w 2 3 e r 4 5 t r e 3 w 2 w s x c f g h t g f c d x a z c f d e o I u j h j u I o l k I u j h n j m 8 u h g t r f g b n j h u 8 u I o 9 o 8 u 7 u 8 I 9 o 9 I 8 j 8 l o 9 8 I o 9 9 I 8 u 6 t 5 7 u 8 I 8 9 I 9 8 7 8 u 8 9 I 8 u 9 I 9 o 0 9 I 8 I 9 o 9 o 0 l 9 I 8
the h.265. difference is macro blocks haven now been renamed to coding tree units and they can range from the original sixteen by sixteen and size way up to 64 by 64. And this is what makes the HEVC a such a large advancement. With the pictures below a complete explanation of how efficient the coding tree units are compared to macroblocks.
HEVC divides the picture in coding tree units
the height and widt of the coding block units are allocated in an sequence so the units have the same size: 64×64, 32×32, or 16×16.
e a s d r f g b v c d a s w s w e r f g h u r f c x z e g j x d s w a z s c r w 3333 44 2 e h 12 2 3 2 r 4 t5 5 f s 1 s 2d 3 e 3 3 2 w 2 3 2 1 2 34 2 3 f r 5 r 5 g 5 5 4 3 4 5 u 4 r 4 5 r 4 r 4 r 4 e 3 2 e 3 4 r 4 3 r 3 4 r 4 3 e 3 4 r 4 3 e 3 4 r 3 3 e 3 4 3 e 3 4 3 e 3 4 5 3 e 3 4 r 5 g 5 4 3 d 3 2 w 4 r 4 3 2 e 3 4 f 4 5 g 5 4 3 d 3 4 f 4 3 d 3 2 e 3 4 r 4 5 t 4 3 d 3 d 3 e 3 d 3 e 3 e 4 r 4 5 t e 3 e 3 2 s 3 d f 4 e 3 e 3 2 w 3 r f r e 3 e 3 4 r 4 5 t 4 3 e d s c 3 e 2 w r4 r 4r 5 t 5 f 5t 5 4 e 4 3 e 4 5 4 3 2 w 2 3 4 r 4 3 2 w 3 2 w e 3 4 r 5 4 r 5 4 e 3 2 s 2 s 3 d 4 5t h s v f 4 r 4 r 3 2 3 4 r 4 3 e 3 e 3 2 w 2 w 3 2 w e 4 r 5 4 3 e w 3 e 3 e 4 e 3 4 e 3 ea w
CTU – Coding Tree Unit is an unit. It is made of three blocks, luma Y and two chroma samples (Cb and Cr),. Each block is called Coding tree block or CTB
Each Coding tree block have got the same size as an CTU – 16×16v32×32 or 64×64 Dependend on which part of the frame on. if the coding tree block is too big a decision is going to be need to be made. that decision is is wether it is going to use inter picture or intra picture prediction.
from there each coding tree block unit is going to be split into multiple coding block units, and from there the decision gets made which coding block is going to be applied with either inter or intra picture prediction. so from here a coding tree unit can be split in to 16×16 coding blocks, while at the next frame another is split in to 8×8
The picture below shows how an 64×64 CTB can be split into Coding blocks.
xx
CB is where the decision is made wethere to perform inter-picture or intra-picture prediction. More , the type of prediction is coded in CU.
Coding block is good for prediction type of decisions, but it can be too large to save motion vectors or intra prediction mode. For example, a very small object like snowfall may be moving in the middle of 8×8 CB – we want to use different MVs depending on the portion in CB.
Thus, PB was introduced. Each Coding Block can be split to Prediction Blocks differently depending on the predictability.
when the prediction is made, the difference needs to be coded (difference between predicted image and actual image) with DCT. So a Coding Block can be too big for because the Coding Block can have both a detailed part (high frequency) and a less detailed part (low frequency). That is why each Coding Block can be split into Transform Blocks. a Transform Block doesn’t have to be aligned with an Prediction block .
Profiles, levels, tiers and structures
The HEVC standard has got two tiers, Main and High, and thirteen levels.
A level is a set of limits for a bitstream, e.g., max picture size,. The tiers were made to deal with applications that differ in terms of their maximum bit rate, so that is different screen sizes There are structures ascociated with the compressing technique. Below are the sections and subsections. The terminology is as the the same as explained in the above paragrapgh
Coding Structures
Coding Tree Unit (CTU)
? Coding Tree Block (CTB)
? 1 luma CTB + 2 chroma CTBs + associated syntax
? Coding Unit (CU)
? Coding Block (CB)
? Prediction Unit (PU)
? Prediction Block (PB)
? Transform Unit (TU)
? Transform Block (TB)
12
Coding Structures
? Versus H.264 (1)
? H.264: 16×16 macroblocks
? Intra blocks: 16 4×4 sub-blocks or 1 16×16 block
? Inter blocks: 16×16, 16×8, 8×16, 8×8 (and subpartitions)
? HEVC: Coding tree blocks (CTBs)
? 16×16, 32×32 or 64×64 blocks
? Quadtree-like subpartitioning into coding blocks (CBs)
? Minimum CB size: 8×8 (or larger if specified)
? Chroma partitioned accordingly
Coding Structures
? Versus H.264 (1)
? H.264: 16×16 macroblocks
? Intra blocks: 16 4×4 sub-blocks or 1 16×16 block
? Inter blocks: 16×16, 16×8, 8×16, 8×8 (partitioning)
? HEVC: Coding tree blocks (CTBs)
? 16×16, 32×32 or 64×64 blocks
? parting into coding blocks (CBs)
? Minimum Coding Block size: 8×8
Coding Structures
? Versus H.264 (2)
? H.264: Prediction
? Transform is always 4×4
? Intra/inter decision on (16×16) level
?
? HEVC: Prediction and transform flexible
? CBs are split into 4×4 prediction blocks
? CBs are split into min. 4×4 transform blocks
? TB structure be partitioned further into the PB structure
? Intra/inter decision on 8×8
? Intra prediction
? Intra Luma and Chroma prediction modes
? 35 modes: Planar + DC + 33 angular prediction modes for all block sizes
? Chroma:
? 5 modes: Derived mode (DM) + Planar + DC+ Horizontal + Vertical
Intra prediction
? Intra prediction in HEVC
? Intra prediction
? Versus H.264
? H.264: Number of total modes depends on block size
? 16 4×4 sub-blocks or 1 16×16 block
? Luma:
? 4×4: DC or directional prediction (8 directions)
? 16×16: DC, plane, horizontal or vertical prediction
? HEVC: 35 modes in total
? 32×32 down to 4×4 sub-blocks
? DC, planar or directional prediction (33 directions)
? 3 most probable modes?Chapter 4
Inter prediction
… Inter prediction
? Versus H.264
? H.264
? 16×16, 16×8, 8×16 or 8×8 partitions
? 8×4, 4×8 or 4×4 sub-partitions for 8×8 partitions
? ¼ luma pixel accuracy
? 6-tap filter for half pixel and averaging for quarter pixel
? Use MV prediction to save bits
? HEVC
? More flexible, symmetric and asymmetric PB partitioning
? ¼ luma pixel accuracy
? 8-tap filter for half pixel and 7-tap filter for quarter pixel
? Choose one MV from multiple candidates to save bits
Transform
? Core transforms: DCT based
? 4×4, 8×8, 16×16, and 32×32
? Near-orthogonal
? Nested transforms
Transform
? Alternative 4×4 Discrete Sine Transform (DST)
? 4×4 intra blocks, luma only
? Reason: Residuals tend to increase with distance from boundary
? 1% intra-only bit rate decrease (hardly any for larger sizes)
? Transform skipping mode
? By-pass the transform stage
? Most effective on “screen content”
? 4×4 TBs only
? Scaling and quantization are similar to H.264Transform
Versus H.264
? H.264: 4×4 Integer transform (DCT approximation)
? Used for all block partitions and modes
? Adaptive 8×8 integer transform in High Profile
? HEVC: H.264-like transform for each transform block (TB)
? More transform sizes: 4×4, 8×8, 16×16, 32×32
? TBs must be squared
? DST for 4×4 luma intra
Scanning and Entropy Coding
? Versus H.264
? H.264
? Scanning: zigzag scan
? Entropy coding: CAVLC and CABAC
? HEVC
? Scanning: diagonal, horizontal, vertical
? Entropy coding: CABAC only
? Similar to CABAC in H.264
? Reduced data dependency, easier to parallelize
Codec 101
when h.264 was first standardized back in 2003 1080p video was the height of
technology now of course we’ve got 4kand more efficient encoding is needed ,
and this is exactly what larger macroblocks give you now the second bigimprovement has to do with intra frame compression in other words comparing areas within the same man looking for redundancy and the
improvement is wait for it the increase in prediction directions what well remember these things coding tree units also called macro blocks they can actually be split further into thing called coding units which can go down to8 by 8 pixels and these get this they can b partitioned even further cut up different ways into prediction units why would we bother doing this so we can mathematically generate pixel values in a block instead of storing them and therefore we can massively reduce the size of each frame here’s how it works I’ve got my tiny little pre prediction unit here a little 4×4 and it’s surrounded by two groups of pixels which I’ll call block a and block B now I can actually use various intra prediction modes on this structure I can use for example DC which will basically fill this with the average of the surrounding pixels this would be good on an outdoor scene with a blue sky where there’s no real pattern to it it’s more of a just a single color, alternatively I can use an angular function so say for example: if I have a line coming down here and I want to continue it through the block using my angular function I simply point to the direction where the line is coming from and it will extrapolate or continue that trend now here again there is a difference with h.265 and it’s a good one back in the old days there were only nine prediction modes but now are around 35 different modes
ss1 s 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 4 4 5 4 4 4 4 5 5 4 4 5 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 8 8 8 8 8 8 8 8 8 8 88 7 7 6 7 6 7 6 8 9 99 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 111v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v
now the advantages to this are many including far greater accuracy when larger blocksizes are usedsay for example in 4k video and
generally much better qualitycompression so they have a main reasons
why h.265 is better
bit exhausting Hey but they’re all theoretical and what you might want to know is when can you use
it and the answer to that unfortunatelyis not as often as you’d like I mean you can upload videos encoded with it to
YouTube and most web browsers don’tsupport playing h.265 natively but it’sbeen adopted as a default codec on 4kblu-ray and of course you can play itback on your PC but you might need to use software
How the alliance will change the future of compression in the industry
the alliance for open media is a organisation that includes tech companies like Microsoft, Google ,Mozilla, Cisco, Intel Netflix, and Amazon have formed the Alliance for open media or AOM to create a new codec for future web video
AOM promises an open-source web format that will scale high quality video
on-the-fly to computers mobile devices and over-the-top hardware on any bandwidth it will also include copy protection for commercial use a requirement for streaming businesses
such as Netflix and Amazon if these companies can stop paying h.264 royalties and dodge h.265 royalties entirely video distribution could become less expensive whether that will trickle down to consumers remains to be seen but it seems likely considering our insatiable appetite for streamed content consider how Netflix alone now accounts
for more than 36% of Internet traffic
w a s d r e d f d s w s c f r t 5 4 e 3 e 4 5 t r 4 3 e w q a s d c x z a k j f r 4 e r 5 4 3 w s d f 5 4 r t g f c x s d 4 e w 2 3 9 o p o I k j l 9 8 u h n b j k m j h u I o l p I k j u h n m j u I o l k m j u I o l p o 9 I 8 u h j k I o 0 o 9 o p 0 o 9 I o 9 0 o 0 9 I f d s w s 3 e r f 4 r e d 3 e d c x s d f g t 5 r e d f v c x s w 2 w s d e 3 e d f r 4 e w 2 we 3 4 e r 4 e w 2 3 e r 4 e d s w 2 we 3 4 r e w 2 3 e r 4 5 t r e 3 w 2 w s x c f g h t g f c d x a z c f d e o I u j h j u I o l k I u j h n j m 8 u h g t r f g b n j h u 8 u I o 9 o 8 u 7 u 8 I 9 o 9 I 8 j 8 l o 9 8 I o 9 9 I 8 u 6 t 5 7 u 8 I 8 9 I 9 8 7 8 u 8 9 I 8 u 9 I 9 o 0 9 I 8 I 9 o 9 o 0 l 9 I 8
Codec royalty industry
Companies invent the codec, they spent a large sum into research and development. The sole reason is to invent a new patent that can be used in codecs. This can range from the algoritm used or the processing power it reduces on the cpu. That part would then be codified in the new standard and the company that invented a part of the patent would then get royalties based on the companies that go on to use the codec. The
AVC was introduced with an cap. If a distribution reached a certain number of devices that use AVC. No increase on payment on the royalties would increase. With HEVC the same scheme was introduced but the cap was higher.
cap.
What does this cap do
• Based on the business model that is applied, the companies don’t get direct revenue on each of the devices that are sold using the patent. So the asymmetric business model says that by giving away a lot for free in the return you make money of the few that do pay. So hundred of thousands of smart tv are sold by a tv manufacturer. The tv manufacturer uses the codec by building into their tv set, because it is less than one hundred thousands no fee have to be paid. Once they sell over that amount a fee of $0.20 per unit is to be paid yearly.
• The cap is also a advantage for the patent holders. They have exclusive access to how many products are being sold with their codec, which is normally a in house kept trade secret. Companies do show their annual income and whether they lost in a quarter or had a good 2 quarters. But you never know what products they sell or make a loss on or even are still idly in the warehouses stacked not sold. What the patent holder do know is that like in the example before how many smart tv’s are sold, and that information can be used in their advantage since the patent holders in AVC are around 26 companies that are invested in the codec. So 26 potential competitors have your annual output and production data.
ss1 s 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 4 4 5 4 4 4 4 5 5 4 4 5 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 8 8 8 8 8 8 8 8 8 8 88 7 7 6 7 6 7 6 8 9 99 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 111v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v1v
The HEVC/H.265 royalties mess
.
a few of these members are part of MPEG-LA, the part of Mpeg that deals with the licensing of the codec. The alliance allows unrestricted access to their AV1 patent.
• Codec development – Cisco , Mozilla, daala
• Mobile – Apple, Android, Windows Phone
• Desktop and mobile browsers – Chrome, Mozilla, Microsoft
• OTT -Amazon Fire TV, Android TV, Chromecast,
• Content – Netflix, YouTube, Prime
• Hardware co-processing – AMD (CPUs, graphics), ARM
what does AV1 offer
AV1 is a codec spec, not too disimilar to VP8, H.264, VP9 and HEVC.
AV1 is based on three basic propositions
1. Royalty free – members of the alliance that own patents or techniques that are not protectec or patented are incorporated, so the distribution of the codec cannot be halted by intellectual property claims.
2. Open source reference implementation – what AV1 allows you to do is use it instantaneous. A ready to use, so no technicalities.
3. Simple , AV1 is designed to be simple and straight forward
Simplicity in AV1
When it comes to standardisation, competitiors are looking for a advantage over one or another, what will happen is that and idea will be suggested and persued. Based on that idea a patent will be applied to that. What other companies will try to then is stop that idea into turning into a specification or a part of the idea. Another company will have a close enough idea like the other but they have a different way of achieving that goal.and they have patentned their approach. Now you have two warring factions on who will be a major stakeholder in that idea if it were to be turned in to a specification.w a s d r e d f d s w s c f r t 5 4 e 3 e 4 5 t r 4 3 e w q a s d c x z a k j f r 4 e r 5 4 3 w s d f 5 4 r t g f c x s d 4 e w 2 3 9 o p o I k j l 9 8 u h n b j k m j h u I o l p I k j u h n m j u I o l k m j u I o l p o 9 I 8 u h j k I o 0 o 9 o p 0 o 9 I o 9 0 o 0 9 I f d s w s 3 e r f 4 r e d 3 e d c x s d f g t 5 r e d f v c x s w 2 w s d e 3 e d f r 4 e w 2 we 3 4 e r 4 e w 2 3 e r 4 e d s w 2 we 3 4 r e w 2 3 e r 4 5 t r e 3 w 2 w s x c f g h t g f c d x a z c f d e o I u j h j u I o l k I u j h n j m 8 u h g t r f g b n j h u 8 u I o 9 o 8 u 7 u 8 I 9 o 9 I 8 j 8 l o 9 8 I o 9 9 I 8 u 6 t 5 7 u 8 I 8 9 I 9 8 7 8 u 8 9 I 8 u 9 I 9 o 0 9 I 8 I 9 o 9 o 0 l 9 I 8
different enough approach, with their own patents,. This is how a deadlock is created, which do you choose? factions start to appear at eevery corner and layer of the codec, which wil lead tot both approaches being accepted with different modifications incorporated into the specification.
But do we really need both of these approaches? The more alternatives we have to do something similar, the more complex the end result. The more complex the end result, the harder it is to implement. The harder it is to implement, well… the closer it looks like HEVC.
Here’s the thing.
From what my understanding, HEVC is your standard designed by stakeholders who have a stake in each part of the codec (specification). HEVC was born out of MPEG given us MPEG-2, H.264 and HEVC. Patents and royalties surrounding these codecs were then handled by another group – MPEG-LA. The amount of members in MPEG-LA who are interested in gaining larger market is only growing. This shows HEVC is a mess of a headache to compete with.
This is where AV1 diverges, AOMedia at the moment is different from MPEG or MPEG-LA. Because of 2 main reasons:
1. It is a newer organization, starting fresh. There’s politics there as there are multiple companies and many people, but since it is newer, the amount of politics involved will be lower than an organization that has been around for 20+ years
2. There’s less money involved. No royalties means no money to split between patent holders. So there is more of an understanding rather than misunderstandings and fights about who gets what part of t codecs specification incorporated.
The end result? The design is simpler, which makes for better implementations that are just easier to develop.
Low bitrate
Then there’s the aspect of working at low bitrates.
With the newer codecs, I can see a real desire to enhance. In many cases, this simply means increasing the resolution and higher frame rates a video codec supports. With VR and HDR and AR on the scene it is needed more than ever to have lowe bitrates and not loosing fidelity and compressing as much as conceivable . Being able to do that with lower bitrates is a boom
As far as I understand, there’s a lot of effort being put into AV1 in the other side of the scale – in working at low resolutions and doing that really well. This is important for Google for example, if you look at what they decided to share about VP9 on YouTube .For YouTube, it isn’t only about 4K and UHD,it is about getting videos to be streamed everywhere..
AV1 as a next generation codec changes the landscape where we currently reside in. the current set of codecs like VP9 and .H264,AV1 and HEVC are heaven send. The progress from codec to codec has taken its time , the process of testing, completing and implementing codecs from the start of.H261 all the way to AV1
when looking at AV1 closer, I take a step back from calling it a next generation codec. Since it does exactly what HEVC does. What sets it apart it the the licensing of the HEVC and the royalty free AV1. The business model side will be argued and compared in my conclusion.
Certain characteristics can be seen when having a look at the timeline of codecs.
• Each codec is a improvement from the one previous and improving bandwidth and picture quality, each coming with different versions.
• It can take between 10 to 7 years till a new version is published
• VP9 and VP8 have a four year gap, it got developed later and was in completion with .H264 and VP9 in a race to completion with HEVC.
• Because AV1 was published 6 years later I would not class it as a next generation codec, but rather a improvement on Vp9 and HEVC
Conclusion
After having a close look at the how exactly HEVC improved from the previous H.264,and the AV1 codec getting traction from being backed by numerous stakeholders.
When looking at the members of the Alliance for open media you can see how holistic the approach is, in the sense of interoperability. What MPEG did was make interoperability a standard feature but comes with a financial cost. The Alliance members include large chip manufacturers like Nvidia,ARM,AMD and Intel. This allows for AV1 to be embedded into designs without cost. Even tough AV1 is meant for real time. In the picture below you can see the stakeholders involved crossover into the most used programs for video playback. The advantage of AV1 is that the playback is accessible on recent browsers, HEVC stuggles in that department and that lack there of is making this codec a historical change of distribution away from MPEG
when I looked at the member list is could see this organisation working very tactically. They persuadued all the major stakeholders in the hardware and software industry that need to compress in real-time to join the movement away from MPEG. This means that smart tv’s to handheld devices being designed with the AV1 codec. This move to chip away market share away from MPEG will mean that, holding companies for ransom will lead them away and create their own format in which they will not be charged. A great example of this is that youtube has got plans to use it to replace VP9 as a standard. Specialising in HD and UHD content for web distribution . While on the othere hand HDR and high frame rate fitting perfect for content providers like Netflix and Amazon.
