[FAQ] Video Levels and Color Spaces Demystified -- Get the Best Picture Quality
#1
Motivation
Setting proper video levels in your video chain from Kodi --> GPU --> TV is paramount to getting the best possible picture quality but the hows and whys are often a mystery. This tutorial aims to demystify what actually happens when you change video level settings. After reading this tutorial you should not only have a good understanding of what video level settings are best for your application but more importantly the why behind it.

A Crash Course On color spaces
Video content made for consumption on a television are encoded in the Y'CbCr color space. As it turns out, our eyes are much more sensitive to changes in the brightness of an image (a fully de-saturated Black & White image) than they are to changes to the color of an image. This means that color information (the Cb & Cr components) can be compressed at a much higher rate compared to the brightness information (the Y' component) with little to no loss in perceived picture quality with a significant reduction in storage space and bandwidth.

Although Y'CbCr is great for encoding color information for storage and transmission, it cannot be used to display that information; for that we need to convert to another color space called RGB. If you were to magnify the pixels of your TV you would see that each pixel is actually made up of three smaller sub-pixels: Red, Green and Blue. When each of these three sub-pixels are driven to various intensities, our brain combines them to visualize a single unique color -- this is called an additive color system. The actual math to convert from the Y'CbCr to RGB color space is beyond the scope of this tutorial as it involves a matrix transformation who's coefficients are defined by standards such as BT.709 for HDTV.

Limited vs Full Video Levels
Now that we have a preliminary understanding of color spaces, lets talk about video levels. Video levels are 8bit values and thus range from values 0 - 255. TV video content is completely contained within the range 16 - 235 and we call this limited range. Anything from level 16 and below is treated as pure black (often called blacker-than-black) and anything from 235 - 255 is treated as pure white (often called whiter-than-white.) In contrast, PC's generate video content using the Full 0-255 range where pure black is level 0 and pure white is level 255. When attempting to play limited TV video content on a PC that uses Full video levels (such is an HTPC), we can get undesirable results if we set our video levels incorrectly -- such as complete loss of shadow detail (aka crushed blacks) or conversely really washed out colors.

The 8 images below show all the possible combinations of Full or Limited video levels in the chain from Kodi --> GPU --> TV. After the images are tables that summarize the results and offer advice on the combination that should be used for your application. Note that the happy face was added so you can better visualize the scaling that may occur.

Image

Image

Image

Image

Image

Image

Image

Image

Notable Occurences
  • Setting Kodi to Full causes a color space transformation from Y'CbCr to RGB and the video levels are expanded from 16-235 to 0-255 -- WtW & BtB are clipped in the process
  • Setting Kodi to Limited causes only a color space transformation from Y'CbCr to RGB, video levels are kept as-is
  • Setting GPU to Full leaves the incoming video untouched, i.e. pass-through
  • Setting GPU to Limited causes the video to compress from 0-255 to 16-235
  • Setting your TV to Full or Limited defines where the TV expects Reference Black and Reference White to be: 0/255 for the former and 16/235 for the latter.
Summary of Results
  • Full, Full, Full -- Video Level CORRECT : WtW & BtW Clipped, Single scaling
  • Full, Full, Limited -- Video Level WRONG : Crushed Blacks
  • Full, Limited, Full -- Video Level WRONG : Washed Out Colors
  • Full, Limited, Limited -- Video Level CORRECT : WtW & BtW Clipped, Double scaling
  • Limited, Full, Full -- Video Level WRONG : Washed Out Colors
  • Limited, Full, Limited -- Video Level CORRECT : WtW & BtB preserved, Zero scaling
  • Limited, Limited, Full -- Video Level WRONG : Extreme Washed Out Colors
  • Limited, Limited, Limited -- Video Level WRONG : Washed Out Colors

Which Combination to Use
Out of the eight combinations, only three have correct video levels by the time they reach the TV:
  1. Full,Full,Full -- Both Desktop & Video Content will look correct but video content is also scaled during the Y'CbCr --> RGB color space transformation. BtB & WtW are clipped so calibration will be more difficult
  2. Full, Limited, Limited -- This should be avoided since there are two levels of scaling which will likely cause unnecessary banding.
  3. Limited, Full, Limited -- Best option for embedded video player as there is zero video content scaling during the Y'CbCr --> RGB color space transformation. BtB & WtW are preserved to make calibration easier. If using a Desktop it will look overly dark

Pulling it All Together with Calibration
Go get the free AVSHD709 calibration media from HERE.
  1. Make sure to grab the MP4 version
  2. If all you want to do is basic calibration: Video Levels (brightness/contrast), overscan, picture & tint then just keep the 'Basic Calibration' folder as well as the 'Misc Patterns' folders, especially the ones that have the grayscale ramps (smooth & step.)
  3. The one important thing this awesome calibration disk is missing is Audio sync test patterns. Fortunately you can find two great free ones [url=http://editorsean.com/articles/audio-video-sync-alignment-latency-test/ "editorsean.com/articles/audio-video-sync-alignment-latency-test/]HERE[/url]. Make sure to get the MP4 versions.
  4. If you used the Limited -> Full -> Limited combination (and disabled hardware decoding) you can verify everything is working by playing the *Black Clipping Pattern* and verifying that you can get levels below 16 to flash when you raise your TV's brightness setting.

Changing Video Level Setings
  • Kodi can be set to use limited video levels through the settings menu.
    **NOTE: There is a limitation of the VAAPI Hardware Decoder where it converts to Full video levels despite setting Kodi to use limited. To get limited video content to output from Kodi, you must disable hardware decoding in Video Settings for the time being. This will be fixed in Kodi 17

  • Your GPU can (and should) be set to output FULL video levels by running the command: `xrandr --output HDMI1 --set "Broadcast RGB" "Full"`.

  • What to do with your TV depends on the TV you have. When you set your GPU to output 'Full' levels, it actually notifies your TV of this fact via 'Infoframes.' If your TV honors infoframe information (and many don't) your TV will likely default to 'Full' video levels and you'll need to change it manually to 'Limited.' Note that your TV might call Limited and Full something else; my Samsung HU8550 calls Full as 'Normal' and Limited as 'Low.' Consult your manual to be sure.

P.S.
I've requested an account on the wiki so this can be added there
Reply
#2
Nice summary! But you should mention/consider this work:

http://forum.kodi.tv/showthread.php?tid=231955

Fixes VAAPI issue + enables pass through of video levels without xrandr hack.
Reply
#3
Thanks for posting,

I believe that the RPi only does one conversion rather than two. I had asked the RPi developers about it and they said that the only conversion taking placing is from Y'CbCr to the RGB range which the RPi is set to.

Since I have a TV that only accepts 16-235 the RPi is the best option for me as it goes from Y'CbCr-->Limited and not Full-->Limited-->Limited or Limited-->Full-->Limited
Reply
#4
Which version of Kodi are your results from, and on which platform? Because results are not consistent V15 to V16 and Windows vs. OE necessarily.
Addons I wrote &/or maintain:
OzWeather (Australian BOM weather) | Check Previous Episode | Playback Resumer | Unpause Jumpback | XSqueezeDisplay | (Legacy - XSqueeze & XZen)
Sorry, no help w/out a *full debug log*.
Reply
#5
(2015-12-14, 17:39)Hufvudet Wrote: Nice summary! But you should mention/consider this work:

http://forum.kodi.tv/showthread.php?tid=231955

Fixes VAAPI issue + enables pass through of video levels without xrandr hack.
Oops, meant to link that to the part that says "This will be fixed in Kodi 17", I updated my post. Thanks

(2015-12-14, 18:56)Jdiesel Wrote: Thanks for posting,

I believe that the RPi only does one conversion rather than two. I had asked the RPi developers about it and they said that the only conversion taking placing is from Y'CbCr to the RGB range which the RPi is set to.

Well the RPi has a GPU and so it has the potential to scale. I think what the devs were saying is that that their GPU is essentially set to pass-through such that it does not alter the incoming video levels before they go out; this is one and the same as setting the GPU to full.
Reply
#6
(2015-12-15, 08:50)GrammatonKlerik Wrote: Well the RPi has a GPU and so it has the potential to scale. I think what the devs were saying is that that their GPU is essentially set to pass-through such that it does not alter the incoming video levels before they go out; this is one and the same as setting the GPU to full.

The RPi outputs RGB in either 16-235 or 0-255 based on a config setting (don't think it does it automatically?) I don't think it usually (ever?) outputs YCrCb - so it isn't a pass through (and because most of us output 709 YCrCb - as we run 1080p or 720p output - when you played SD YCrCb you'd not be able to use a pass-through as you'd have to do a 601 to 709 YCrCb conversion)

AIUI on the Pi/Pi2 any YCrCb video (601 for SD, 709 for HD - always 16-235 in either colour space) will be converted to either 16-235 RGB or 0-255 RGB via a single matrix conversion (which will also handle the 601 or 709 colour space differences) based on the output level space requirements. So both Limited and Full range outputs will have a single conversion on the Pi and Pi 2. (According to the Pi Kodi developers) The Limited output setting will keep black at 16 and white at 235, the Full output setting will scale black (i.e. 0% RGB) from 16 to 0 and white from 235 to 255 (i.e. 100% RGB)
Reply
#7
Does the pi(2) output btb and wtw too in rgb limited for video only? Whereas the GUI itself would be compressed from full level to videolevels. That would be great. In fact, that's the only objection I have towards the EGL VAAPI build; if I use the Chromium browser addon for OE, the levels will be wrong.
Reply
#8
(2015-12-15, 11:08)noggin Wrote:
(2015-12-15, 08:50)GrammatonKlerik Wrote: Well the RPi has a GPU and so it has the potential to scale. I think what the devs were saying is that that their GPU is essentially set to pass-through such that it does not alter the incoming video levels before they go out; this is one and the same as setting the GPU to full.

The RPi outputs RGB in either 16-235 or 0-255 based on a config setting (don't think it does it automatically?) I don't think it usually (ever?) outputs YCrCb - so it isn't a pass through (and because most of us output 709 YCrCb - as we run 1080p or 720p output - when you played SD YCrCb you'd not be able to use a pass-through as you'd have to do a 601 to 709 YCrCb conversion)

AIUI on the Pi/Pi2 any YCrCb video (601 for SD, 709 for HD - always 16-235 in either colour space) will be converted to either 16-235 RGB or 0-255 RGB via a single matrix conversion (which will also handle the 601 or 709 colour space differences) based on the output level space requirements. So both Limited and Full range outputs will have a single conversion on the Pi and Pi 2. (According to the Pi Kodi developers) The Limited output setting will keep black at 16 and white at 235, the Full output setting will scale black (i.e. 0% RGB) from 16 to 0 and white from 235 to 255 (i.e. 100% RGB)

1. I hook up a recent 1080p projector to a Pi2 and intend to play a YCrCb REC.709 1080p plunge pattern file (AVSHD).
2. I verify that my Pi2 and projector "negotiate" YCrCb limited
3. I set kodi (OE) to limited

Q1, Would i see the WtW and BtB plunge bars?
Q2, Would the above settings result in the best possible PQ (fewest conversion, correct video levels etc) for playback of YCrCb REC.709 files on a Pi2?
Reply
#9
(2015-12-15, 18:32)Lamm Wrote:
(2015-12-15, 11:08)noggin Wrote:
(2015-12-15, 08:50)GrammatonKlerik Wrote: Well the RPi has a GPU and so it has the potential to scale. I think what the devs were saying is that that their GPU is essentially set to pass-through such that it does not alter the incoming video levels before they go out; this is one and the same as setting the GPU to full.

The RPi outputs RGB in either 16-235 or 0-255 based on a config setting (don't think it does it automatically?) I don't think it usually (ever?) outputs YCrCb - so it isn't a pass through (and because most of us output 709 YCrCb - as we run 1080p or 720p output - when you played SD YCrCb you'd not be able to use a pass-through as you'd have to do a 601 to 709 YCrCb conversion)

AIUI on the Pi/Pi2 any YCrCb video (601 for SD, 709 for HD - always 16-235 in either colour space) will be converted to either 16-235 RGB or 0-255 RGB via a single matrix conversion (which will also handle the 601 or 709 colour space differences) based on the output level space requirements. So both Limited and Full range outputs will have a single conversion on the Pi and Pi 2. (According to the Pi Kodi developers) The Limited output setting will keep black at 16 and white at 235, the Full output setting will scale black (i.e. 0% RGB) from 16 to 0 and white from 235 to 255 (i.e. 100% RGB)

1. I hook up a recent 1080p projector to a Pi2 and intend to play a YCrCb REC.709 1080p plunge pattern file (AVSHD).
2. I verify that my Pi2 and projector "negotiate" YCrCb limited
3. I set kodi (OE) to limited

Q1, Would i see the WtW and BtB plunge bars?
Q2, Would the above settings result in the best possible PQ (fewest conversion, correct video levels etc) for playback of YCrCb REC.709 files on a Pi2?

Not sure what happens if you have YCbCr output over HDMI configured. I wasn't sure that it was an option - but I think it is. However I don't know if that means an YCbCr->RGB->YCbCr conversion rather than just a YCbCr->RGB conversion takes place. (Not sure that YCbCr straight-through - or a 601 YCbCr-> 709 YCbCr conversion - happens in this situation?)

Q1. I'd expect yes - certainly if RGB 16-235 output enabled (though haven't checked so may be wrong)
Q2. I'd pragmatically expect RGB output would have one fewer conversion, as I think (though may be wrong) that there will be an RGB conversion come what may. And there is always the 601->709 conversion issue when you playback SD material in an HD output mode with upscaling (as the YCbCr colour spaces are different for SD and HD - so you'd need to do a conversion in that case whether you output RGB or YCbCr)
Reply
#10
(2015-12-15, 20:20)noggin Wrote: Not sure what happens if you have YCbCr output over HDMI configured. I wasn't sure that it was an option - but I think it is. However I don't know if that means an YCbCr->RGB->YCbCr conversion rather than just a YCbCr->RGB conversion takes place. (Not sure that YCbCr straight-through - or a 601 YCbCr-> 709 YCbCr conversion - happens in this situation?)

Q1. I'd expect yes - certainly if RGB 16-235 output enabled (though haven't checked so may be wrong)
Q2. I'd pragmatically expect RGB output would have one fewer conversion, as I think (though may be wrong) that there will be an RGB conversion come what may. And there is always the 601->709 conversion issue when you playback SD material in an HD output mode with upscaling (as the YCbCr colour spaces are different for SD and HD - so you'd need to do a conversion in that case whether you output RGB or YCbCr)
You can set RPi2 output to YCbCr or even YCbCr Full in the configuration. That said, even on RGB Limited output, RPi2 has some issues which sort of took me by surprise. I was going to use RPi2 as a reference, but had to abandon that idea.Sad Some time back I had posted a link here to a test pattern from Spears & Munsil HD benchmark disc. That pattern can be used to identify issues with chroma upsampling/pixel alignment/clipping/conversion and range.
Reply
#11
(2015-12-15, 18:32)Lamm Wrote: 2. I verify that my Pi2 and projector "negotiate" YCrCb limited

The Pi will never negotiate YCbCr output. We will always default to RGB limited (CEA/TV modes) or RGB full (DMT/monitor modes).
You can force YCbCr mode with hdmi_pixel_encoding in config.txt but it's not really recommended.
The composition of video + GUI overlay is utput as RGB data, so forcing YCbCr will involve an extra conversion after this so is unlikely to improve quality.

In theory all the hdmi_pixel_encoding modes should look the same (to one lsb) as the pixel encoding is output in the AV infoframe and the display should respect that.
However we've found not all displays pay attention to the pixel encoding, so you may find only limited or only full works correctly.
Most likely displays will default to RGB limited for CEA/TV modes and RGB full for DMT/monitor modes, so leaving this as the default is most likely to be correct.
Reply
#12
(2015-12-15, 11:08)noggin Wrote: AIUI on the Pi/Pi2 any YCrCb video (601 for SD, 709 for HD - always 16-235 in either colour space) will be converted to either 16-235 RGB or 0-255 RGB via a single matrix conversion (which will also handle the 601 or 709 colour space differences) based on the output level space requirements. So both Limited and Full range outputs will have a single conversion on the Pi and Pi 2. (According to the Pi Kodi developers) The Limited output setting will keep black at 16 and white at 235, the Full output setting will scale black (i.e. 0% RGB) from 16 to 0 and white from 235 to 255 (i.e. 100% RGB)

This is how I understand it as well in reference to what the GPU does. But there is the driver layer before that, namely VAAPI, which can also do said conversion so by the time the GPU gets the video stream it's already in the RGB colorspace (full or limited.) I'll be quite honest, I have a limited amount of knowledge of all the driver layers that can, and often do, touch the video stream starting from reading an mkv file with video encoded in AVC/H.264 all the way to rendering the video on a target device such as an UHD 4K TV. I wrote this FAQ during my research on color space and video levels so others didn't have to go down the same rabbit hole I did. I wish there was a flowchart/diagram that shows how this all works at a systems level including details such as where VAAPI, VDPAU, GLX, EGL, Vulkan, GLSL Shaders, etc. all fit into this puzzle.

For those here that have this level of understanding, and I'm sure there are a few of you, I would REALLY appreciate if you could help lay this out in a way that is easily consumable much like how I tried to do with this FAQ.
Reply
#13
(2015-12-15, 23:19)GrammatonKlerik Wrote:
(2015-12-15, 11:08)noggin Wrote: AIUI on the Pi/Pi2 any YCrCb video (601 for SD, 709 for HD - always 16-235 in either colour space) will be converted to either 16-235 RGB or 0-255 RGB via a single matrix conversion (which will also handle the 601 or 709 colour space differences) based on the output level space requirements. So both Limited and Full range outputs will have a single conversion on the Pi and Pi 2. (According to the Pi Kodi developers) The Limited output setting will keep black at 16 and white at 235, the Full output setting will scale black (i.e. 0% RGB) from 16 to 0 and white from 235 to 255 (i.e. 100% RGB)

This is how I understand it as well in reference to what the GPU does. But there is the driver layer before that, namely VAAPI, which can also do said conversion so by the time the GPU gets the video stream it's already in the RGB colorspace (full or limited.) I'll be quite honest, I have a limited amount of knowledge of all the driver layers that can, and often do, touch the video stream starting from reading an mkv file with video encoded in AVC/H.264 all the way to rendering the video on a target device such as an UHD 4K TV. I wrote this FAQ during my research on color space and video levels so others didn't have to go down the same rabbit hole I did. I wish there was a flowchart/diagram that shows how this all works at a systems level including details such as where VAAPI, VDPAU, GLX, EGL, Vulkan, GLSL Shaders, etc. all fit into this puzzle.

For those here that have this level of understanding, and I'm sure there are a few of you, I would REALLY appreciate if you could help lay this out in a way that is easily consumable much like how I tried to do with this FAQ.

VAAPI is only relevant to Intel platforms and is Intel GPU specific AIUI. nVidia uses VDPAU, and ARM stuff uses a multitude of solutions.
Reply
#14
Here it's something I don't understand.
For all HD movies/clips the standard is Rec.709 YCbCr limited. Most TV are default to this standard and RGB/full is a distinct option, mostly for games or smart (browsing, etc).
Professional calibration for TV are done in Rec.709 YCbCr.
Why a mediaplayer then will output in other color space? Why conversions?
Reply
#15
(2015-12-16, 09:05)hansolo Wrote: Here it's something I don't understand.
For all HD movies/clips the standard is Rec.709 YCbCr limited. Most TV are default to this standard and RGB/full is a distinct option, mostly for games or smart (browsing, etc).
Professional calibration for TV are done in Rec.709 YCbCr.
Why a mediaplayer then will output in other color space? Why conversions?

Mainly because PCs and GPUs are based around RGB colour space internally not YCbCr. VGA was an RGB standard (based on 0-255 colour space), and the form of DVI most commonly used by PCs was RGB 0-255 too. It was only really when PCs started having to handle DVDs and DVB/ATSC tuners, and start outputting HDMI video in 16-235 that 16-235 YCbCr colour space became an issue for PCs - but often you were playing 16-235 YCbCr content on a 0-255 RGB VGA/DVI monitor etc.

More than anything it's a heritage thing I think. PCs come from an RGB 0-255 heritage. Video comes from a 16-235 YCbCr heritage.

If you start thinking about 4:2:2 and 4:2:0 you also see that these colour spaces are not ideal for GUI and photo rendering as they have reduced chroma resolution. If you've ever run Windows in a 4:2:2 YCbCr mode you know how nasty it can look with smeared colours.
Reply

Logout Mark Read Team Forum Stats Members Help
[FAQ] Video Levels and Color Spaces Demystified -- Get the Best Picture Quality1