I’m sure you people think I have nothing better to do than talk about video rendering on the Internet, but that's not true. I wrote this a couple days ago after trying about 30 times to explain how HDR -> HDR or HDR -> SDR tone mapping works. After some reading on a couple of forums related to HDR photography, I think I finally figured it out. The passage below explains how colors are created by video renderers and how this relates to tone mapped images.
Floating Point Math and Illegal RGB Values
When doing color matrixing, we are generally working with floating point numbers. This is true when converting from Y'CbCr to RGB.
Floating point numbers are numbers that contain decimal places that can move up or down (float) to accommodate a wider range of values. For RGB, floating point values for red, green and blue range between
0 and
1.0, where
0 = black and
1.0 = white. Decimal amounts such as 0.804329 are also possible. These decimal values can be converted to 0-255 or 0-1023 bit depths by rounding to the nearest decimal --
0.804239 floating point = 205 RGB (0.804239 x 255 = 205.080945). Whole numbers without decimals (0-255) are called integers, and this is how color information is presented. So a conversion from floating point (0.804) to integer (205) is required when presenting color information in standard bit depths. A 32-bit floating point number has six significant digits (e.g. 0.523456), where the significance starts after the first nonzero digit. Large floating point, decimal numbers offer a lot of precision in displaying colors to lower bit depths. This makes floating point numbers desirable for color mapping.
Having a floating decimal allows for the values to fluctuate above 1.0 and below 0. This should not occur in a properly graded color space, but is very common after tone mapping. Reducing (Y) luminance and then combining it to produce Y'CbCr will result in many floating point RGB values that are below 0 or above 1.0. This is due to the non-linear relationship between luminance and color. When adjusting white (luminance), other colors will be negatively affected because they contain different amounts of white and will not scale linearly.
Let's use an example.
The image below is taken from a camera and plotted on a standard
CIE color diagram:
The dots that fall within the gamut all have
RGB values > 0 and less than 1.0, if they fit into the gamut. But, you'll notice some dots fall outside the gamut. Values in the top right have
B (blue) < 0. Values in the bottom left have both
R (red) < 0,
and
G (green) < 0. These are negative RGB values, which means the colors are too saturated to fit into the gamut. A color such as this cannot be shown by the available colors of the display. This is all too common in tone mapped images, as many of the dots will fall outside the gamut when converted from Y'CbCr to RGB after tone mapping.
A value such as
(-0.5, 0.0, 0.0) means red information would have to be removed from any other color it is combined with. This color must be mapped back into the gamut so it can be converted into a standard integer value. The easiest way to do this is to make it
(0.0, 0.0, 0.0). The oversaturated red is now pure black.
It is also possible to have color values greater than 1.0. A standard red would have a value of
(1.0, 0.0, 0.0). Yet a tone mapped red can come out with values
(2.0, 0.0, 0.0), which would produce a very intense, glowing red. This red pixel has too much luminance and needs to be moved back into the gamut. It becomes
(1.0, 0.0, 0.0). It is simply not possible to display a 1,000 nits red on a display calibrated for 1,000 nits white.
In summary, dealing with out-of-gamut colors is a consequence of manipulating luminance and attempting to create new RGB integer values due to the non-linearity of color spaces. Pixels that were previously in gamut will become out of gamut. The techniques to deal with this are varied but will always result in some compromise to reproduce the original position of the color in the CIE diagram. Once moved into the gamut, the color is no longer the same hue, saturation and luminance as when it started, which will change the integrity of the original tone mapped image. Such color inaccuracies should be expected from any HDR display with a peak nits lower than the HDR master.
The recommended approach to handling colors after tone mapping is to preserve the hue as best as possible and adjust the saturation and luminance until the color fits into the target bit depth. This is the approach recommended by Dolby, as changes in hue will be more noticeable than changes in saturation or luminance.
P.S. So, if you download a tone mapping algorithm such as BT.2390 to map Y, find a Y'CbCr to RGB conversion formula and clamp all RGB values to 0.0 and 1.0, you could create your own "dumb" tone mapping algorithm. This is actually not far off how most HDR displays do it.
