2015-07-25, 21:58
Introduction to HDR to SDR Tone Mapping
Topics Covered:
Intro to Tone Mapping for Projectors and Low Peak Nits Displays
The majority of HDR tone mapping development in madVR has focused on tone mapping for projectors with a calibrated peak brightness of 50-150 nits. The limited light output of a front projector creates a challenge in displaying HDR videos due to the high mastered nit levels and extreme levels of contrast demanded by PQ HDR content.
The HDR10 standard considers a display to be well-suited for HDR playback, or “UHD Premium,” when it has a minimum brightness of 540 peak nits. While conservative by the standards of many modern flat panel TVs, 540 nits remains hundreds of nits above the maximum calibrated peak brightness (or even the vivid or dynamic picture modes) of all current bulb-based or laser HDR projectors.
The rendering intent of HDR10 video can be best understood by looking at the PQ EOTF used to master all PQ (Perceptual Quantizer) source values. Approximately 58% of the available bits on the PQ source scale are distributed between 0-100 nits (known as PQ “reference white”) with the remaining bits up to 10,000 nits reserved to convey highlight peaks, or HDR “specular highlights.” Each PQ bit is fixed to the original PQ EOTF, so it is intended to be displayed at a precise amount of luminance by any display.
While there is the odd movie such as The Meg with Frame Average Light Levels that consistently reach over 1,000 nits, all brightness peaks greater than 100 nits are still considered specular highlights meant to be used sparingly and not where the majority of HDR videos are mastered. Specular peaks such as fire and flames, headlights, outdoor lighting, sunlight, reflections, open skies and any other sources of direct or indirect lighting are what differentiate HDR content from SDR content and make the HDR image really pop with its enhanced contrast. But, as demonstrated by almost all movies decoded by madMeasureHDR, specular highlights tend to only make up 5-10% of the total mastered brightness levels of most current HDR10 videos. This infrequent use of the brightness levels above 100 nits would be considered a “normal” use of the PQ HDR luminance (bit depth) scale.
Because most bits and source levels (up to 90% or greater) are contained within the first 100 nits of the PQ curve, the PQ EOTF is designed to display most source values 1:1 on true HDR displays (which remember are recommended to have a minimum of 540 nits of peak luminance) and deal with any specular highlight peaks above 100 nits with some type of roll-off curve. Industry standard reports such as Report BT.2390 reinforced this rigid rendering intent by recommending that HDR10 displays start the roll-off point for the tone mapping curve at or above 100 nits to compress the entire 10,000 nits PQ scale if the display could produce at least 479 peak nits in a 10% window.
HDR10 was intended to be an open standard, so not all HDR displays follow this advice and may tone map some of the source values below 100 nits. But maintaining the creative intent of PQ HDR video generally requires linear tracking of the PQ EOTF as far as possible to avoid excessively dimming the Average Picture Level (APL) by starting the roll-off point too early on the EOTF and lowering the brightness of too many source values. This is due to the PQ EOTF being a fixed tone curve designed to be displayed without any increases or decreases to the mastered source luminance to ensure each HDR video is consistently reproduced in the home environment with a neutral presentation that closely mimics the source as it appeared on the original HDR mastering monitor. The PQ HDR standard was designed as a display-referred (or HDR mastering display-referred) method of representing consumer HDR video.
If this mastering display-referred approach is applied to projectors barely able to represent light levels above legacy SDR video, it quickly becomes apparent that the PQ source scale is inadequate for these displays. An HDR display with a peak brightness only slightly higher than 100 nits simply lacks the headroom to have linear tracking of the lower PQ EOTF curve up to 100 nits and still have enough light output (nits) left in reserve to render the brighter specular highlights with representative contrast.
If you start the roll-off of the PQ curve lower, you still may find the display has not reduced the brightness of reference white far enough to adequately compress the brightest specular highlights. For example, starting a roll-off of the PQ EOTF on a 100 nits projector at a mere 20 PQ nits may only reduce reference white from 100 nits to 50 nits and leave only an additional 50 nits (a 2:1 contrast ratio between the highlights and reference white) to handle all brighter source values mastered up to 10,000 nits.
If an HDR scene was graded to 4,000 nits, the contrast ratio between the mastered specular highlights and reference white is 40:1 (4,000 nits frame peak/100 nits reference white = 40:1). Representing this 40:1 contrast ratio on a 100 nits display with a 2:1 contrast ratio between the highlights and reference white makes it quite difficult to render all of the individual source luminance steps from the much brighter reference scene without flattening many of the PQ steps. This can lead to many pixels ending up grouped together at the top of the display curve, either clipping the specular highlights or flattening all detail in these image areas.
This type of aggressive roll-off of the PQ EOTF would only work if the specular highlights represented a very small portion of the overall picture. Further, using a lower roll-off point on a fixed tone curve will create many flat spots from 20 nits to 100 nits where larger numbers of pixels under 100 nits are reduced in visible distance creating more visible overlap of the picture in places where these pixels are compressed.
Trying to find the ideal roll-off point on a projector for a single HDR movie often leads to a need to manually adjust the contrast for every scene in the movie as bright highlights pop-up in each scene.
For example, a fixed roll-off may work for most scenes in a movie until you come across one scene with a patch of very bright specular highlights that don’t quite fit within the existing tone curve:
YouTube Link to Source of Screenshots Below
Sully UHD Blu-ray - VAVA 4K Laser Projector:
When encountering a scene such as this, you might reach for the remote to turn down the brightness or contrast until the highlights are brought back into the frame without looking dark or compressed:
Sully UHD Blu-ray - VAVA 4K Laser Projector:
However, this ideal tone curve might end up looking too dim in the next scene when those bright highlights are no longer present and more content is presented near black:
Sully UHD Blu-ray - VAVA 4K Laser Projector:
A better way to represent HDR video on displays with a limited peak brightness is to use a tone mapping curve that can automatically adjust the source steps to balance the contrast of the image for the entire display curve without having to follow the rigid PQ EOTF at any point and determine how much of the EOTF must be compressed for each movie scene.
It so happens the legacy SDR gamma scale is already optimized for this type of automatic source remapping. Unlike the display-referred HDR PQ standard, SDR Gamma is a scene-referred tone curve (similar to Hybrid-Log Gamma, HLG). This means any SDR source is automatically stretched or compressed by the gamma curve to maintain the same contrast ratio (but not the same absolute brightness) as the original source mastering display at any displayed brightness. The gamma curve will increase or decrease the luminance of all source values based on the calibrated peak nits of the display panel. However, the contrast difference between the brightest and dimmest source values remains the same when scaled in power law gamma.
HDR to SDR tone mapping uses the built-in contrast scaling of the SDR gamma luminance scale to remap all source values of HDR sources to match any SDR gamma curve. This wholesale remapping of the source steps offers far greater control over the Average Picture Level compared to PQ EOTF rendering by changing the composition of the entire image to have the same local contrast at any desired brightness within the available display curve.
In Part II, we will discuss how PQ HDR source values are mapped and rendered on the relative SDR gamma curve.
Part II: HDR to SDR Gamma Compression
Topics Covered:
- Part I: Intro to Tone Mapping for Projectors and Low Nits Displays;
- Part II: HDR to SDR Gamma Compression;
- Part III: Dynamic Display Target Nits.
Intro to Tone Mapping for Projectors and Low Peak Nits Displays
The majority of HDR tone mapping development in madVR has focused on tone mapping for projectors with a calibrated peak brightness of 50-150 nits. The limited light output of a front projector creates a challenge in displaying HDR videos due to the high mastered nit levels and extreme levels of contrast demanded by PQ HDR content.
The HDR10 standard considers a display to be well-suited for HDR playback, or “UHD Premium,” when it has a minimum brightness of 540 peak nits. While conservative by the standards of many modern flat panel TVs, 540 nits remains hundreds of nits above the maximum calibrated peak brightness (or even the vivid or dynamic picture modes) of all current bulb-based or laser HDR projectors.
The rendering intent of HDR10 video can be best understood by looking at the PQ EOTF used to master all PQ (Perceptual Quantizer) source values. Approximately 58% of the available bits on the PQ source scale are distributed between 0-100 nits (known as PQ “reference white”) with the remaining bits up to 10,000 nits reserved to convey highlight peaks, or HDR “specular highlights.” Each PQ bit is fixed to the original PQ EOTF, so it is intended to be displayed at a precise amount of luminance by any display.
While there is the odd movie such as The Meg with Frame Average Light Levels that consistently reach over 1,000 nits, all brightness peaks greater than 100 nits are still considered specular highlights meant to be used sparingly and not where the majority of HDR videos are mastered. Specular peaks such as fire and flames, headlights, outdoor lighting, sunlight, reflections, open skies and any other sources of direct or indirect lighting are what differentiate HDR content from SDR content and make the HDR image really pop with its enhanced contrast. But, as demonstrated by almost all movies decoded by madMeasureHDR, specular highlights tend to only make up 5-10% of the total mastered brightness levels of most current HDR10 videos. This infrequent use of the brightness levels above 100 nits would be considered a “normal” use of the PQ HDR luminance (bit depth) scale.
Because most bits and source levels (up to 90% or greater) are contained within the first 100 nits of the PQ curve, the PQ EOTF is designed to display most source values 1:1 on true HDR displays (which remember are recommended to have a minimum of 540 nits of peak luminance) and deal with any specular highlight peaks above 100 nits with some type of roll-off curve. Industry standard reports such as Report BT.2390 reinforced this rigid rendering intent by recommending that HDR10 displays start the roll-off point for the tone mapping curve at or above 100 nits to compress the entire 10,000 nits PQ scale if the display could produce at least 479 peak nits in a 10% window.
HDR10 was intended to be an open standard, so not all HDR displays follow this advice and may tone map some of the source values below 100 nits. But maintaining the creative intent of PQ HDR video generally requires linear tracking of the PQ EOTF as far as possible to avoid excessively dimming the Average Picture Level (APL) by starting the roll-off point too early on the EOTF and lowering the brightness of too many source values. This is due to the PQ EOTF being a fixed tone curve designed to be displayed without any increases or decreases to the mastered source luminance to ensure each HDR video is consistently reproduced in the home environment with a neutral presentation that closely mimics the source as it appeared on the original HDR mastering monitor. The PQ HDR standard was designed as a display-referred (or HDR mastering display-referred) method of representing consumer HDR video.
If this mastering display-referred approach is applied to projectors barely able to represent light levels above legacy SDR video, it quickly becomes apparent that the PQ source scale is inadequate for these displays. An HDR display with a peak brightness only slightly higher than 100 nits simply lacks the headroom to have linear tracking of the lower PQ EOTF curve up to 100 nits and still have enough light output (nits) left in reserve to render the brighter specular highlights with representative contrast.
If you start the roll-off of the PQ curve lower, you still may find the display has not reduced the brightness of reference white far enough to adequately compress the brightest specular highlights. For example, starting a roll-off of the PQ EOTF on a 100 nits projector at a mere 20 PQ nits may only reduce reference white from 100 nits to 50 nits and leave only an additional 50 nits (a 2:1 contrast ratio between the highlights and reference white) to handle all brighter source values mastered up to 10,000 nits.
If an HDR scene was graded to 4,000 nits, the contrast ratio between the mastered specular highlights and reference white is 40:1 (4,000 nits frame peak/100 nits reference white = 40:1). Representing this 40:1 contrast ratio on a 100 nits display with a 2:1 contrast ratio between the highlights and reference white makes it quite difficult to render all of the individual source luminance steps from the much brighter reference scene without flattening many of the PQ steps. This can lead to many pixels ending up grouped together at the top of the display curve, either clipping the specular highlights or flattening all detail in these image areas.
This type of aggressive roll-off of the PQ EOTF would only work if the specular highlights represented a very small portion of the overall picture. Further, using a lower roll-off point on a fixed tone curve will create many flat spots from 20 nits to 100 nits where larger numbers of pixels under 100 nits are reduced in visible distance creating more visible overlap of the picture in places where these pixels are compressed.
Trying to find the ideal roll-off point on a projector for a single HDR movie often leads to a need to manually adjust the contrast for every scene in the movie as bright highlights pop-up in each scene.
For example, a fixed roll-off may work for most scenes in a movie until you come across one scene with a patch of very bright specular highlights that don’t quite fit within the existing tone curve:
YouTube Link to Source of Screenshots Below
Sully UHD Blu-ray - VAVA 4K Laser Projector:
When encountering a scene such as this, you might reach for the remote to turn down the brightness or contrast until the highlights are brought back into the frame without looking dark or compressed:
Sully UHD Blu-ray - VAVA 4K Laser Projector:
However, this ideal tone curve might end up looking too dim in the next scene when those bright highlights are no longer present and more content is presented near black:
Sully UHD Blu-ray - VAVA 4K Laser Projector:
A better way to represent HDR video on displays with a limited peak brightness is to use a tone mapping curve that can automatically adjust the source steps to balance the contrast of the image for the entire display curve without having to follow the rigid PQ EOTF at any point and determine how much of the EOTF must be compressed for each movie scene.
It so happens the legacy SDR gamma scale is already optimized for this type of automatic source remapping. Unlike the display-referred HDR PQ standard, SDR Gamma is a scene-referred tone curve (similar to Hybrid-Log Gamma, HLG). This means any SDR source is automatically stretched or compressed by the gamma curve to maintain the same contrast ratio (but not the same absolute brightness) as the original source mastering display at any displayed brightness. The gamma curve will increase or decrease the luminance of all source values based on the calibrated peak nits of the display panel. However, the contrast difference between the brightest and dimmest source values remains the same when scaled in power law gamma.
HDR to SDR tone mapping uses the built-in contrast scaling of the SDR gamma luminance scale to remap all source values of HDR sources to match any SDR gamma curve. This wholesale remapping of the source steps offers far greater control over the Average Picture Level compared to PQ EOTF rendering by changing the composition of the entire image to have the same local contrast at any desired brightness within the available display curve.
In Part II, we will discuss how PQ HDR source values are mapped and rendered on the relative SDR gamma curve.
Part II: HDR to SDR Gamma Compression
