2015-04-17, 21:33
Part III: Dynamic Display Target Nits
Note: Part I of Introduction to HDR to SDR Tone Mapping covers some fundamental issues in tone mapping for projectors. Part II offers a detailed technical definition of the display target nits and HDR to SDR tone mapping.
Topics Covered:
What Is a Dynamic Display Target Nits?
The dynamic target nits option changes the display target nits for each detected movie scene in an HDR10 video. These changes in display target nits alter the amount of dynamic range captured by the tone curve for each movie scene with nits levels mastered above the display real display peak nits entered in madVR.
Changes to the display target nits resemble a change in displayed gamma: e.g., gamma 2.20, 2.25, 2.30, 2.35, 2.40, 2.45, etc., with the dynamic display target nits simply selecting the best "gamma curve" for each movie scene that provides the ideal amount of dynamic range compression to prevent the image from appearing too dim or too bright (clipped in the highlights).
The display target nits is adjusted on a scene-by-scene basis based on detected changes to madVR's real-time brightness histogram that looks for visible changes in average and peak pixel luminance. A movie scene is not described on the basis of having a distinct beginning and end, but as having a visible change in Average Picture Level (APL) that signals that a noticeable change in the composition of the image has occurred. By this definition, scene changes can be frequent. For instance, a muzzle flash from a gun would not be considered a scene change, but a conversation between two people that starts in front of a bright window and moves continuously into another room would be considered a visible change in APL that could benefit from a change in display target nits.
Changes to the display target nits occur regularly and in small increments to keep scene and corresponding APL changes smooth and largely invisible to the viewer. This prevents brightness pumping artifacts caused by frame-by-frame adjustments to the tone curve by not reacting to sudden camera cuts or large increases or decreases in peak or average brightness.
How Much Dynamic Range Should Be Compressed?
Determining the ideal display target nits for each movie scene requires some upfront understanding of the display limits. There is only so much dynamic range from the HDR source video that you can fit into a display with less peak brightness. If a display is asked to render one pixel at 1 nit and another at 1,000 nits, it must balance a contrast ratio of 1,000:1 between the brightest and dimmest objects in the scene. If the display can only produce 100 peak nits, it can at most create a native contrast ratio of 100:1 between a pixel mastered at 1 nit and the display limit of 100 peak nits. In order to replicate the native 1,000:1 contrast ratio on a 100 peak nits display, that 1 nit pixel must be rendered 10x darker (1,000 peak nits / 100 actual display nits) than originally mastered along with most other pixels between 0-100 nits that make up the majority of the image. At 10x compression, that 1,000 nit scene would likely appear very dark on a 100 nits display and necessitate using a much lower compression ratio to fit the scene to the display.
So while it might be desirable to compress 1,000 scene nits of dynamic range into a 100 nit display range (1,000 display target nits at 100 real display peak nits), it will quickly become apparent that the display lacks the necessary contrast to make this work for most movie scenes. Remember, image tones can only be redistributed by a tone curve within the available display range and not manufactured, so you only have so much contrast to work with for any display. The greater the disparity between the available display peak nits and the source peak nits, the more certain image tones will have be prioritized over others to prevent the entire image from appearing too dim from more linear display compression.
Because the display target nits is in fact scaled linearly by the display gamma curve at a ratio of display target nits / real display peak nits, the brightness of the entire image will decrease linearly with increases to the display target nits. The majority of a PQ HDR video is mastered within the bottom third of the PQ EOTF curve (or display gamma curve) where reference white is rendered, meaning linear gamma compression from high display target nits / real display peak nits compression ratios will cause the image to darken quite quickly as the display runs out of available brightness. The lower the peak nits of the target display, the faster the image will fade to black as the display struggles to create the necessary contrast.
Peak Luminance of PQ Reference White at Various Display Target Nits at 100 real display peak nits:
Mission Impossible: Ghost Protocol (1,089 frame peak nits)
100 display target nits (1:1) 100 nits reference white:
200 display target nits (2:1) 50 nits reference white:
300 display target nits (3:1) 33 nits reference white:
400 display target nits (4:1) 25 nits reference white:
500 display target nits (5:1) 20 nits reference white:
1,000 display target nits (10:1) 10 nits reference white:
The FALL Algo and Scene-Based Display Target Nits
The ideal display target nits for a given movie scene would take into account the available contrast of the end display and find a balance between preserving the source APL (preserving as much of the mastered brightness levels as possible) and adequately compressing the dynamic range of each scene (preserving the mastered HDR contrast without clipping the specular highlights).
Some discretion is required in this decision, as any display with less peak brightness than the source mastering monitor will have to give up some amount of brightness in order to replicate the higher contrast of the brighter mastering display and the perfect balance of brightness vs. contrast enhancement for each scene is often in the eye of the beholder. So the ideal HDR tone mapping solution for dimmer HDR displays would permit some type of user adjustment to the balance of brightness preservation and adding HDR contrast.
madVR tackles the delicate balance of brightness vs. contrast by calculating what the Average Picture Level of the image will be at the end display after tone mapping and display gamma compression are applied. This uses the real display peak nits provided to madVR as the target display range with the goal of never allowing any bright HDR scene to be compressed below a certain level of the target display range.
The original brightness of each movie scene is determined based on its Average Picture Level that is described in static HDR10 metadata as the Maximum Frame Average Light Level, or MaxFALL, which madVR expands by calculating a FALL value for each video frame (the frame FALL). Because display Gamma Compression is applied linearly to the entire gamma curve, it makes most sense to know the APL of each scene before it is compressed to determine how much of the original scene APL can be lost before the starts to become too dim at the end display.
The frame FALL is defined by madVR as the average peak nits of all pixels in one frame — the midpoint brightness of all pixels in each video frame. A frame FALL is assigned to each scene by averaging the peak luminance of all pixels in the first frame of the scene. This is done using a real-time luminance and hue brightness histograms.
The real-time luminance and hue histograms measure the peak luminance of each incoming PQ source value and group all pixels into columns that define the mastered nits and relative positioning of each pixel within the image (e.g., shadow, midtone or highlight). From left-to-right on the histogram, pixel luminance is distributed across the lower shadows, middle midtones and upper highlights with the highest hills on the right side of the histogram representing the scene peak nits and the average luminance of all pixels in the histogram becoming the calculated frame FALL.
madVR OSD - Luminance & Hue Brightness Histograms:
The frame FALL of any scene tells you two important things: it tells you how far above black the scene is mastered and how much of the image is likely mastered above 100 nits as specular highlights.
Knowing the average brightness of each movie scene relative the available real display peak nits of the end display gives you an estimate of much headroom you have above the black level of the target display to lineary (or gamma) compress the dynamic range of the scene before visible near detail black is lost. This provides madVR with the ability to choose a gamma compression ratio that moves each scene back into the available display range without ever moving too much of the image into the darkest nits levels.
The frame FALL also offers some indication of where most brightness levels are distributed within the scene. High frame FALLs generally have large portions of the image mastered as specular highlight detail above 100 nits. These brighter HDR scenes require that much higher display target nits are used to adequately move the highlights far enough within the smaller display range to avoid mapping too many pixels near the display peak limit. This most often creates obvious display clipping where large numbers of pixels become grouped together near peak white causing visible losses of detail and color saturation from blown out highlight detail.
Low frame FALLs, on the other hand, tend to have smaller specular highlights that make up a small fraction of the overall frame. These less bright HDR scenes can be compressed with lower display target nits that sacrifice some specular highlight detail that often goes unnoticed anyways, but spares these scenes from becoming too dark by limiting tone compression for the majority of the image.
Examples of Typical HDR10 Movie Scene FALLs:
Low FALL: 0-10 nits (MI Ghost Protocol):
Average FALL: 10-25 nits (MI Ghost Protocol):
High FALL: > 25 nits (MI Ghost Protocol):
How the FALL Algo Works:
Scenes are assigned a display target nits using a general formula that takes into account both the frame peak of the scene to determine if dynamic range compression is required and a frame FALL value that is used to judge the amount of dynamic range compression that can be applied. These changes to the display target nits occur whenever a scene change is determined to have occured based on the dynamic hue and luminance histograms.
If the scene peak nits is equal to or less than the real display peak nits entered in madVR, the display target nits will match the real display nits and all source values are sent without tone mapping. If the nits level of a scene rises above the real display peak nits, the display target nits will increase above the real display peak nits to better compress any source values that are mastered above the available light output of the display.
Increases to the real display peak nits are applied based on the frame FALL of the scene. The frame FALL of each scene is added to the real display peak nits and then multiplied by an adjustable multiplier to determine the final display target nits for the scene. As the scene APL increases, the display target nits will also increase linearly to ensure high scene APLs are matched to higher levels of dynamic range compression.
If the calibrated peak nits of the target display is 150 nits or less, scenes that require dynamic range compression will use a default 150 display target nits as a base value. This fixed 150 display nits ensures the calculated display target nits for each scene remains consistently high enough to provide enough dynamic range compression for displays that lack the necessary peak brightness to compress bright HDR scenes back into the the upper end of the display range without excessive range clipping.
FALL Algo Formula:
Max(150 nits, real display peak nits) + [2 x scene FALL x (dynamic tuning multiplier)] = final display target nits
The display target nits moves linearly with changes to the scene FALL.
Scene Peak Nits <= real display peak nits: display target nits = real display peak nits
When a scene can be rendered accurately by a display, no compression is applied. Scenes with brightness levels below the real display peak nits are rendered with 1:1 PQ EOTF tracking and that represents the source luminance as it was originally intended.
Scene Peak Nits > real display peak nits: display target nits > real display peak nits
Low frame FALL:
When a scene with a peak nits above the real display peak nits has low frame FALL, only a small increase is made to the display target nits to reflect the low black floor of the scene and the small amount of mastered specular highlights. Some clipping of the specular highlights in permitted to protect the majority of the scene from unwanted tone compression.
High frame FALL:
When a scene with a peak nits above the real display peak nits has high frame FALL, a much larger increase is made to the display target nits to reflect the high black floor of the scene and the larger amount of specular highlight detail mastered above 100 nits. More aggressive dynamic range compression is applied to move the scene levels far enough into display range with enough headroom to prevent high scene APLs from clipping at the display peak and washing out large portions of the image.
Below is an example of how the FALL algo impacts the frame FALL after tone mapping when applied to scenes with varying FALL values. As you can see, any increase to the scene FALL results in linear increases to the display target nits. Yet, brighter scenes still end up with higher FALL values after tone mapping than less bright scenes and all frame FALLs after compression become closer in average brightness — the average brightness levels of the entire video are compressed to a narrower display range.
Example of Applying Different Display Target Nits to Various Scene FALLs:
Scene Peak: 100 nits, frame FALL: 2 nits
100 display target nits / 100 real display peak nits (1:1 compression) = 2 nits frame FALL after tone mapping
Scene Peak: 500 nits, frame FALL: 8 nits
166 display target nits / 100 real display peak nits (1.66:1 compression) = 5 nits frame FALL after tone mapping
Scene Peak: 1,000 nits, frame FALL: 16 nits
182 display target nits / 100 real display peak nits (1.82:1 compression) = 9 nits frame FALL after tone mapping
Scene Peak: 1,000 nits, frame FALL: 30 nits
210 display target nits / 100 real display peak nits (2.1:1 compression) = 14 nits frame FALL after tone mapping
Scene Peak: 1,500 nits, frame FALL: 60 nits
270 display target nits / 100 real display peak nits (2.7:1 compression) = 22 nits frame FALL after tone mapping
The original FALL values for each scene in this example range from 2 nits - 60 nits (58 nits). After applying the dynamic display target nits to the source values of all scenes, the scene FALLs are compressed to within 20 nits (2 nits - 22 nits).
The compressed dynamic range of all scenes now more accurately reflects the smaller dynamic range of the end display. The result is a similar APL for each scene with small and smooth changes in display target nits that blend together as scene changes occur. Keeping the display curve in the "sweet spot" where average brightness remains consistent from scene-to-scene also provides the secondary benefit of hiding display target nits changes that might otherwise result in visible brightness pumping artifacts.
Histogram-Shaped Tone Mapping — Local Tone Mapping
The display target nits value provides a simple method to handle the variable dynamic range of HDR videos by providing means to dynamically scale scene contrast as the video plays. However, this form of tone mapping uses global curves with linear tone compression that are designed to preserve global contrast and the average brightness of each reference scene.
The problem with this global approach to tone mapping when applied to the typical movie scene is that most movies are not color graded to have equal or consistent amounts of content rendered in the shadows, midtones and highlights. Each scene will have its own display curve that will inevitably lead to disproportionate losses of fine detail in certain areas of the image and reductions in local contrast as certain image areas end up compressed more than others.
For instance, a scene may have a high measured frame FALL, but have almost equal amounts of content mastered as specular highlights and shadow detail. Use of high display target nits for these mixed-brightness scenes will linearly compress both the specular highlights and shadows at the same time, which will create desirable compression for the bright specular highlights, but this aggressive gamma compression will also lead to visible losses of near black detail.
Similarly, a scene with a high peak brightness but an average frame FALL may have most of the image rendered in the shadows and midtones with smaller specular highlights that occupy a limited amount of the screen space. This would lead to the selection of a low to moderate display target nits for the scene that would result in a significant loss of contrast from the original scene as the low display target nits must forcibly compresses the highlights in order to preserve the majority of the scene APL.
To restore some of the local details and local contrast typically lost in HDR tone mapping, madVR applies uses local tone mapping to further optimize the gamma curve. Local tone mapping enhances local contrast and better preserves local details by reducing tone compression for areas of the image that have the most visible pixels and enhancing the contrast of these areas compared to any nearby objects.
madVR calls its approach to local tone mapping Histogram-Shaped Tone Mapping (HSTM). HSTM uses the real-time pixel luminance histogram to prioritize tone compression for areas of the image that have the highest pixel distribution. Because luminance histograms report the distribution of pixels concentrated at a certain luminance and tone (shadow, midtone or highlight), a large number of pixels mapped to the same mastered luminance will create a large hill in the histogram.
Hills that are above the midpoint of the histogram have the most visible pixels and are compressed less to restore some lost brightness, detail and color volume. Hills that are below the midpoint of the histogram chart are compressed more to appear darker and less colorful to offset the expansion of the larger hills on the gamma curve.
Curve expansion and compression from HSTM changes the shape of the gamma curve and will vary from scene-to-scene depending on the unique pixel distribution of each scene. Bright HDR scenes see improvements in contrast through local contrast enhancement. Mixed-brightness and dim HDR scenes see improvements in near black shadow detail through local contrast recovery. The end result is that any of the shadows, midtones or highlights are boosted by HSTM depending on where the bulk of the pixel luminance is distributed in each scene.
What Is Histogram-Shaped Tone Mapping?
*To be added at a later date.
Spears & Munsil UHD HDR Benchmark 10,000 nits MaxCLL (HSTM Off):
Spears & Munsil UHD HDR Benchmark 10,000 nits MaxCLL (HSTM On):
Summary of Primary Pixel Shaders Tone Mapping Image Processing Steps:
1. FALL Algo:
The ideal display target nits is calculated for each scene change as detected by the real-time luminance and hue histograms.
2. PQ BT.2390 Tone Mapping Curve:
madVR calculates a compressed display curve for the scene that matches the selected display target nits by applying a PQ BT.2390 tone mapping curve to roll-off the frame peak nits so the brightest pixels in the scene are mapped to the display target nits.
3. Histogram-Shaped Tone Mapping:
The compressed PQ BT.2390 gamma curve is then optimized using the pixel luminance histogram to improve local detail preservation and enhance contrast. Histogram hills above the midpoint of the graph with the highest concentration of pixels with similar mastered luminance are expanded to be brighter and more colorful, and pixel distributions below the midpoint of the histogram with low concentrations of similar mastered pixels are further compressed to be darker than more visible areas. Depending on the distribution of pixel luminance in each scene, these HSTM adjustments will either lead to improved HDR contrast (local contrast enhancement) or a restoration of detail in the shadows and midtones (local contrast recovery).
4. Recalculate Final Display FALL:
The final HSTM processing result is rescaled in displayed FALL to have the same average on-screen brightness (APL) as the selected display target nits before HSTM tone mapping was applied.
Dynamic Display Target Nits (Continues Below...)
Note: Part I of Introduction to HDR to SDR Tone Mapping covers some fundamental issues in tone mapping for projectors. Part II offers a detailed technical definition of the display target nits and HDR to SDR tone mapping.
Topics Covered:
- What is a dynamic display target nits;
- How much dynamic range should be compressed;
- The FALL Algo and scene-based display target nits;
- Histogram-Shaped Tone Mapping;
- HDR to SDR display target nits waveforms with DaVinci Resolve.
What Is a Dynamic Display Target Nits?
The dynamic target nits option changes the display target nits for each detected movie scene in an HDR10 video. These changes in display target nits alter the amount of dynamic range captured by the tone curve for each movie scene with nits levels mastered above the display real display peak nits entered in madVR.
Changes to the display target nits resemble a change in displayed gamma: e.g., gamma 2.20, 2.25, 2.30, 2.35, 2.40, 2.45, etc., with the dynamic display target nits simply selecting the best "gamma curve" for each movie scene that provides the ideal amount of dynamic range compression to prevent the image from appearing too dim or too bright (clipped in the highlights).
The display target nits is adjusted on a scene-by-scene basis based on detected changes to madVR's real-time brightness histogram that looks for visible changes in average and peak pixel luminance. A movie scene is not described on the basis of having a distinct beginning and end, but as having a visible change in Average Picture Level (APL) that signals that a noticeable change in the composition of the image has occurred. By this definition, scene changes can be frequent. For instance, a muzzle flash from a gun would not be considered a scene change, but a conversation between two people that starts in front of a bright window and moves continuously into another room would be considered a visible change in APL that could benefit from a change in display target nits.
Changes to the display target nits occur regularly and in small increments to keep scene and corresponding APL changes smooth and largely invisible to the viewer. This prevents brightness pumping artifacts caused by frame-by-frame adjustments to the tone curve by not reacting to sudden camera cuts or large increases or decreases in peak or average brightness.
How Much Dynamic Range Should Be Compressed?
Determining the ideal display target nits for each movie scene requires some upfront understanding of the display limits. There is only so much dynamic range from the HDR source video that you can fit into a display with less peak brightness. If a display is asked to render one pixel at 1 nit and another at 1,000 nits, it must balance a contrast ratio of 1,000:1 between the brightest and dimmest objects in the scene. If the display can only produce 100 peak nits, it can at most create a native contrast ratio of 100:1 between a pixel mastered at 1 nit and the display limit of 100 peak nits. In order to replicate the native 1,000:1 contrast ratio on a 100 peak nits display, that 1 nit pixel must be rendered 10x darker (1,000 peak nits / 100 actual display nits) than originally mastered along with most other pixels between 0-100 nits that make up the majority of the image. At 10x compression, that 1,000 nit scene would likely appear very dark on a 100 nits display and necessitate using a much lower compression ratio to fit the scene to the display.
So while it might be desirable to compress 1,000 scene nits of dynamic range into a 100 nit display range (1,000 display target nits at 100 real display peak nits), it will quickly become apparent that the display lacks the necessary contrast to make this work for most movie scenes. Remember, image tones can only be redistributed by a tone curve within the available display range and not manufactured, so you only have so much contrast to work with for any display. The greater the disparity between the available display peak nits and the source peak nits, the more certain image tones will have be prioritized over others to prevent the entire image from appearing too dim from more linear display compression.
Because the display target nits is in fact scaled linearly by the display gamma curve at a ratio of display target nits / real display peak nits, the brightness of the entire image will decrease linearly with increases to the display target nits. The majority of a PQ HDR video is mastered within the bottom third of the PQ EOTF curve (or display gamma curve) where reference white is rendered, meaning linear gamma compression from high display target nits / real display peak nits compression ratios will cause the image to darken quite quickly as the display runs out of available brightness. The lower the peak nits of the target display, the faster the image will fade to black as the display struggles to create the necessary contrast.
Peak Luminance of PQ Reference White at Various Display Target Nits at 100 real display peak nits:
Mission Impossible: Ghost Protocol (1,089 frame peak nits)
100 display target nits (1:1) 100 nits reference white:
200 display target nits (2:1) 50 nits reference white:
300 display target nits (3:1) 33 nits reference white:
400 display target nits (4:1) 25 nits reference white:
500 display target nits (5:1) 20 nits reference white:
1,000 display target nits (10:1) 10 nits reference white:
The FALL Algo and Scene-Based Display Target Nits
The ideal display target nits for a given movie scene would take into account the available contrast of the end display and find a balance between preserving the source APL (preserving as much of the mastered brightness levels as possible) and adequately compressing the dynamic range of each scene (preserving the mastered HDR contrast without clipping the specular highlights).
Some discretion is required in this decision, as any display with less peak brightness than the source mastering monitor will have to give up some amount of brightness in order to replicate the higher contrast of the brighter mastering display and the perfect balance of brightness vs. contrast enhancement for each scene is often in the eye of the beholder. So the ideal HDR tone mapping solution for dimmer HDR displays would permit some type of user adjustment to the balance of brightness preservation and adding HDR contrast.
madVR tackles the delicate balance of brightness vs. contrast by calculating what the Average Picture Level of the image will be at the end display after tone mapping and display gamma compression are applied. This uses the real display peak nits provided to madVR as the target display range with the goal of never allowing any bright HDR scene to be compressed below a certain level of the target display range.
The original brightness of each movie scene is determined based on its Average Picture Level that is described in static HDR10 metadata as the Maximum Frame Average Light Level, or MaxFALL, which madVR expands by calculating a FALL value for each video frame (the frame FALL). Because display Gamma Compression is applied linearly to the entire gamma curve, it makes most sense to know the APL of each scene before it is compressed to determine how much of the original scene APL can be lost before the starts to become too dim at the end display.
The frame FALL is defined by madVR as the average peak nits of all pixels in one frame — the midpoint brightness of all pixels in each video frame. A frame FALL is assigned to each scene by averaging the peak luminance of all pixels in the first frame of the scene. This is done using a real-time luminance and hue brightness histograms.
The real-time luminance and hue histograms measure the peak luminance of each incoming PQ source value and group all pixels into columns that define the mastered nits and relative positioning of each pixel within the image (e.g., shadow, midtone or highlight). From left-to-right on the histogram, pixel luminance is distributed across the lower shadows, middle midtones and upper highlights with the highest hills on the right side of the histogram representing the scene peak nits and the average luminance of all pixels in the histogram becoming the calculated frame FALL.
madVR OSD - Luminance & Hue Brightness Histograms:
The frame FALL of any scene tells you two important things: it tells you how far above black the scene is mastered and how much of the image is likely mastered above 100 nits as specular highlights.
Knowing the average brightness of each movie scene relative the available real display peak nits of the end display gives you an estimate of much headroom you have above the black level of the target display to lineary (or gamma) compress the dynamic range of the scene before visible near detail black is lost. This provides madVR with the ability to choose a gamma compression ratio that moves each scene back into the available display range without ever moving too much of the image into the darkest nits levels.
The frame FALL also offers some indication of where most brightness levels are distributed within the scene. High frame FALLs generally have large portions of the image mastered as specular highlight detail above 100 nits. These brighter HDR scenes require that much higher display target nits are used to adequately move the highlights far enough within the smaller display range to avoid mapping too many pixels near the display peak limit. This most often creates obvious display clipping where large numbers of pixels become grouped together near peak white causing visible losses of detail and color saturation from blown out highlight detail.
Low frame FALLs, on the other hand, tend to have smaller specular highlights that make up a small fraction of the overall frame. These less bright HDR scenes can be compressed with lower display target nits that sacrifice some specular highlight detail that often goes unnoticed anyways, but spares these scenes from becoming too dark by limiting tone compression for the majority of the image.
Examples of Typical HDR10 Movie Scene FALLs:
Low FALL: 0-10 nits (MI Ghost Protocol):
Average FALL: 10-25 nits (MI Ghost Protocol):
High FALL: > 25 nits (MI Ghost Protocol):
How the FALL Algo Works:
Scenes are assigned a display target nits using a general formula that takes into account both the frame peak of the scene to determine if dynamic range compression is required and a frame FALL value that is used to judge the amount of dynamic range compression that can be applied. These changes to the display target nits occur whenever a scene change is determined to have occured based on the dynamic hue and luminance histograms.
If the scene peak nits is equal to or less than the real display peak nits entered in madVR, the display target nits will match the real display nits and all source values are sent without tone mapping. If the nits level of a scene rises above the real display peak nits, the display target nits will increase above the real display peak nits to better compress any source values that are mastered above the available light output of the display.
Increases to the real display peak nits are applied based on the frame FALL of the scene. The frame FALL of each scene is added to the real display peak nits and then multiplied by an adjustable multiplier to determine the final display target nits for the scene. As the scene APL increases, the display target nits will also increase linearly to ensure high scene APLs are matched to higher levels of dynamic range compression.
If the calibrated peak nits of the target display is 150 nits or less, scenes that require dynamic range compression will use a default 150 display target nits as a base value. This fixed 150 display nits ensures the calculated display target nits for each scene remains consistently high enough to provide enough dynamic range compression for displays that lack the necessary peak brightness to compress bright HDR scenes back into the the upper end of the display range without excessive range clipping.
FALL Algo Formula:
Max(150 nits, real display peak nits) + [2 x scene FALL x (dynamic tuning multiplier)] = final display target nits
The display target nits moves linearly with changes to the scene FALL.
Scene Peak Nits <= real display peak nits: display target nits = real display peak nits
When a scene can be rendered accurately by a display, no compression is applied. Scenes with brightness levels below the real display peak nits are rendered with 1:1 PQ EOTF tracking and that represents the source luminance as it was originally intended.
Scene Peak Nits > real display peak nits: display target nits > real display peak nits
Low frame FALL:
When a scene with a peak nits above the real display peak nits has low frame FALL, only a small increase is made to the display target nits to reflect the low black floor of the scene and the small amount of mastered specular highlights. Some clipping of the specular highlights in permitted to protect the majority of the scene from unwanted tone compression.
High frame FALL:
When a scene with a peak nits above the real display peak nits has high frame FALL, a much larger increase is made to the display target nits to reflect the high black floor of the scene and the larger amount of specular highlight detail mastered above 100 nits. More aggressive dynamic range compression is applied to move the scene levels far enough into display range with enough headroom to prevent high scene APLs from clipping at the display peak and washing out large portions of the image.
Below is an example of how the FALL algo impacts the frame FALL after tone mapping when applied to scenes with varying FALL values. As you can see, any increase to the scene FALL results in linear increases to the display target nits. Yet, brighter scenes still end up with higher FALL values after tone mapping than less bright scenes and all frame FALLs after compression become closer in average brightness — the average brightness levels of the entire video are compressed to a narrower display range.
Example of Applying Different Display Target Nits to Various Scene FALLs:
Scene Peak: 100 nits, frame FALL: 2 nits
100 display target nits / 100 real display peak nits (1:1 compression) = 2 nits frame FALL after tone mapping
Scene Peak: 500 nits, frame FALL: 8 nits
166 display target nits / 100 real display peak nits (1.66:1 compression) = 5 nits frame FALL after tone mapping
Scene Peak: 1,000 nits, frame FALL: 16 nits
182 display target nits / 100 real display peak nits (1.82:1 compression) = 9 nits frame FALL after tone mapping
Scene Peak: 1,000 nits, frame FALL: 30 nits
210 display target nits / 100 real display peak nits (2.1:1 compression) = 14 nits frame FALL after tone mapping
Scene Peak: 1,500 nits, frame FALL: 60 nits
270 display target nits / 100 real display peak nits (2.7:1 compression) = 22 nits frame FALL after tone mapping
The original FALL values for each scene in this example range from 2 nits - 60 nits (58 nits). After applying the dynamic display target nits to the source values of all scenes, the scene FALLs are compressed to within 20 nits (2 nits - 22 nits).
The compressed dynamic range of all scenes now more accurately reflects the smaller dynamic range of the end display. The result is a similar APL for each scene with small and smooth changes in display target nits that blend together as scene changes occur. Keeping the display curve in the "sweet spot" where average brightness remains consistent from scene-to-scene also provides the secondary benefit of hiding display target nits changes that might otherwise result in visible brightness pumping artifacts.
Histogram-Shaped Tone Mapping — Local Tone Mapping
The display target nits value provides a simple method to handle the variable dynamic range of HDR videos by providing means to dynamically scale scene contrast as the video plays. However, this form of tone mapping uses global curves with linear tone compression that are designed to preserve global contrast and the average brightness of each reference scene.
The problem with this global approach to tone mapping when applied to the typical movie scene is that most movies are not color graded to have equal or consistent amounts of content rendered in the shadows, midtones and highlights. Each scene will have its own display curve that will inevitably lead to disproportionate losses of fine detail in certain areas of the image and reductions in local contrast as certain image areas end up compressed more than others.
For instance, a scene may have a high measured frame FALL, but have almost equal amounts of content mastered as specular highlights and shadow detail. Use of high display target nits for these mixed-brightness scenes will linearly compress both the specular highlights and shadows at the same time, which will create desirable compression for the bright specular highlights, but this aggressive gamma compression will also lead to visible losses of near black detail.
Similarly, a scene with a high peak brightness but an average frame FALL may have most of the image rendered in the shadows and midtones with smaller specular highlights that occupy a limited amount of the screen space. This would lead to the selection of a low to moderate display target nits for the scene that would result in a significant loss of contrast from the original scene as the low display target nits must forcibly compresses the highlights in order to preserve the majority of the scene APL.
To restore some of the local details and local contrast typically lost in HDR tone mapping, madVR applies uses local tone mapping to further optimize the gamma curve. Local tone mapping enhances local contrast and better preserves local details by reducing tone compression for areas of the image that have the most visible pixels and enhancing the contrast of these areas compared to any nearby objects.
madVR calls its approach to local tone mapping Histogram-Shaped Tone Mapping (HSTM). HSTM uses the real-time pixel luminance histogram to prioritize tone compression for areas of the image that have the highest pixel distribution. Because luminance histograms report the distribution of pixels concentrated at a certain luminance and tone (shadow, midtone or highlight), a large number of pixels mapped to the same mastered luminance will create a large hill in the histogram.
Hills that are above the midpoint of the histogram have the most visible pixels and are compressed less to restore some lost brightness, detail and color volume. Hills that are below the midpoint of the histogram chart are compressed more to appear darker and less colorful to offset the expansion of the larger hills on the gamma curve.
Curve expansion and compression from HSTM changes the shape of the gamma curve and will vary from scene-to-scene depending on the unique pixel distribution of each scene. Bright HDR scenes see improvements in contrast through local contrast enhancement. Mixed-brightness and dim HDR scenes see improvements in near black shadow detail through local contrast recovery. The end result is that any of the shadows, midtones or highlights are boosted by HSTM depending on where the bulk of the pixel luminance is distributed in each scene.
What Is Histogram-Shaped Tone Mapping?
*To be added at a later date.
Spears & Munsil UHD HDR Benchmark 10,000 nits MaxCLL (HSTM Off):
Spears & Munsil UHD HDR Benchmark 10,000 nits MaxCLL (HSTM On):
Summary of Primary Pixel Shaders Tone Mapping Image Processing Steps:
1. FALL Algo:
The ideal display target nits is calculated for each scene change as detected by the real-time luminance and hue histograms.
2. PQ BT.2390 Tone Mapping Curve:
madVR calculates a compressed display curve for the scene that matches the selected display target nits by applying a PQ BT.2390 tone mapping curve to roll-off the frame peak nits so the brightest pixels in the scene are mapped to the display target nits.
3. Histogram-Shaped Tone Mapping:
The compressed PQ BT.2390 gamma curve is then optimized using the pixel luminance histogram to improve local detail preservation and enhance contrast. Histogram hills above the midpoint of the graph with the highest concentration of pixels with similar mastered luminance are expanded to be brighter and more colorful, and pixel distributions below the midpoint of the histogram with low concentrations of similar mastered pixels are further compressed to be darker than more visible areas. Depending on the distribution of pixel luminance in each scene, these HSTM adjustments will either lead to improved HDR contrast (local contrast enhancement) or a restoration of detail in the shadows and midtones (local contrast recovery).
4. Recalculate Final Display FALL:
The final HSTM processing result is rescaled in displayed FALL to have the same average on-screen brightness (APL) as the selected display target nits before HSTM tone mapping was applied.
Dynamic Display Target Nits (Continues Below...)
