2018-06-21, 02:16
HDR Tone Mapping Explained (HDR to SDR)
Some of these rules only apply to those outputting HDR content to SDR gamma. HDR curves (output video in HDR format) involve fewer conversions from madVR. The target peak nits also behaves differently: decreasing the target brightness will eventually make the image appear darker, not brighter as with SDR output. The rules for selecting a target display brightness are inherently different.
Topics Covered:
What is HDR Video?
HDR video is video that separates the source into two video ranges: the standard video range you are accustomed to and an extended range reserved for very bright detail. These bright details are used to light scenes with improved contrast and create a better sense of a natural and realistically lit world.
HDR Video Ranges:
Lower Range (0-100 nits):
Known as normal diffuse or SDR reference white.
The majority of an HDR image is located here.
Upper Range (100-10,000 nits):
Known as the HDR specular highlights.
These are hot spots in the image used to convey bright detail.
Example of "HDR hot spots" in a heatmap applied to HDR video games.
HDR highlights are shown in Orange, Red and Purple:
Click Here for Screen Size Image (Original by EvilBoris @resetera.com)
You would see a similar effect if a heatmap was applied to the color gamut: Most colors would fit within the BT.709 color space, while some highly-saturated colors would extend into DCI-P3.
Dynamic Range Compression
HDR specular highlights must be compressed when the end display has a peak brightness that is less than the source peak brightness. Compression of the fixed PQ luminance scale above 100 nits is handled by a roll-off curve that lowers the distance between each PQ step. After compression is applied, the brightest pixels in the source become mapped to the peak white output of the target display.
If a display clips the highlights, they will all be mapped to white and blown out without visible detail. So it is desirable to start the roll-off low enough on the PQ curve to retain all highlight detail at the expense of making everything above the roll-off point darker. Or trade-off some highlight detail for greater brightness.
Displays with a low peak brightness that is only marginally above SDR reference white (0-100 nits — such as most HDR projectors) will typically lower the brightness of reference white to provide space to represent the specular highlights. Tone mapping of reference white in HDR videos is generally not recommended because the majority of the source levels are mastered here.
If reference white must be compressed by the display, it is at least desirable to retain the visible space between each PQ value to avoid flattening large portions of the image. The amount 0-100 nits should be lowered in brightness depends on the source peak brightness and the contrast desired for each movie or each movie scene.
The shape of the roll-off curve is determined by a tone mapping operator (TMO) that uses an S-curve to create different levels of compression that preserve various features in the image. Tone mapping curves simply cannot preserve all image detail, so those deemed most visually important are given priority.
Tone mapping compression starts at a static knee point on the PQ curve:
The placement of the knee point where the roll-off point starts is not standardized across all tone mapping operators. BT.2390 used by madVR chooses the knee point based on a formula that uses several input values as described in Report BT.2390 (largely, inputting the target display peak brightness and comparing it to the measured scene or frame peak brightness to determine how early the roll-off must begin to compress each scene peak to fit the target nits).
Display Target Nits — Adjusting Dynamic Range
The target peak nits defines the size of the container used to compress the HDR video as well as its displayed brightness. A higher target peak nits provides a larger container to compress the source range and a less aggressive roll-off of the specular highlights is used. A lower target peak nits provides a smaller container to compress the source range and a more aggressive roll-off of the specular highlights is necessary.
The choice to use a high or low target peak nits is a balance between retaining as much of the original brightness of the source values as possible versus increasing contrast to retain more of the source's HDR contrast. Increase contrast (high target peak nits) and you sacrifice some brightness — The midtones of the image are rolled-off to improve the contrast between the specular highlights and midtones. Decrease contrast (low target peak nits) and you raise brightness — The majority of the roll-off is applied to the specular highlights with more brightness allocated to the shadows and midtones.
The target peak nits can then be thought of as a dynamic range slider that defines the aggressiveness of the tone curve:
Low target peak nits:
Brighter Images with Lower Contrast = Lower Perceived Dynamic Range.
High target peak nits:
Darker Images with Higher Contrast = Higher Perceived Dynamic Range.
Changes to the target nits impact the Average Picture Level (APL) — The average brightness of the displayed image.
PQ to SDR Gamma Display Mapping — Displayed Luminance
When PQ is converted to SDR gamma, the brightness of each displayed value is determined by a combination of the target peak nits and the real display peak nits of the target display.
Because the target peak nits defines the dynamic range captured by the tone mapping curve, the relative SDR gamma curve will always mirror the precise contrast ratio of the original display target nits at a ratio of of target peak nits/real display peak nits. This occurs through linear gamma compression at the display panel that stretches or compresses the input source values to fit the peak brightness of the display gamma curve.
For example,
target peak nits = real display peak nits
This is a 1:1 compression ratio. No additional source compression happens at the display.
PQ source values follow the original PQ EOTF — the HDR source luminance is faithfully represented at a 1:1 ratio.
target peak nits > real display peak nits
The display target nits is rescaled at the display panel by the relative SDR gamma curve at a compression ratio equal to target peak nits/real display peak nits.
Example: Linear Gamma Rescaling of 100 target peak nits, 200 target peak nits, 300 target peak nits and 400 target peak nits at 100 actual display nits:
100 target peak nits / 100 real display nits = 1:1 display gamma compression ratio;
200 target peak nits / 100 real display nits = 2:1 display gamma compression ratio;
300 target peak nits / 100 real display nits = 3:1 display gamma compression ratio;
400 target peak nits / 100 real display nits = 4:1 display compression ratio.
These ratios mean the brightness of each PQ value will be neutral, 2x darker, 3x darker or 4x darker when displayed. This has the following impact on the APL:
100 target peak nits / 100 real display nits (1:1) = 100 nits reference white;
200 target peak nits / 100 real display nits (2:1) = 50 nits reference white;
300 target peak nits / 100 real display nits (3:1) = 33 nits reference white;
400 target peak nits / 100 real display nits (4:1) = 25 nits reference white.
Lowering reference white in each scene is intended to create contrast between the specular highlights and reference white that is necessary to accurately represent the many bright specular highlights found in HDR movies without clipping them on dimmer displays. For instance, if a movie scene contained highlight information with a mastered peak of 1,000 nits and the display could only produce 100 nits, reference white might be lowered to 50 nits in order to place the brighter specular highlights at 100 nits. This would provide just enough contrast between 100 nits and 50 nits to more faithfully represent the intended contrast of the scene.
As the majority of HDR video is mastered within 0-100 nits, this contrast enhancement has the effect of slightly darkening the overall image. How much compression is required to create the desired HDR effect depends on the mastered peak brightness of each scene in the movie: Higher contrast compression is necessary for very bright scenes, moderate contrast compression is necessary for mid-bright scenes and low contrast compression is suitable for dimmer scenes.
So how are these brightness values communicated by madVR to the end display?
Well, since SDR gamma is relative to brightness of the end display, madVR can assume each PQ value will be rescaled by the end display based on its known SDR peak brightness. The SDR gamma scale assumes every display is 100 nits, but this input can be expanded or compressed linearly by the display. When the calibrated brightness of the display can be assumed, tone mapping can then use the SDR gamma luminance scale as a standard PQ scale that sends luminance values that are equal to what each PQ value will be at the displayed brightness.
Example #1: PQ Values Sent to SDR Display Calibrated to 100 peak nits:
50 nits pixel sent to 100 display nits = 1/2 the total display luminance curve.
Example #2: PQ Values Sent to SDR Display Calibrated to 200 peak nits:
50 nits pixel sent to 200 display nits = 1/4 of the total display luminance curve.
In Example #1, madVR sends an RGB value that is 1/2 of the SDR gamma scale. In Example #2, an RGB value that is 1/4 of the SDR gamma scale is sent to the display.
The luminance scale used by HDR to SDR tone mapping is intended to mirror a true PQ HDR mastering monitor that clips at 100 peak nits, 200 peak nits, 300 peak nits, 400 peak nits, etc. As the target display brightness increases closer to 10,000 nits, RGB values sent by madVR will align closer and closer to a true 10,000 nits reference PQ mastering monitor. For example, R100, G100, B100 eventually equals exactly 0.306 PQ nits rather than a fixed ratio of the SDR display nits.
If target peak nits = real display peak nits, each PQ value is displayed at the exact desired brightness.
If target peak nits > real display peak nits, each PQ value is rendered by the display at lower brightness than originally calculated.
This difference in target nits and display nits has the net effect of shifting all RGB values downwards on the gamma curve. A 50 nits value is displayed at 50 nits at 200 target peak nits on a true 200 nits display, but it is only (2:1) 25 nits on a 100 nits display, and (4:1) 12.5 nits on a true 50 nits display. If the target peak nits is raised high enough above the actual display nits, black crush will occur in places where the visible steps near black become too low in luminance and too close together to show any visible difference between each value. This will make the entire image very dark!
Rescaling HDR source levels towards black at higher target peak nits with the SDR gamma curve is not ideal for maintaining high APLs, but it offers the unique advantage of being able to dynamically tailor displayed contrast to match each movie scene. Each scene in a movie will have a unique frame peak brightness and it may only be necessary to sacrifice some brightness for better contrast if the scene has a high dynamic range that is well above the available display nits.
Contrast adjustments of this nature improve tone mapping flexibility for low peak brightness displays such as projectors that must constantly strike a balance between presenting extended HDR contrast and maintaining a bright and viewable image, all within a very narrow range of available projected brightness.
HDR to SDR Tone Mapping Explained in Images
To compress the PQ HDR luminance scale to fit within the target display brightness, the tone curve must flatten portions of the original PQ scale until the brightest source values are lowered to match the peak white of the target display. PQ values represent fixed luminance and compressing this scale means values at the top of the scale must be stacked on top of the ones below. Lowering the distance between each PQ step has the effect of clipping some detail in the most compressed areas of the roll-off curve where many pixels become mapped to the same PQ value.
Segments of the Absolute 10-bit PQ Luminance Scale up to 1,000 nits:
PQ Value: 0 - Output Luminance: 0 nits
PQ Value: 100 - Output Luminance: 0.306 nits
PQ Value: 200 - Output Luminance: 2.258 nits
PQ Value: 300 - Output Luminance: 9.211 nits
PQ Value: 400 - Output Luminance: 29.386 nits
PQ Value: 500 - Output Luminance: 82.382 nits
PQ Value: 600 - Output Luminance: 214.612 nits
PQ Value: 700 - Output Luminance: 536.845 nits
PQ Value: 769 - Output Luminance: 998.932 nits
Source
You can think of a tone mapping compression like falling blocks in the game Tetris. The lower the target peak nits, the lower tone mapping must shift the entire image downwards.
LINK: TETRIS WORLD CHAMPIONSHIP
If you made the PQ steps perfectly linear, where each step was evenly spaced out with equal increases luminance, tone mapping a 1,000 nit source with a standard BT.2390 roll-off to a display capable of 214 actual nits might look something like this:
Tone mapping to 214 real display nits:
You can see the roll-off starts at 100 nits with the remaining space allocated to mapping the brighter source values. Above the knee point, values are adjusted downwards creating some overlap as some source values become the same value. In reality, it is more likely the source values will become increasingly closer to each other rather than the same value (e.g., from five steps difference to two steps difference), but overlapping PQ values will occur when the display is limited in visible brightness steps.
So what happens if the target nits is 214 nits, but the end display is only capable of 100 nits? Well, the same tone curve is applied, but the entire is image is rescaled by the relative SDR gamma curve. The result is each visible step becomes smaller and closer together and the overall image is dimmed.
214 real display nits rescaled to 100 actual nits:
You can see the individual steps of the larger target are maintained 1:1 at a lower brightness, but the difference in luminance between those steps has been reduced by the rescaling of the SDR gamma curve (each step becomes 2.14x darker). The whole image has been shrunk down to a lower brightness by the gamma curve to maintain the original 1:1 contrast ratio of the larger target peak nits at a 2.14x lower luminance.
Displaying 214 PQ nits at 100 actual display nits lowers reference white from 100 nits to 47 nits. This creates additional space at the top of the range for a more gradual roll-off of the specular highlights that should lead to less visible specular highlight clipping. However, the display rescaling is now less accurate in mapping the absolute luminance of the original PQ source values. Contrast is improved compared to 100 display target nits, but the resulting image uses altered proportions (steps) and displays a visibly lower APL.
*The actual representation of brightness by the SDR gamma curve is not perfectly linear. So this example is only hypothetical.
The accordion-like compression of the gamma curve permits very small adjustments between the source luminance steps that are more difficult when using the original fixed PQ luminance scale. This rescaling occurs while still maintaining some visible distance between most source values.
The placement of the fixed knee point where the roll-off begins is determined by the measured frame peak. Increases to the frame peak nits necessitates a lower roll-off point on the tone curve to accommodate the larger number of source values between reference white (100 nits) and the display target nits to provide a more gentle roll-off of the specular highlights that prevents the highlights from being clipped. As the display target nits increases, the knee point is moved up on the tone curve to alleviate compression of 0-100 nits and use the increased space above 100 nits for the highlight roll-off.
Minimum Display Target Nits Where the BT.2390 Knee Point is Placed at Exactly 100 nits:
Frame Peak: 100 nits -> Target Nits: 100 nits
Frame Peak: 200 nits -> Target Nits: 126 nits
Frame Peak: 300 nits -> Target Nits: 144 nits
Frame Peak: 400 nits -> Target Nits: 158 nits
Frame Peak: 500 nits -> Target Nits: 170 nits
Frame Peak: 600 nits -> Target Nits: 180 nits
Frame Peak: 700 nits -> Target Nits: 190 nits
Frame Peak: 800 nits -> Target Nits: 198 nits
Frame Peak: 900 nits -> Target Nits: 206 nits
Frame Peak: 1000 nits -> Target Nits: 214 nits
Frame Peak: 2000 nits -> Target Nits: 270 nits
Frame Peak: 3000 nits -> Target Nits: 310 nits
Frame Peak: 4000 nits -> Target Nits: 343 nits
Frame Peak: 10000 nits -> Target Nits: 479 nits
HDR to SDR Gamma Compression (Continued...)
What Does Changing the Target Nits Look Like in Action?
I can show you some examples of what changing the display target nits looks like visually in terms of changing the shape of the highlight roll-off and how it alters the contrast of the image.
The sample display is an LED monitor set to SDR output that reaches a peak of approximately 175 real display peak nits.
Note: highlight recovery strength set to high
Avengers - Infinity War: 824 nits frame peak
500 target nits
275 target nits
Here, you can see both the difference in display luminance mapping and the space allocated to the highlight roll-off. The frame is paused so you can clearly see the detail of the bright starburst. A target of 500 display nits visibly lowers the APL of the image, particularly the background elements, but the more gradual roll-off provided by the larger target nits creates additional space to better render the fine detail of the starburst.
At 275 nits, the visible luminance steps are a little too close together at the peak of the roll-off, and as result, some clipping occurs at the display peak where portions of the rays of light lose texture detail, even washing out some of the facial detail of the character in the background.
2001 - A Space Odyssey: 1320 nits frame peak
500 target nits
175 target nits
The second example shows some lost texture detail in a very bright specular reflection. At first glance, the clipping of detail caused by the tone mapping roll-off is more subtle in this example. The frame peaks at 1320 nits, which is far brighter than either of the targets used. If you look at the reflection on the top and bottom of the astronaut's helmet, you can see that it becomes more reflective at 500 nits, like someone applied a buffing cloth to the surface of the helmet. The added depth and contrast provided by the higher target nits allows the small specular reflections to appear more three-dimensional so they appear brighter than the rest of the helmet.
The other hint of clipping caused by tone mapping compression is the bright green hallway light in the background that is blown out at 175 nits and restored at 500 nits.
Mission Impossible - Ghost Protocol: 914 nits frame peak
450 target nits
175 target nits
The third example shows how changing the display target nits can help to shift the image from a flat SDR presentation to more of a high dynamic range presentation. This scene in a dimly-lit garage features a single light source (a beam of bright sunlight) focused on a small portion of the image that peaks at 914 nits. The 1:1 representation of this scene at 175 nits with a neutral reference white makes the scene appear somewhat lifeless and SDR due to some obvious clipping of the specular highlights and the lack of contrast between the bright sunbeams and reference white. This makes the "bad guy" look somewhat less menacing and thoughtful.
Increasing the display target nits to 500 nits provides some noticeable improvement in the visible detail of the background elements, especially the foreign text on the crate in the back of the frame and the cardboard box placed to the left of the crate. But the higher target nits also noticeably changes the contrast of the image similar to how changing the exposure on a camera is used to capture additional dynamic range.
The 2.86:1 ratio of display target nits to real display peak nits lowers reference white from 100 nits to approximately 35 nits, which causes some shadows to creep in on the face of the bad guy that are likely not inherent to the source footage. But the perceived increase in dynamic range better recreates the intended contrast of the scene by tightly focusing the rays of sunlight on the nearby canvas with less clipping and adding some contrast between the sunbeams and dimly-lit garage. This makes the bad guy appear a little more menacing and better uses the specular highlights as the director intended them to be used.
What Happens If the Source Has a Low Dynamic Range?
Blade Runner 2049: 116 nits frame peak
175 target nits
425 target nits
The intent of the target display value is to preserve HDR contrast. If the source happens to have a low dynamic range, this contrast can be unnecessary and will only serve to create a dark image.
Blade Runner 2049 is a classic example of an HDR movie with a low dynamic range. The majority of the presentation stays well under 200 nits. Here, high target nits simply cause the image to lose its vibrancy and colorfulness with no gain in true contrast. In this example, 425 target nits at 175 actual display nits appears as a mismatch in gamma that leaves the presentation looking dull and dark.
Closely mapping the target nits to the display will better preserve the brightness and color vibrancy of the source. You don't ever want to set this value below the peak brightness of your display because the image will be mapped with incorrect brightness and end up too bright and washed out.
This example also highlights how color accuracy can be distorted by using a high target nits. Tone mapping is designed to desaturate all colors as they lose luminance. The greater the mismatch between the target brightness and actual display brightness, the more oversaturated and dark color tones will appear, as all colors are displayed at a darker luminance than calculated by the tone mapping algorithm.
Summary
Because HDR content is about high dynamic range, it makes the most sense to preserve the HDR contrast of the source as much as possible. Sources with a lower dynamic range can use lower target nits to maintain the source brightness without having a negative impact on perceived contrast or detail. However, sources with a higher dynamic range require a choice between lowering the brightness of the source to represent its high contrast at a lower displayed brightness, or more accurately mapping the source brightness at the expense of clipping some bright detail through range compression.
When your display has limited luminance, using values for the target peak nits that are above the actual brightness of your display is acceptable to improve the HDR presentation. How far you go above the display peak is a matter of taste and your willingness to lower the APL of the source in order to increase contrast.
Some of these rules only apply to those outputting HDR content to SDR gamma. HDR curves (output video in HDR format) involve fewer conversions from madVR. The target peak nits also behaves differently: decreasing the target brightness will eventually make the image appear darker, not brighter as with SDR output. The rules for selecting a target display brightness are inherently different.
Topics Covered:
- What is HDR Video;
- Dynamic Range Compression;
- Mapping of PQ to Display Brightness;
- HDR to SDR Tone Mapping Explained.
What is HDR Video?
HDR video is video that separates the source into two video ranges: the standard video range you are accustomed to and an extended range reserved for very bright detail. These bright details are used to light scenes with improved contrast and create a better sense of a natural and realistically lit world.
HDR Video Ranges:
Lower Range (0-100 nits):
Known as normal diffuse or SDR reference white.
The majority of an HDR image is located here.
Upper Range (100-10,000 nits):
Known as the HDR specular highlights.
These are hot spots in the image used to convey bright detail.
Example of "HDR hot spots" in a heatmap applied to HDR video games.
HDR highlights are shown in Orange, Red and Purple:
Click Here for Screen Size Image (Original by EvilBoris @resetera.com)
You would see a similar effect if a heatmap was applied to the color gamut: Most colors would fit within the BT.709 color space, while some highly-saturated colors would extend into DCI-P3.
Dynamic Range Compression
HDR specular highlights must be compressed when the end display has a peak brightness that is less than the source peak brightness. Compression of the fixed PQ luminance scale above 100 nits is handled by a roll-off curve that lowers the distance between each PQ step. After compression is applied, the brightest pixels in the source become mapped to the peak white output of the target display.
If a display clips the highlights, they will all be mapped to white and blown out without visible detail. So it is desirable to start the roll-off low enough on the PQ curve to retain all highlight detail at the expense of making everything above the roll-off point darker. Or trade-off some highlight detail for greater brightness.
Displays with a low peak brightness that is only marginally above SDR reference white (0-100 nits — such as most HDR projectors) will typically lower the brightness of reference white to provide space to represent the specular highlights. Tone mapping of reference white in HDR videos is generally not recommended because the majority of the source levels are mastered here.
If reference white must be compressed by the display, it is at least desirable to retain the visible space between each PQ value to avoid flattening large portions of the image. The amount 0-100 nits should be lowered in brightness depends on the source peak brightness and the contrast desired for each movie or each movie scene.
The shape of the roll-off curve is determined by a tone mapping operator (TMO) that uses an S-curve to create different levels of compression that preserve various features in the image. Tone mapping curves simply cannot preserve all image detail, so those deemed most visually important are given priority.
Tone mapping compression starts at a static knee point on the PQ curve:
The placement of the knee point where the roll-off point starts is not standardized across all tone mapping operators. BT.2390 used by madVR chooses the knee point based on a formula that uses several input values as described in Report BT.2390 (largely, inputting the target display peak brightness and comparing it to the measured scene or frame peak brightness to determine how early the roll-off must begin to compress each scene peak to fit the target nits).
Display Target Nits — Adjusting Dynamic Range
The target peak nits defines the size of the container used to compress the HDR video as well as its displayed brightness. A higher target peak nits provides a larger container to compress the source range and a less aggressive roll-off of the specular highlights is used. A lower target peak nits provides a smaller container to compress the source range and a more aggressive roll-off of the specular highlights is necessary.
The choice to use a high or low target peak nits is a balance between retaining as much of the original brightness of the source values as possible versus increasing contrast to retain more of the source's HDR contrast. Increase contrast (high target peak nits) and you sacrifice some brightness — The midtones of the image are rolled-off to improve the contrast between the specular highlights and midtones. Decrease contrast (low target peak nits) and you raise brightness — The majority of the roll-off is applied to the specular highlights with more brightness allocated to the shadows and midtones.
The target peak nits can then be thought of as a dynamic range slider that defines the aggressiveness of the tone curve:
Low target peak nits:
Brighter Images with Lower Contrast = Lower Perceived Dynamic Range.
High target peak nits:
Darker Images with Higher Contrast = Higher Perceived Dynamic Range.
Changes to the target nits impact the Average Picture Level (APL) — The average brightness of the displayed image.
PQ to SDR Gamma Display Mapping — Displayed Luminance
When PQ is converted to SDR gamma, the brightness of each displayed value is determined by a combination of the target peak nits and the real display peak nits of the target display.
Because the target peak nits defines the dynamic range captured by the tone mapping curve, the relative SDR gamma curve will always mirror the precise contrast ratio of the original display target nits at a ratio of of target peak nits/real display peak nits. This occurs through linear gamma compression at the display panel that stretches or compresses the input source values to fit the peak brightness of the display gamma curve.
For example,
target peak nits = real display peak nits
This is a 1:1 compression ratio. No additional source compression happens at the display.
PQ source values follow the original PQ EOTF — the HDR source luminance is faithfully represented at a 1:1 ratio.
target peak nits > real display peak nits
The display target nits is rescaled at the display panel by the relative SDR gamma curve at a compression ratio equal to target peak nits/real display peak nits.
Example: Linear Gamma Rescaling of 100 target peak nits, 200 target peak nits, 300 target peak nits and 400 target peak nits at 100 actual display nits:
100 target peak nits / 100 real display nits = 1:1 display gamma compression ratio;
200 target peak nits / 100 real display nits = 2:1 display gamma compression ratio;
300 target peak nits / 100 real display nits = 3:1 display gamma compression ratio;
400 target peak nits / 100 real display nits = 4:1 display compression ratio.
These ratios mean the brightness of each PQ value will be neutral, 2x darker, 3x darker or 4x darker when displayed. This has the following impact on the APL:
100 target peak nits / 100 real display nits (1:1) = 100 nits reference white;
200 target peak nits / 100 real display nits (2:1) = 50 nits reference white;
300 target peak nits / 100 real display nits (3:1) = 33 nits reference white;
400 target peak nits / 100 real display nits (4:1) = 25 nits reference white.
Lowering reference white in each scene is intended to create contrast between the specular highlights and reference white that is necessary to accurately represent the many bright specular highlights found in HDR movies without clipping them on dimmer displays. For instance, if a movie scene contained highlight information with a mastered peak of 1,000 nits and the display could only produce 100 nits, reference white might be lowered to 50 nits in order to place the brighter specular highlights at 100 nits. This would provide just enough contrast between 100 nits and 50 nits to more faithfully represent the intended contrast of the scene.
As the majority of HDR video is mastered within 0-100 nits, this contrast enhancement has the effect of slightly darkening the overall image. How much compression is required to create the desired HDR effect depends on the mastered peak brightness of each scene in the movie: Higher contrast compression is necessary for very bright scenes, moderate contrast compression is necessary for mid-bright scenes and low contrast compression is suitable for dimmer scenes.
So how are these brightness values communicated by madVR to the end display?
Well, since SDR gamma is relative to brightness of the end display, madVR can assume each PQ value will be rescaled by the end display based on its known SDR peak brightness. The SDR gamma scale assumes every display is 100 nits, but this input can be expanded or compressed linearly by the display. When the calibrated brightness of the display can be assumed, tone mapping can then use the SDR gamma luminance scale as a standard PQ scale that sends luminance values that are equal to what each PQ value will be at the displayed brightness.
Example #1: PQ Values Sent to SDR Display Calibrated to 100 peak nits:
50 nits pixel sent to 100 display nits = 1/2 the total display luminance curve.
Example #2: PQ Values Sent to SDR Display Calibrated to 200 peak nits:
50 nits pixel sent to 200 display nits = 1/4 of the total display luminance curve.
In Example #1, madVR sends an RGB value that is 1/2 of the SDR gamma scale. In Example #2, an RGB value that is 1/4 of the SDR gamma scale is sent to the display.
The luminance scale used by HDR to SDR tone mapping is intended to mirror a true PQ HDR mastering monitor that clips at 100 peak nits, 200 peak nits, 300 peak nits, 400 peak nits, etc. As the target display brightness increases closer to 10,000 nits, RGB values sent by madVR will align closer and closer to a true 10,000 nits reference PQ mastering monitor. For example, R100, G100, B100 eventually equals exactly 0.306 PQ nits rather than a fixed ratio of the SDR display nits.
If target peak nits = real display peak nits, each PQ value is displayed at the exact desired brightness.
If target peak nits > real display peak nits, each PQ value is rendered by the display at lower brightness than originally calculated.
This difference in target nits and display nits has the net effect of shifting all RGB values downwards on the gamma curve. A 50 nits value is displayed at 50 nits at 200 target peak nits on a true 200 nits display, but it is only (2:1) 25 nits on a 100 nits display, and (4:1) 12.5 nits on a true 50 nits display. If the target peak nits is raised high enough above the actual display nits, black crush will occur in places where the visible steps near black become too low in luminance and too close together to show any visible difference between each value. This will make the entire image very dark!
Rescaling HDR source levels towards black at higher target peak nits with the SDR gamma curve is not ideal for maintaining high APLs, but it offers the unique advantage of being able to dynamically tailor displayed contrast to match each movie scene. Each scene in a movie will have a unique frame peak brightness and it may only be necessary to sacrifice some brightness for better contrast if the scene has a high dynamic range that is well above the available display nits.
Contrast adjustments of this nature improve tone mapping flexibility for low peak brightness displays such as projectors that must constantly strike a balance between presenting extended HDR contrast and maintaining a bright and viewable image, all within a very narrow range of available projected brightness.
HDR to SDR Tone Mapping Explained in Images
To compress the PQ HDR luminance scale to fit within the target display brightness, the tone curve must flatten portions of the original PQ scale until the brightest source values are lowered to match the peak white of the target display. PQ values represent fixed luminance and compressing this scale means values at the top of the scale must be stacked on top of the ones below. Lowering the distance between each PQ step has the effect of clipping some detail in the most compressed areas of the roll-off curve where many pixels become mapped to the same PQ value.
Segments of the Absolute 10-bit PQ Luminance Scale up to 1,000 nits:
PQ Value: 0 - Output Luminance: 0 nits
PQ Value: 100 - Output Luminance: 0.306 nits
PQ Value: 200 - Output Luminance: 2.258 nits
PQ Value: 300 - Output Luminance: 9.211 nits
PQ Value: 400 - Output Luminance: 29.386 nits
PQ Value: 500 - Output Luminance: 82.382 nits
PQ Value: 600 - Output Luminance: 214.612 nits
PQ Value: 700 - Output Luminance: 536.845 nits
PQ Value: 769 - Output Luminance: 998.932 nits
Source
You can think of a tone mapping compression like falling blocks in the game Tetris. The lower the target peak nits, the lower tone mapping must shift the entire image downwards.
LINK: TETRIS WORLD CHAMPIONSHIP
If you made the PQ steps perfectly linear, where each step was evenly spaced out with equal increases luminance, tone mapping a 1,000 nit source with a standard BT.2390 roll-off to a display capable of 214 actual nits might look something like this:
Tone mapping to 214 real display nits:
You can see the roll-off starts at 100 nits with the remaining space allocated to mapping the brighter source values. Above the knee point, values are adjusted downwards creating some overlap as some source values become the same value. In reality, it is more likely the source values will become increasingly closer to each other rather than the same value (e.g., from five steps difference to two steps difference), but overlapping PQ values will occur when the display is limited in visible brightness steps.
So what happens if the target nits is 214 nits, but the end display is only capable of 100 nits? Well, the same tone curve is applied, but the entire is image is rescaled by the relative SDR gamma curve. The result is each visible step becomes smaller and closer together and the overall image is dimmed.
214 real display nits rescaled to 100 actual nits:
You can see the individual steps of the larger target are maintained 1:1 at a lower brightness, but the difference in luminance between those steps has been reduced by the rescaling of the SDR gamma curve (each step becomes 2.14x darker). The whole image has been shrunk down to a lower brightness by the gamma curve to maintain the original 1:1 contrast ratio of the larger target peak nits at a 2.14x lower luminance.
Displaying 214 PQ nits at 100 actual display nits lowers reference white from 100 nits to 47 nits. This creates additional space at the top of the range for a more gradual roll-off of the specular highlights that should lead to less visible specular highlight clipping. However, the display rescaling is now less accurate in mapping the absolute luminance of the original PQ source values. Contrast is improved compared to 100 display target nits, but the resulting image uses altered proportions (steps) and displays a visibly lower APL.
*The actual representation of brightness by the SDR gamma curve is not perfectly linear. So this example is only hypothetical.
The accordion-like compression of the gamma curve permits very small adjustments between the source luminance steps that are more difficult when using the original fixed PQ luminance scale. This rescaling occurs while still maintaining some visible distance between most source values.
The placement of the fixed knee point where the roll-off begins is determined by the measured frame peak. Increases to the frame peak nits necessitates a lower roll-off point on the tone curve to accommodate the larger number of source values between reference white (100 nits) and the display target nits to provide a more gentle roll-off of the specular highlights that prevents the highlights from being clipped. As the display target nits increases, the knee point is moved up on the tone curve to alleviate compression of 0-100 nits and use the increased space above 100 nits for the highlight roll-off.
Minimum Display Target Nits Where the BT.2390 Knee Point is Placed at Exactly 100 nits:
Frame Peak: 100 nits -> Target Nits: 100 nits
Frame Peak: 200 nits -> Target Nits: 126 nits
Frame Peak: 300 nits -> Target Nits: 144 nits
Frame Peak: 400 nits -> Target Nits: 158 nits
Frame Peak: 500 nits -> Target Nits: 170 nits
Frame Peak: 600 nits -> Target Nits: 180 nits
Frame Peak: 700 nits -> Target Nits: 190 nits
Frame Peak: 800 nits -> Target Nits: 198 nits
Frame Peak: 900 nits -> Target Nits: 206 nits
Frame Peak: 1000 nits -> Target Nits: 214 nits
Frame Peak: 2000 nits -> Target Nits: 270 nits
Frame Peak: 3000 nits -> Target Nits: 310 nits
Frame Peak: 4000 nits -> Target Nits: 343 nits
Frame Peak: 10000 nits -> Target Nits: 479 nits
HDR to SDR Gamma Compression (Continued...)
What Does Changing the Target Nits Look Like in Action?
I can show you some examples of what changing the display target nits looks like visually in terms of changing the shape of the highlight roll-off and how it alters the contrast of the image.
The sample display is an LED monitor set to SDR output that reaches a peak of approximately 175 real display peak nits.
Note: highlight recovery strength set to high
Avengers - Infinity War: 824 nits frame peak
500 target nits
275 target nits
Here, you can see both the difference in display luminance mapping and the space allocated to the highlight roll-off. The frame is paused so you can clearly see the detail of the bright starburst. A target of 500 display nits visibly lowers the APL of the image, particularly the background elements, but the more gradual roll-off provided by the larger target nits creates additional space to better render the fine detail of the starburst.
At 275 nits, the visible luminance steps are a little too close together at the peak of the roll-off, and as result, some clipping occurs at the display peak where portions of the rays of light lose texture detail, even washing out some of the facial detail of the character in the background.
2001 - A Space Odyssey: 1320 nits frame peak
500 target nits
175 target nits
The second example shows some lost texture detail in a very bright specular reflection. At first glance, the clipping of detail caused by the tone mapping roll-off is more subtle in this example. The frame peaks at 1320 nits, which is far brighter than either of the targets used. If you look at the reflection on the top and bottom of the astronaut's helmet, you can see that it becomes more reflective at 500 nits, like someone applied a buffing cloth to the surface of the helmet. The added depth and contrast provided by the higher target nits allows the small specular reflections to appear more three-dimensional so they appear brighter than the rest of the helmet.
The other hint of clipping caused by tone mapping compression is the bright green hallway light in the background that is blown out at 175 nits and restored at 500 nits.
Mission Impossible - Ghost Protocol: 914 nits frame peak
450 target nits
175 target nits
The third example shows how changing the display target nits can help to shift the image from a flat SDR presentation to more of a high dynamic range presentation. This scene in a dimly-lit garage features a single light source (a beam of bright sunlight) focused on a small portion of the image that peaks at 914 nits. The 1:1 representation of this scene at 175 nits with a neutral reference white makes the scene appear somewhat lifeless and SDR due to some obvious clipping of the specular highlights and the lack of contrast between the bright sunbeams and reference white. This makes the "bad guy" look somewhat less menacing and thoughtful.
Increasing the display target nits to 500 nits provides some noticeable improvement in the visible detail of the background elements, especially the foreign text on the crate in the back of the frame and the cardboard box placed to the left of the crate. But the higher target nits also noticeably changes the contrast of the image similar to how changing the exposure on a camera is used to capture additional dynamic range.
The 2.86:1 ratio of display target nits to real display peak nits lowers reference white from 100 nits to approximately 35 nits, which causes some shadows to creep in on the face of the bad guy that are likely not inherent to the source footage. But the perceived increase in dynamic range better recreates the intended contrast of the scene by tightly focusing the rays of sunlight on the nearby canvas with less clipping and adding some contrast between the sunbeams and dimly-lit garage. This makes the bad guy appear a little more menacing and better uses the specular highlights as the director intended them to be used.
What Happens If the Source Has a Low Dynamic Range?
Blade Runner 2049: 116 nits frame peak
175 target nits
425 target nits
The intent of the target display value is to preserve HDR contrast. If the source happens to have a low dynamic range, this contrast can be unnecessary and will only serve to create a dark image.
Blade Runner 2049 is a classic example of an HDR movie with a low dynamic range. The majority of the presentation stays well under 200 nits. Here, high target nits simply cause the image to lose its vibrancy and colorfulness with no gain in true contrast. In this example, 425 target nits at 175 actual display nits appears as a mismatch in gamma that leaves the presentation looking dull and dark.
Closely mapping the target nits to the display will better preserve the brightness and color vibrancy of the source. You don't ever want to set this value below the peak brightness of your display because the image will be mapped with incorrect brightness and end up too bright and washed out.
This example also highlights how color accuracy can be distorted by using a high target nits. Tone mapping is designed to desaturate all colors as they lose luminance. The greater the mismatch between the target brightness and actual display brightness, the more oversaturated and dark color tones will appear, as all colors are displayed at a darker luminance than calculated by the tone mapping algorithm.
Summary
Because HDR content is about high dynamic range, it makes the most sense to preserve the HDR contrast of the source as much as possible. Sources with a lower dynamic range can use lower target nits to maintain the source brightness without having a negative impact on perceived contrast or detail. However, sources with a higher dynamic range require a choice between lowering the brightness of the source to represent its high contrast at a lower displayed brightness, or more accurately mapping the source brightness at the expense of clipping some bright detail through range compression.
When your display has limited luminance, using values for the target peak nits that are above the actual brightness of your display is acceptable to improve the HDR presentation. How far you go above the display peak is a matter of taste and your willingness to lower the APL of the source in order to increase contrast.
