"Neutral" tonemapping darkens the image considerably

[hybridherbst]

Description

For one of our viewers, I was using an earlier version of Neutral / Khronos PBR Tonemapper as it was in use by model-viewer. In that version, there was no/almost no darkening on surfaces compared to Linear/None, and only a curve in the highlights to prevent clipping, to the best of my understanding.

To my surprise, switching to the current version of "Neutral" now darkens the image considerably – which doesn't feel intended, and is different to before. Darker areas are now so dark that it loses detail entirely compared to the Linear look (see for example the exhaust area in the pictures below, or the rubber pipes in the back).

I see the same behavior in both three.js and model-viewer, so I think raising the issue here as the "source" of the tonemapper is appropriate.

Long story short – is this intended? Does the shipped version of the Khronos PBR Tonemapper now resemble ACES so much more than previous in-progress versions? To me, that feels like a step back – previously my understanding was that Neutral is designed to keep albedo data as much as possible, but now it feels the goal is to be as close as possible to ACES without having the same hue shifts?

Live Demo – Damaged Helmet

• Open https://modelviewer.dev/editor
• Load DamagedHelmet (DamagedHelmet (3).glb.zip)
• Load Venice Sunrise for comparison (venice_sunrise_1k.hdr.zip)
• Open https://threejs.org/examples/webgl_tonemapping.html in parallel

Alternative reproduction: Neutral also darkens "Damaged Helmet" considerably on https://tone-mapping.glitch.me/.

Tone Mapping	model-viewer	three.js
Linear
Neutral
ACES
Base Color Map (Reference)

Since the model-viewer editor doesn't support switching to None/Linear for comparison (#4541), you can paste and run this code snippet in the chrome console to switch to Linear:

const mv = document.querySelector("model-viewer-preview").shadowRoot.querySelector("model-viewer");
const sc = mv[Object.getOwnPropertySymbols(mv).find((x) => x.description === "scene")];
sc.toneMapping = 1;
sc.queueRender();

Live Demo – Caterpillar Work Boot

Here's another real-world scanned PBR model that fares even worse with Neutral tonemapping – I'd say Neutral looks the worst of all tonemapping options, heavily deviating from the intended look of the model:

Tone Mapping	Rendering
Base Color (Map * Factor) Reference
Linear
Neutral
AgX
ACES

The asset: https://cloud.needle.tools/api/v1/public/8ab475a/8ab475a/caterpillar_work_boot.metal.glb (from https://sketchfab.com/3d-models/caterpillar-work-boot-d551ce74dcd24528a05cbb0f4b7434d7 with converted pbrSpecularGlossiness in gltf-transform)

Version

latest

[elalish]

Indeed, this was an intentional change and was included both to get increased color accuracy and to address the feedback that models were looking "washed out". The key was realizing that Linear tone mapping does not actually preserve baseColors: PBR itself desaturates them by 4% on average. A correction for this was added to PBR Neutral so that baseColors can come through to the final render as unaltered as possible - find more details here.

Of course, if you're taking Linear as "correct" (perhaps because the model was designed while looking at Linear output), then PBR Neutral will look dark. However, if you choose your sRGB base colors based on how the product should look on screen, then PBR Neutral will look correct and Linear will appear washed out.

A side effect is that PBR Neutral now has a contrast toe like most Filmic tone mappers (including ACES), which gives deeper blacks and makes it easier to transition ACES-designed models to PBR Neutral without updating the GLB (which is pretty common). You'll notice with Linear that you can't get a matte, dielectric black darker than about [30, 30, 30] without adding AO or reducing lighting.

[hybridherbst]

Thanks a lot for the additional insights!

Doing some more tests, I think I can see what you mean with Linear being washed out when compared strictly to the
base color map, or when using the Furnace HDR., But I don't think Neutral comes closer to sRGB base colors than Linear under actual lighting conditions (right column):

At least to my eyes, Neutral looks oversatured and darkened compared to the base color reference. Am I missing something?

Rendering Type	Reference	Furnace HDR	model-viewer Neutral HDR
Base Color Map * Base Color Factor
3D: Base Color Map * Base Color Factor, Unlit, Linear
3D: PBR Linear
3D: PBR Neutral
3D: PBR AgX

A side note, I would argue that

You'll notice with Linear that you can't get a matte, dielectric black darker than about [30, 30, 30]

is very much by design for the entire concept of PBR rendering – a shadow cast onto a black matte object needs to be visibly darker than the object, same as in the "real world".

[hybridherbst]

You're saying that PBR Linear looks more like the reference than PBR Neutral?

Yes, indeed.

Here's unlit vs. neutral vs. linear copied from above –
In my eyes, linear matches albedo map much better than "Neutral" under regular lighting conditions (not furnace). "Neutral" oversaturates and darkens here, to the point of this becoming a different product, it becomes much more "yellow".

Unlit (Base Color Map * Factor)	Neutral	Linear

Here's an image with artifical brightness=3 to see that the left and right have a closer match than the middle one:

And here's a histogram – note the stronger blue-green separation on the "Neutral" image:

[elalish]

Thanks, I do see what you mean, though I don't completely agree. Still, it's worth looking into. First, how exactly are you doing these comparisons? It's slightly tricky with <model-viewer> because ACES has a different effective exposure than Linear/Neutral do, noted here:

For an apples-to-apples comparison of ACES to Neutral with custom lighting, set the Neutral exposure to 1 and the ACES exposure to 0.77 to account for ACES being artificially bright. This compensation is automatic for our built-in lighting.

Since tone mapping is non-linear, brightness applied before and after are not equivalent.

Actually, I found a more serious issue - this boot is using the approximate conversion from spec gloss to metal rough, so it includes a specular factor and a non-physical IoR, which means the base color isn't the right reference on its own. In fact, I would bet that IoR = 1000 is the main reason for the discrepancy - we adjust for the average PBR desaturation of 4% based on IoR = 1.5. With 1000, that desaturation is gone and our compensation is going too far.

I kind of hate that automatic spec-gloss conversion - technically it's the right way if you're actually doing something colorful with the specular channel, but 99% of the time it's just gray, so simply ditching the specular factor entirely and not touching IoR makes a more sensible PBR model.

[hybridherbst]

The file in question indeed has relevant color information in the specular channel, this data is not hand-authored but the result of a PBR scanning pipeline. Here's only the specular channel:

I might not know enough about the spec-gloss conversion (this particular file was converted with gltf-transform); my understanding was that KHR_materials_specular is a 100% replacement for the deprecated KHR_materials_pbrSpecularGlossiness the file used originally.

The extension notes that IOR should be set to 0, not 1000: https://github.com/KhronosGroup/glTF/blob/main/extensions/2.0/Khronos/KHR_materials_specular/README.md#specular-glossiness-materials

But in practice that doesn't seem to make a difference between 0.0 and 1000.0, at 1.5 (just tested out of curiosity) Neutral tonemapping becomes even darker.

IOR	Result with Linear	Result with Neutral
1000.0
0.0
1.5

[elalish]

My point here is that there are currently a bunch of variables. PBR Neutral is darker than Linear intentionally - Linear cannot be considered a reference. The baseColor texture can be, but only if things are not too complicated (by things like specular and ior extensions). Can you choose or create a similar model that is baseline metallic-roughness so we can be precise about the differences?

[hybridherbst]

I'll try to find/make a better model.