[css-images-3] Allow dithering as a gradient-painting strategy

[tabatkins]

Currently, the Images spec strictly defines the color of each pixel in a gradient - the gradient-line is colored according to the defined interpolation between color stops, and then the image is painted according to what point on the gradient line a given pixel is closest to.

However, WebKit apparently dithers its gradients to produce a smoother overall appearance. I think this should be allowed! Random dithering is a great rendering strategy for making gradients look even smoother, especially when the endpoints are fairly close in color-space; it avoids the hard line where the color shifts up or down one channel-unit, which can be pretty visible in some circumstances.

So I propose that we add some normative text to allow dithering as a rendering strategy. I don't think we need to put in any strict requirements on how far from ideal the dithering is allowed to be; we can leave that up to QoI and just rely on reasonable fuzzy-matching for reftests.

[frivoal]

In theory, I agree. In practice, how do we write tests for gradients after that? Is fuzzy matching of ref-tests actually workable here?

I don't think we need to enforce the quality of the dithering, but I do thing it'd be unfortunate if the flexibility introduced to allow dithering made it impossible to test the color math at all, and we could no longer reliably test for (for example) whether a gradient is being computed in sRGB vs Lab.

[tabatkins]

Maybe we can follow the lead of the box-shadow spec and say that it must be within 5% of the ideal model described by the spec?

[frivoal]

I guess that leaves room for low quality implementations as well as nice dithering, without really allowing for outright wrong. 5% is kind of arbitrary, but why not.

[tabatkins]

It was arbitrary for box-shadow too, but eh.

And yes, allowing for low-quality impls is fine; the whole point is to provide a better-quality image, and so if you're screwing that up that's your problem.

[svgeesus]

In practice, how do we write tests for gradients after that?

In practice, we already don't write reftests for gradients, and there is no API to sample the color at a given point because security/privacy

[svgeesus]

Dithering on gradients is a long-established and common practice. It should not be seen as low quality (although it can be used on low quality devices, such as those with effectively 6 bits per component). Rather, a small amount of noise added to the gradient helps prevent mach banding.

[svgeesus]

5% of what? the RGB values? Those aren't uniform; 5% at the dark parts of the curve are a lot, while 5% in the light parts is a lot less visible.

If we want to legislate on allowed differences of color, there are already specifications for that. Delta-E (several formulae of varying complexity and accuracy, the simplest being Euclidean distance in Lab/LCH) already predicts just noticeable differences.

[tabatkins]

While 5% (distance in RGB) is a lot and non-uniform, (1) it should be well more than any reasonable dithering strategy causes, and (2) we already use the exact same spec limitation to define how shadow blur works, by defining the "ideal" blur as a gaussian and allowing 5% deviation from that (which admits reasonable box-blurs, but not things that are way off), and shadows cover the full spectrum of dark to light.

[AmeliaBR]

we already don't write reftests for gradients

Maybe that's a policy we should change, especially if we introduce features like alternative interpolation spaces? I assume it's that's way because we didn't have a good standard for measuring a “good enough” gradient in a fuzzy reftest.

5% of what?

Ideally: 5% of the change in color along the gradient vector. E.g., represent the color at this point as a vector in the correct color-space, then the absolute distance/magnitude between that value and colors at the nearest color stops should be ±5% of the mathematical value.

That means, no dithering that is more extreme than the actual change in color stops, no dithering at all in solid stripe gradients (5% of 0 is 0 allowed deviation), but on the other hand, some variation in the “direction” of the interpolation is allowed so long as the magnitude change is reasonable (e.g., a gradient between grays could use “pseudogray” effects: very, very unsaturated colors to create finer progressions of luminance).

[AmeliaBR]

The allowed range would actually need to be expressed something like [floor(ideal - 5%), ceil(ideal + 5%)], where floor/ceil are applied per display color channel, extending to the next value the display can represent. The whole point of dithering is to soften changes between the color values that the display can actually represent.

[faceless2]

I'd actually been meaning to write up some tests for gradients to test the decision (edit: actually it's still merely a suggestion) in #4647, that gradients interpolate in the colorspace of their components if they're all the same, and Lab otherwise.

The approach I had in mind was this: https://jsbin.com/nowiwil/1/edit?html,css,output

Broadly: with 8-bit color, a wide enough gradient and no dithering, the mid-point of the gradient is effectively a solid block of color. Of course if dithering is introduced this approach won't be possible.

(aside: we use simple RGB-differencing when we compare our renderings against the reference images in our test harness. We did try switching to Delta-E, but the slowdown was absurd - a factor of 10 or so. No doubt things would be better in C, but there's bound to be a cost if you require Delta-E for differences)

[css-meeting-bot]

The CSS Working Group just discussed Allow dithering as a gradient-painting strategy, and agreed to the following:

• RESOLVED: Add this to Images 3 and 4

The full IRC log of that discussion

<dael> topic: Allow dithering as a gradient-painting strategy
<dael> github: https://github.com//issues/4793
<dael> TabAtkins: webkit uses dithering to make gradients better. This is great. Spec does not allow and dictates every pixel. Technically wk violates but I think spec should allow in both css and canvas
<smfr> (and svg gradients)
<chris> q+
<dino> Related PR on WPT - https://github.com/web-platform-tests/wpt/pull/21647
<dael> TabAtkins: If no one objects I'd like to add a few paragraphs to images to allow dithering. Not sure if you want to say yeah you can dither or if we want to put some restrictions where you should be close like we do for box shodow.
<dbaron> A downside of this means it becomes even more difficult to reftest gradients...
<dael> bkardell_: If it displays and you only have 256 colros no dithering?
<dael> TabAtkins: css doesn't recognize less the 24 bit color. Otherwise no it does define what the display color should be
<dael> dino: CSS spec defines theoretical color. If you display on black and white that's not css problm
<dael> dino: I referenced PR where i changed test to allow dither. It's 2 out of 256 slots can be different in any channel. In practice our dithering is in that range an not all bands.
<bradk> That was me talking, not Brian
<dael> dino: Reason we do this is b/c it does make gradients look better.
<TabAtkins> s/bkardell_ /bradk/
<AmeliaBR> q?
<dino> ack dino
<dael> Rossen_: Let's see if we can move to resolve
<dauwhe> q?
<dael> Rossen_: Sorry chris I didn't see the queue
<bradk> Sorry
<Rossen_> ack chris
<dael> chris: I agree we should allow this. Surprised reading of spec means disallowed. This is wide practice where you need to do things like dither. FOr testing we need to sample on small area so it averages and youg et mean result. I'd be astonished if we disallow
<dael> leaverou: Shouldn't disallow better visual result for testing
<fantasai> +1 to leaverou
<dael> TabAtkins: No one is arguing don't allow diteher. I defined what the color of a gradient should be and didn't think about jiggering it around.
<smfr> you can still do reference testing with some SVG filter stuff to batch the pixel values
<Rossen_> s/jiggering/dithering/
<dael> chris: Disagreement I have is 5%, what does that mean. 2 counts in rgb is noticable in some places. I was suggesting delta-e, maybe that's hard to do. 5% is meaningless.
<dael> TabAtkins: I don't care that much, you won't dither badly. It's what boundries can we put on tests to pass a good impl and not pass a bad one. We did 5% on shadows b/c that lets you say here's the ideal and get close to it. I was going for same deal where it detects when you've gone widely off
<dael> AmeliaBR: Do we have useful reftests for shadow blurring or is that arbitrary rule in spec?
<dael> TabAtkins: Don't know
<dael> dbaron: At least one test which I wrote
<bradk> Are monitors with larger than normal color gambits not allowed to display more colors in the gradients?
<dael> chris: In general hardly have tests for gradients b/c hard to figure out how to do automatic reftest
<fantasai> Here is dbaron's test http://test.csswg.org/suites/css-backgrounds-3_dev/nightly-unstable/html4/box-shadow-blur-definition-001.htm
<dael> dino: You use svg image and then mask out areas tht should be solid. We have tests in wk
<dino> s/dino/smfr/
<dael> dbaron: Is there a spacial requirements for the dithering? Is it color distance, spacial distance or both?
<chris> can you sent those wk tests to wpt?
<dael> TabAtkins: I had example text and my intention was color distance of ideal color
<dael> dbaron: wk test depends on spacial distance
<dael> TabAtkins: Can you describe that?
<chris> q+
<dael> dbaron: That's what I head from smfr but maybe he can exlain. I thought her said it was for solid segments and masking pieces of boundry
<Rossen_> ack dbaron
<dael> smfr: With continuous gradients you can do component transfer on top to posterize the gradient, convert a range to a flat color. Hides some issues but let's you test interp color stops
<dael> TabAtkins: Artifical banding to get close to idea
<dael> smfr: Correct
<chris> q?
<dino> If you pick the components correctly, you can emphasize the banding in the gradient, and then mask out the relevant portions.
<dael> chris: Two types of dithering. One is higher precision. Calc color and dither to get mix.
<dbaron> smfr also said before the minuted part that they or (or you can) do that for solid parts
<dael> chris: More common is dither position on gradient so if you have linear instea dof a vertical line you fuzz that. I think FF is doing that. We need to allow both
<dael> chris: smfr or dino can you confirm?
<dael> chris: [repeats] I'm assuming you're doing the latter, dither on gradient axis.
<dael> smfr: I don't think it's directional, jut adding noise
<dael> dino: When I suggested test that yous' how I'm testing, not what we do. It's a noise filter on result
<dael> chris: I see
<dael> smfr: It's not wk it's the underlying graphics that's using gq shaders and stuff
<fantasai> Fwiw wrt :--statename vs :state(statename), I think latter is better, clearer, more consistent with ::highlight(), easier to extend
<dael> chris: Seem to have consensus spec shoudl say this, question of how
<smfr> example of a test using feCompontentTransfer: https://codepen.io/smfr/pen/OJVpwoR
<Rossen_> q?
<Rossen_> ack chris
<dael> AmeliaBR: Can we resolve on general spec and then get people to give specific examples of how to test?
<dael> Rossen_: Yep.
<dael> Rossen_: Obj to Add this to Images ?
<dael> TabAtkins: I'll put it in 3 and 4
<dael> Rossen_: Obj to Add this to Images 3 and 4
<fantasai> chris, let's talk about diff specs? I can't solve a problem I don't understand
<smfr> s/gq shaders/gpu shaders/
<dael> RESOLVED: Add this to Images 3 and 4
<dael> TabAtkins: Anyone with tests they can contribute to wpt please convert them so they can be used as ref tests

[tabatkins]

So the outcome is just that impls that are already testing their dithered gradients, please share details of your testing strategy so we can put some testing requirements into the spec that'll make gradients testable without overly constraining implementations.