Skip to content

Erode, dilate, threshold shader filters match closer to CPU filters #6405

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 7, 2023
Merged

Erode, dilate, threshold shader filters match closer to CPU filters #6405

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0feb343
use luma for erode, dilate to match cpu method; see #6398
wong-justin Sep 7, 2023
85e1107
use floor() in threshold filter to match cpu method; see #6398
wong-justin Sep 7, 2023
3ae8a2a
texelSize uses pixelDensity; see #6398, #6399
wong-justin Sep 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions src/webgl/p5.RendererGL.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1061,6 +1061,9 @@ p5.RendererGL = class RendererGL extends p5.Renderer {

pg.clear(); // prevent undesirable feedback effects accumulating secretly

let pd = this._pInst.pixelDensity();
let texelSize = [1 / (this.width * pd), 1 / (this.height * pd)];

// apply blur shader with multiple passes
if (operation === constants.BLUR) {

Expand All @@ -1074,7 +1077,7 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
this.filterGraphicsLayerTemp.image(this, -this.width/2, -this.height/2);

pg.shader(this.filterShader);
this.filterShader.setUniform('texelSize', [1/this.width, 1/this.height]);
this.filterShader.setUniform('texelSize', texelSize);
this.filterShader.setUniform('steps', Math.max(1, filterParameter));

// horiz pass
Expand All @@ -1096,7 +1099,7 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
else {
pg.shader(this.filterShader);
this.filterShader.setUniform('tex0', this);
this.filterShader.setUniform('texelSize', [1/this.width, 1/this.height]);
this.filterShader.setUniform('texelSize', texelSize);
this.filterShader.setUniform('canvasSize', [this.width, this.height]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these values also be multiplied by pixel density?

Copy link
Contributor

@davepagurek davepagurek Sep 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good question. I think ideally, we'd want to make the shader handle pixel density rather than the user calling the filter function so that it's easier to make density-agnostic sketches. That means the shader would likely want to know about both the width and the pixel density. I think right now this is possible by comparing texelSize and canvasSize, but wouldn't be possible if we also multiply by pixel density here. (Maybe it's worth adding pixelDensity as a separate uniform so it doesn't always have to be calculated by dividing the other two? Not necessary in this PR though!)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking maybe all shader uniforms should account for pixel density, ie. texelSize = 1 / (width * pixelDensity), 1 / (height * pixelDensity) and canvasSize = width * pixelDensity, height * pixelDensity. That way they would all be doing work at the same level of resolution. Although having canvasSize != width, height is probably not what the user expects.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the main reason why we might not want that for all uniforms is that if the pixel density is always premultiplied, then the shader can't distinguish between a canvas at 2x density and a canvas at 2x the size. That would mean that on the user's screen, a filter(BLUR, 3) on a 2x display might end up looking less blurry than on a 1x display, since the blur radius would be smaller relative to the physical size of the canvas. I think I naturally interpret the 3 in filter(BLUR, 3) as being the same units the rest of p5 uses, pixels times pixel density, rather than real pixels.

I think it's ok having most uniforms come with pixel density premultiplied as long as we also pass the density into the shader on its own too though.

// filterParameter uniform only used for POSTERIZE, and THRESHOLD
// but shouldn't hurt to always set
Expand Down
14 changes: 13 additions & 1 deletion src/webgl/shaders/filters/dilate.frag
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,15 @@ varying vec2 vTexCoord;
uniform sampler2D tex0;
uniform vec2 texelSize;

float luma(vec3 color) {
// weighted grayscale with luminance values
// weights 77, 151, 28 taken from src/image/filters.js
return dot(color, vec3(0.300781, 0.589844, 0.109375));
}

void main() {
vec4 color = texture2D(tex0, vTexCoord);
float lum = luma(color.rgb);

// set current color as the brightest neighbor color

Expand All @@ -20,7 +27,12 @@ void main() {

for (int i = 0; i < 4; i++) {
vec4 neighborColor = neighbors[i];
color = max(color, neighborColor);
float neighborLum = luma(neighborColor.rgb);

if (neighborLum > lum) {
color = neighborColor;
lum = neighborLum;
}
}

gl_FragColor = color;
Expand Down
14 changes: 13 additions & 1 deletion src/webgl/shaders/filters/erode.frag
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,15 @@ varying vec2 vTexCoord;
uniform sampler2D tex0;
uniform vec2 texelSize;

float luma(vec3 color) {
// weighted grayscale with luminance values
// weights 77, 151, 28 taken from src/image/filters.js
return dot(color, vec3(0.300781, 0.589844, 0.109375));
}

void main() {
vec4 color = texture2D(tex0, vTexCoord);
float lum = luma(color.rgb);

// set current color as the darkest neighbor color

Expand All @@ -20,7 +27,12 @@ void main() {

for (int i = 0; i < 4; i++) {
vec4 neighborColor = neighbors[i];
color = min(color, neighborColor);
float neighborLum = luma(neighborColor.rgb);

if (neighborLum < lum) {
color = neighborColor;
lum = neighborLum;
}
}

gl_FragColor = color;
Expand Down
3 changes: 2 additions & 1 deletion src/webgl/shaders/filters/threshold.frag
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ float luma(vec3 color) {
void main() {
vec4 color = texture2D(tex0, vTexCoord);
float gray = luma(color.rgb);
float threshold = filterParameter;
// floor() used to match src/image/filters.js
float threshold = floor(filterParameter * 255.0) / 255.0;
float blackOrWhite = step(threshold, gray);
gl_FragColor = vec4(vec3(blackOrWhite), color.a);
}