Skip to content

Commit

Permalink
test: Fix image comparison threshold
Browse files Browse the repository at this point in the history
Screenshot-based layout tests use the Jimp module to compare images.
But the threshold parameter was set too low (0), causing failures due
to tiny color changes that can't be seen by a human.  The most recent
Chrome update on Windows (86) caused a test failure because some
rendering changes on the edges of an image caused a few pixel values
to change very very slightly.  This fixes the issue by increasing the
threshold from 0 to 0.05.

Change-Id: Ia202c147d30f93779b0b0fed281ee1d256dee371
  • Loading branch information
joeyparrish committed Oct 7, 2020
1 parent e347dbb commit 160a98c
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion karma.conf.js
Original file line number Diff line number Diff line change
Expand Up @@ -584,7 +584,12 @@ function WebDriverScreenshotMiddlewareFactory(launcher) {

// Compare the new screenshot to the old one and produce a diff image.
// Initially, the image data will be raw pixels, 4 bytes per pixel.
const diff = Jimp.diff(oldScreenshot, newScreenshot, /* threshold= */ 0);
// The threshold parameter affects the sensitivity of individual pixel
// comparisons. Setting it too low means small rendering changes in the
// browser can cause failures even when a human can't see the difference,
// and setting it too high means human-noticeable changes could go
// undetected by a test.
const diff = Jimp.diff(oldScreenshot, newScreenshot, /* threshold= */ 0.05);

// Write the diff to disk. This is used to review when there are changes.
fs.writeFileSync(
Expand Down

0 comments on commit 160a98c

Please sign in to comment.