PIL Image to numpy array gives slightly different Image array in 9.x versions

[DrChrisLevy]

What did you do?

• created a PIL image object then convert to Numpy array in 9.X versions

What did you expect to happen?

• I expected to get the exact same array when using PIL version 8.4.0

What actually happened?

• the Numpy array in version 9.0.0 or 9.0.1 or 9.1 differs slightly than the array obtained in version 8.4.0
• I have a bunch of unit tests failing in a computer vision project using Tensorflow and image embeddings. They started failing when upgrading PIL from 8.4 to 9.X and the reason for the upgrade is security related.

What are your OS, Python and Pillow versions?

• OS: Linux 556361527a6d 5.4.0-1072-aws #77~18.04.1-Ubuntu SMP Thu Apr 7 21:38:47 UTC 2022 x86_64 GNU/Linux running in Docker.
• Python: 3.8.13
• Pillow: Pillow-9.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
• Numpy: numpy-1.19.5-cp38-cp38-manylinux2010_x86_64.whl

# run this code in PIL version 8.4.0 and then in  9.1.
# the two arrays will differ in pixel values at some locations in the image tensor.

import numpy as np
from PIL import Image
img_array = np.array(Image.open('my_image.jpg')).astype(np.uint8)

Here are some of the pixel values from version 9.1

a = img_array.flatten()[:1000]
array([ 91, 130, 195,  90, 129, 194,  90, 129, 194,  90, 129, 194,  90,
       129, 194,  89, 128, 193,  89, 128, 193,  89, 128, 193,  92, 131,
       196,  91, 130, 195,  91, 130, 195,  90, 129, 194,  91, 130, 195,
        92, 131, 196,  94, 133, 198,  95, 134, 199,  95, 134, 203,  94,
       133, 202,  93, 132, 201,  92, 131, 200,  92, 131, 200,  93, 132,
       201,  94, 133, 202,  95, 134, 203,  93, 132, 201,  93, 132, 201,
        93, 132, 201,  93, 132, 201,  93, 132, 201,  93, 132, 201,  92,
       131, 200,  92, 131, 200,  96, 133, 203,  95, 132, 202,  93, 130,
       200,  92, 129, 199,  91, 128, 198,  92, 129, 199,  92, 129, 199,
        93, 130, 200,  93, 130, 200,  92, 129, 199,  91, 128, 198,  92,
       129, 199,  93, 130, 200,  93, 130, 200,  93, 130, 200,  90, 129,
       198,  88, 127, 196,  87, 127, 196,  87, 127, 196,  88, 128, 197,
        88, 128, 197,  89, 129, 198,  91, 131, 200,  91, 131, 200,  92,
       132, 201,  91, 131, 200,  91, 131, 200,  90, 130, 199,  90, 130,
       199,  91, 131, 200,  91, 131, 200,  92, 133, 199,  90, 131, 197,
        90, 131, 195,  91, 132, 196,  92, 133, 197,  92, 133, 197,  92,
       133, 197,  91, 132, 196,  91, 132, 196,  89, 130, 194,  89, 130,
       194,  90, 131, 195,  91, 132, 196,  91, 132, 196,  91, 132, 196,
        91, 132, 196,  91, 132, 196,  89, 130, 194,  90, 131, 195,  91,
       132, 196,  89, 130, 194,  87, 128, 192,  87, 128, 192,  90, 131,
       195,  92, 133, 197,  90, 131, 195,  90, 131, 195,  90, 131, 195,
        90, 131, 195,  90, 131, 195,  90, 131, 195,  90, 131, 195,  90,
       131, 195,  89, 130, 194,  90, 131, 195,  90, 131, 195,  90, 131,
       195,  90, 131, 195,  89, 130, 194,  89, 130, 194,  88, 129, 193,
        90, 131, 195,  90, 131, 195,  90, 131, 195,  90, 131, 195,  90,
       131, 195,  90, 131, 195,  90, 131, 195,  90, 131, 195,  89, 132,
       200,  87, 130, 198,  84, 127, 195,  85, 128, 196,  87, 130, 198,
        89, 132, 200,  87, 130, 198,  86, 129, 197,  84, 127, 195,  85,
       128, 196,  85, 128, 196,  86, 129, 197,  87, 130, 198,  87, 130,
       198,  86, 129, 197,  86, 129, 197,  86, 129, 197,  86, 129, 197,
        86, 129, 197,  87, 130, 198,  87, 130, 198,  87, 130, 198,  88,
       131, 199,  88, 131, 199,  87, 130, 198,  87, 130, 198,  87, 130,
       198,  87, 130, 198,  87, 130, 198,  86, 129, 197,  86, 129, 197,
        86, 129, 197,  84, 129, 194,  86, 131, 196,  88, 133, 198,  88,
       133, 198,  86, 131, 196,  85, 130, 195,  85, 130, 195,  85, 130,
       195,  83, 128, 193,  84, 129, 194,  84, 129, 194,  85, 130, 195,
        85, 130, 195,  86, 131, 196,  86, 131, 196,  87, 132, 197,  87,
       130, 198,  87, 130, 198,  87, 130, 198,  87, 130, 198,  87, 130,
       198,  87, 130, 198,  87, 130, 198,  87, 130, 198,  86, 129, 197,
        86, 129, 197,  86, 129, 197,  87, 130, 198,  87, 130, 198,  88,
       131, 199,  88, 131, 199,  88, 131, 199,  85, 130, 197,  85, 130,
       197,  85, 130, 197,  85, 130, 197,  85, 130, 197,  85, 130, 197,
        85, 130, 197,  86, 131, 198,  88, 133, 200,  88, 133, 200,  88,
       133, 200,  87, 132, 199,  87, 132, 199,  86, 131, 198,  86, 131,
       198,  83, 130, 198,  84, 131, 203,  86, 130, 201,  91, 130, 195,
        92, 132, 194,  87, 132, 199,  84, 132, 196,  88, 129, 191,  85,
       129, 190,  80, 131, 196,  79, 129, 192,  89, 128, 183,  46,  68,
       109,  53,  50,  71,  51,  48,  67,  62,  80, 118, 104, 137, 190,
        89, 126, 197,  89, 128, 205,  93, 130, 208,  94, 132, 205,  95,
       130, 198,  89, 125, 187,  86, 118, 177,  81, 114, 165,  70, 101,
       148,  62,  94, 133,  55,  83, 120,  47,  76, 106,  43,  71,  95,
        39,  67,  88,  35,  62,  83,  32,  60,  82,  40,  66,  99,  36,
        66, 104,  40,  69, 109,  50,  83, 126,  68, 102, 148,  83, 121,
       170,  92, 132, 184,  94, 137, 192,  83, 128, 185,  83, 129, 188,
        83, 129, 189,  83, 129, 189,  84, 130, 190,  84, 130, 190,  85,
       129, 190,  86, 130, 193,  83, 128, 193,  83, 128, 195,  83, 128,
       195,  83, 128, 195,  83, 128, 195,  84, 129, 196,  84, 129, 196,
        85, 130, 197,  82, 127, 194,  83, 128, 195,  84, 129, 196,  85,
       130, 197,  85, 130, 197,  84, 129, 196,  83, 128, 195,  82, 127,
       194,  84, 129, 196,  84, 129, 196,  84, 129, 196,  84, 129, 196,
        85, 130, 197,  85, 130, 197,  85, 130, 197,  85, 130, 197,  82,
       127, 194,  83, 128, 195,  83, 128, 195,  83, 128, 195,  84, 129,
       196,  84, 129, 196,  85, 130, 197,  85, 130, 195,  82, 130, 194,
        82, 130, 192,  82, 130, 192,  82, 130, 192,  81, 129, 191,  81,
       129, 191,  81, 129, 191,  81, 129, 191,  81, 129, 191,  81, 129,
       191,  81, 129, 191,  81, 129, 191,  81, 129, 191,  80, 128, 190,
        80, 128, 190,  80, 128, 190,  79, 127, 193,  79, 127, 193,  79,
       127, 193,  80, 128, 194,  80, 128, 194,  81, 129, 195,  81, 129,
       195,  81, 129, 195,  82, 130, 196,  82, 130, 196,  82, 130, 196,
        82, 130, 196,  82, 130, 196,  82, 130, 196,  82, 130, 196,  82,
       130, 196,  81, 130, 196,  81, 130, 196,  80, 129, 195,  80, 129,
       195,  80, 129, 195,  79, 128, 194,  79, 128, 194,  79, 128, 194,
        79, 128, 194,  80, 129, 195,  82, 131, 197,  83, 132, 198,  83,
       132, 198,  81, 130, 196,  79, 128, 194,  78, 127, 193,  81, 129,
       195,  80, 128, 194,  80, 128, 194,  79, 127, 193,  79, 127, 193,
        80, 128, 194,  80, 128, 194,  81, 129, 195,  81, 129, 195,  81,
       129, 195,  80, 128, 194,  80, 128, 194,  80, 128, 194,  80],
      dtype=uint8)

Here are some of the pixel values from version 8.4

b = img_array.flatten()[:1000]
array([ 91, 130, 195,  90, 129, 194,  90, 129, 194,  90, 129, 194,  90,
       129, 194,  89, 128, 193,  89, 128, 193,  89, 128, 193,  92, 131,
       196,  91, 130, 195,  91, 130, 195,  90, 129, 194,  91, 130, 195,
        92, 131, 196,  94, 133, 198,  95, 134, 199,  95, 134, 203,  94,
       133, 202,  93, 132, 201,  92, 131, 200,  92, 131, 200,  93, 132,
       201,  94, 133, 202,  95, 134, 203,  93, 132, 201,  93, 132, 201,
        93, 132, 201,  93, 132, 201,  93, 132, 201,  93, 132, 201,  92,
       131, 200,  92, 131, 200,  96, 133, 203,  95, 132, 202,  93, 130,
       200,  92, 129, 199,  91, 128, 198,  92, 129, 199,  92, 129, 199,
        93, 130, 200,  93, 130, 200,  92, 129, 199,  91, 128, 198,  92,
       129, 199,  93, 130, 200,  93, 130, 200,  93, 130, 200,  91, 128,
       198,  87, 127, 196,  87, 127, 196,  87, 127, 196,  88, 128, 197,
        88, 128, 197,  89, 129, 198,  91, 131, 200,  91, 131, 200,  92,
       132, 201,  91, 131, 200,  91, 131, 200,  90, 130, 199,  90, 130,
       199,  91, 131, 200,  91, 131, 200,  92, 132, 201,  90, 131, 195,
        90, 131, 195,  91, 132, 196,  92, 133, 197,  92, 133, 197,  92,
       133, 197,  91, 132, 196,  91, 132, 196,  89, 130, 194,  89, 130,
       194,  90, 131, 195,  91, 132, 196,  91, 132, 196,  91, 132, 196,
        91, 132, 196,  91, 132, 196,  89, 130, 194,  90, 131, 195,  91,
       132, 196,  89, 130, 194,  87, 128, 192,  87, 128, 192,  90, 131,
       195,  92, 133, 197,  90, 131, 195,  90, 131, 195,  90, 131, 195,
        90, 131, 195,  90, 131, 195,  90, 131, 195,  90, 131, 195,  90,
       131, 195,  89, 130, 194,  90, 131, 195,  90, 131, 195,  90, 131,
       195,  90, 131, 195,  89, 130, 194,  89, 130, 194,  88, 129, 193,
        90, 131, 195,  90, 131, 195,  90, 131, 195,  90, 131, 195,  90,
       131, 195,  90, 131, 195,  90, 131, 195,  90, 131, 195,  89, 132,
       200,  87, 130, 198,  84, 127, 195,  85, 128, 196,  87, 130, 198,
        89, 132, 200,  87, 130, 198,  86, 129, 197,  84, 127, 195,  85,
       128, 196,  85, 128, 196,  86, 129, 197,  87, 130, 198,  87, 130,
       198,  86, 129, 197,  86, 129, 197,  86, 129, 197,  86, 129, 197,
        86, 129, 197,  87, 130, 198,  87, 130, 198,  87, 130, 198,  88,
       131, 199,  88, 131, 199,  87, 130, 198,  87, 130, 198,  87, 130,
       198,  87, 130, 198,  87, 130, 198,  86, 129, 197,  86, 129, 197,
        86, 129, 197,  84, 129, 194,  86, 131, 196,  88, 133, 198,  88,
       133, 198,  86, 131, 196,  85, 130, 195,  85, 130, 195,  85, 130,
       195,  83, 128, 193,  84, 129, 194,  84, 129, 194,  85, 130, 195,
        85, 130, 195,  86, 131, 196,  86, 131, 196,  87, 132, 197,  87,
       130, 198,  87, 130, 198,  87, 130, 198,  87, 130, 198,  87, 130,
       198,  87, 130, 198,  87, 130, 198,  87, 130, 198,  86, 129, 197,
        86, 129, 197,  86, 129, 197,  87, 130, 198,  87, 130, 198,  88,
       131, 199,  88, 131, 199,  88, 131, 199,  85, 130, 197,  85, 130,
       197,  85, 130, 197,  85, 130, 197,  85, 130, 197,  85, 130, 197,
        85, 130, 197,  86, 131, 198,  88, 133, 200,  88, 133, 200,  88,
       133, 200,  87, 132, 199,  87, 132, 199,  86, 131, 198,  86, 131,
       198,  85, 130, 197,  79, 132, 208,  86, 131, 199,  94, 130, 192,
        92, 132, 194,  85, 133, 199,  84, 132, 198,  88, 129, 191,  88,
       128, 187,  81, 130, 196,  72, 131, 201,  82, 130, 192,  49,  67,
       105,  59,  48,  64,  58,  46,  60,  65,  79, 114,  98, 139, 195,
        88, 125, 206,  90, 127, 207,  93, 130, 208,  95, 131, 205,  95,
       129, 200,  90, 124, 188,  86, 118, 175,  82, 114, 165,  70, 101,
       147,  64,  94, 132,  55,  84, 116,  49,  76, 103,  44,  70,  93,
        40,  67,  86,  36,  62,  79,  34,  60,  75,  40,  66, 101,  37,
        65, 102,  40,  70, 108,  51,  83, 124,  68, 102, 147,  83, 121,
       168,  91, 133, 183,  94, 137, 190,  83, 128, 183,  83, 129, 188,
        83, 129, 188,  83, 129, 189,  84, 130, 190,  84, 130, 190,  84,
       130, 190,  86, 130, 191,  83, 128, 195,  83, 128, 195,  83, 128,
       195,  83, 128, 195,  83, 128, 195,  84, 129, 196,  84, 129, 196,
        85, 130, 197,  82, 127, 194,  83, 128, 195,  84, 129, 196,  85,
       130, 197,  85, 130, 197,  84, 129, 196,  83, 128, 195,  82, 127,
       194,  84, 129, 196,  84, 129, 196,  84, 129, 196,  84, 129, 196,
        85, 130, 197,  85, 130, 197,  85, 130, 197,  85, 130, 197,  82,
       127, 194,  83, 128, 195,  83, 128, 195,  83, 128, 195,  84, 129,
       196,  84, 129, 196,  85, 130, 197,  85, 130, 197,  82, 130, 192,
        82, 130, 192,  82, 130, 192,  82, 130, 192,  81, 129, 191,  81,
       129, 191,  81, 129, 191,  81, 129, 191,  81, 129, 191,  81, 129,
       191,  81, 129, 191,  81, 129, 191,  81, 129, 191,  80, 128, 190,
        80, 128, 190,  80, 128, 190,  79, 127, 193,  79, 127, 193,  79,
       127, 193,  80, 128, 194,  80, 128, 194,  81, 129, 195,  81, 129,
       195,  81, 129, 195,  82, 130, 196,  82, 130, 196,  82, 130, 196,
        82, 130, 196,  82, 130, 196,  82, 130, 196,  82, 130, 196,  82,
       130, 196,  81, 130, 196,  81, 130, 196,  80, 129, 195,  80, 129,
       195,  80, 129, 195,  79, 128, 194,  79, 128, 194,  79, 128, 194,
        79, 128, 194,  80, 129, 195,  82, 131, 197,  83, 132, 198,  83,
       132, 198,  81, 130, 196,  79, 128, 194,  78, 127, 193,  81, 129,
       195,  80, 128, 194,  80, 128, 194,  79, 127, 193,  79, 127, 193,
        80, 128, 194,  80, 128, 194,  81, 129, 195,  81, 129, 195,  81,
       129, 195,  80, 128, 194,  80, 128, 194,  80, 128, 194,  80],
      dtype=uint8)

a - b

array([ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0, -1,  1,  0,  1,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  1, -2,  0,  0,  2,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, -2,  0,  1,  5, -1,
       -5,  0, -1,  2, -3,  0,  3,  0,  0,  0,  2, -1,  0,  0,  0, -2,  0,
        0,  0, -3,  1,  3, -1,  1,  0,  7, -2, -9,  7, -2, -9, -3,  1,  4,
       -6,  2,  7, -7,  2,  7, -3,  1,  4,  6, -2, -5,  1,  1, -9, -1,  1,
       -2,  0,  0,  0, -1,  1,  0,  0,  1, -2, -1,  1, -1,  0,  0,  2, -1,
        0,  0,  0,  0,  1, -2,  0,  1,  0, -1,  4, -2,  0,  3, -1,  1,  2,
       -1,  0,  2, -1,  0,  4, -2,  0,  7,  0,  0, -2, -1,  1,  2,  0, -1,
        1, -1,  0,  2,  0,  0,  1,  0,  0,  2,  1, -1,  1,  0,  0,  2,  0,
        0,  2,  0,  0,  0,  0,  0,  1,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        1, -1,  0,  0,  0,  2,  0,  0, -2,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, -2,
        0,  0,  2,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0])

[nulano]

This sounds like a duplicate of #6113, #5190, and many others.

In summary, different JPEG decoders are allowed to produce a different output. Before Pillow 9.0.0, libjpeg was used in Linux and macOS wheels, and libjpeg-turbo was used in Windows wheels. Since Pillow 9.0.0, libjpeg-turbo is used in all wheels: python-pillow/pillow-wheels#237

If you would like to keep using libjpeg, you will have to compile Pillow from source.

[DrChrisLevy]

Thanks for the quick reply.
It would be good then to update the release notes for 9.0.0 under under the section titled "Switched to libjpeg-turbo in macOS and Linux wheels". Then more context could be provided around getting slightly different pixel values etc. Basically what you said above.
I guess this can be closed.

[radarhere]

I've created #6264 to add that note.