Xiaomi 14 ultra photo cannot process for 1.118.2 version

[cmingkit]

The bug

It can't generate the THUMBNAILS, even set the IMMICH_PROCESS_INVALID_IMAGES=true

Here is the log:
[Nest] 7 - 10/19/2024, 11:06:19 AM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order.
[Nest] 7 - 10/19/2024, 11:06:19 AM ERROR [Microservices:JobService] Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order.
at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)
at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)
at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25)
at async /usr/src/app/dist/services/job.service.js:163:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

Following is sample image from Xiaomi 14 ultra

The OS that Immich Server is running on

Ubuntu 22.04.5 LTS

Version of Immich Server

v1.118.2

Version of Immich Mobile App

v1.118.2

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

version: "3.8"

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
    ports:
      - 2283:8080
    depends_on:
      - immich-server
      - immich-web
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/sharefolder/immich/data/

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=some-random-text
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. try to synchronize photo from
2. also try to upload in website

Relevant log output

[Nest] 7  - 10/19/2024, 11:06:19 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order.
[Nest] 7  - 10/19/2024, 11:06:19 AM   ERROR [Microservices:JobService] Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order.
    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)
    at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)
    at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25)
    at async /usr/src/app/dist/services/job.service.js:163:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

Additional information

No response

[cmingkit]

Previous version without this issue

[alextran1502]

Your docker compose file is out of date. Please update it an retry

[cmingkit]

I copied the wrong one, already use the latest one to update
https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml

and would you mind to use my photo to test at your own environment, thanks.

[mertalev]

Can you clarify if the error is specific to this image, or if no images from this phone can be processed?

[cmingkit]

All photos took by this phone can't be processed. I have Fujifilm photos are work and my friend sent me the whatsapp phone no this problem through this phone to upload.

And this problem start from 1.118.2 version.

[cmingkit]

It still work on Oct 15, 2024

above is upload on Oct 16, 2024 and my Xaioma 14 ultra without any camera app or OTA update in these 2 days.

Following is front page of "Photos" page

[cmingkit]

seems like my phone's photo is not compatible with Jpegli
Could I restore my immich to v1.118.1?

Better JPEG compression
Immich now uses Jpegli, a new library leveraging the advancements of JPEG XL to shrink JPEG file size at the same (or higher) quality. This change narrows the gap between JPEG and WebP compression considerably, especially at high quality.

[mertalev]

Could you share a sample file from this phone? I'd like to see if there's anything we can do.

In the meantime, is it possibly for you to take images in HEIC format instead of JPEG? The compression and quality should be better as well.

[cmingkit]

I tried HEIF (not HEIC), it work and can be workaround.

Following is JPEG files from my phone

https://mkcqb.dynv6.net:666/$/nR3Yo

https://mkcqb.dynv6.net:666/$/wwHPb

https://mkcqb.dynv6.net:666/$/d94Op

https://mkcqb.dynv6.net:666/$/wQiib

Thank you for your quick reply

[CrushedAsian255]

If the issue stems from Jpegli, it should also be reported to the Jpegli repository.

[cmingkit]

Agree if the root cause from Jpegli, I am also trying to build the Jpegli in my desktop and run the djpegli tool.
However, not sure is it only the Xiaomi phone issue. There is still possible to keep this ticket open to see the overall impact.

[jonnyawsom3]

A lot of phones seem to be creating invalid files since they've been trying to add UltraHDR gainmaps to them as seen here libjxl/libjxl#3882

Could be yet another instance, but more testing is needed

[cmingkit]

OK, Could immich consider has option for user not use libjxl in future version? From user propective, I think the previous version is good at this part for handling the image and I am no concern on the compression and performance.

[mertalev]

So for reference, the error is thrown on this line in the jpegli decoder. The equivalent section in libjpeg-turbo is here. Now, I'm not sure if this image would return false for marker_is_icc or trip any of the warnings, but even if it does it will just move on. I think a warning is more appropriate than an error here.

Also, I find it odd that JPEGLI_ERROR makes the image processing fail even when libvips is set to not fail on error.

[CrushedAsian255]

I think we keep jpegli and if a failure occurs fall back on libjpeg-turbo.