Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement QOI importing. #80768

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

pidogs
Copy link
Contributor

@pidogs pidogs commented Aug 18, 2023

I added support for the QOI format of images. More information can be found at https://qoiformat.org/.
Only importing was implemented not exporting of QOI images.

There are two checks for if the image is formatted correctly.

  1. Checks if the file starts with "qoif"
  2. Checks if the header items are not blank or out of range.

Should there be any other checks before decoding the image?

QOI is an opensource image format under the CC0 License.

@Mickeon
Copy link
Contributor

Mickeon commented Aug 18, 2023

Could you make a proposal to gauge interest on the QOI format? I find it quite interesting myself, but in core? Unsure right now?

@Calinou
Copy link
Member

Calinou commented Aug 18, 2023

Could you make a proposal to gauge interest on the QOI format? I find it quite interesting myself, but in core? Unsure right now?

There's already an open proposal: godotengine/godot-proposals#3726

@pidogs
Copy link
Contributor Author

pidogs commented Aug 19, 2023

It is fair if it is not included in the core, but more and more programs are natively supporting it.
I find it most useful when on a low power machine and exporting from gimp, the process is very quick compared to png exporting.

@Calinou
Copy link
Member

Calinou commented Aug 19, 2023

I find it most useful when on a low power machine and exporting from gimp, the process is very quick compared to png exporting.

You could use BMP or TGA exporting if speed is a priority. These should be even faster to write than QOI, as they're not compressed.

@pidogs
Copy link
Contributor Author

pidogs commented Aug 19, 2023

You are right but in that case why does Godot support more than one image format if every image can be converted into one format?

I'm just taking things to the extreme, no hard feelings on the final decision.

@Mickeon
Copy link
Contributor

Mickeon commented Aug 19, 2023

Sorry. I'm aware they are alternative suggestions, but both BMP and TGA cannot be compared for how space efficient QOI is.

@Calinou
Copy link
Member

Calinou commented Aug 19, 2023

Binary size of Linux x86_64 release export templates with LTO (stripped):

Master: 58,341,216 bytes
This PR: 58,349,408 bytes

This is a difference of exactly 8192 bytes. It's not huge, but it's not negligible either considering this affects every exported project that uses official export templates. The exact size difference will vary depending on the platform.

Sorry. I'm aware they are alternative suggestions, but both BMP and TGA cannot be compared for how space efficient QOI is.

QOI isn't very space-efficient compared to PNG or WebP. My point is, you can use BMP/TGA during iteration and convert to lossless WebP (which is more efficient than PNG) once you're done iterating. This way, you'll get very fast save times while iterating and an efficient saved file once you're done.

There's unfortunately no image format that can do both good compression or fast saving without having to use another format.1 In my experience, PNG in uncompressed form is still fairly slow to save for some reason.

Footnotes

  1. If JPEG-XL had an uncompressed option, it could perhaps do that, but I think it only provides lossless or lossy options.

@aaronfranke
Copy link
Member

aaronfranke commented Aug 19, 2023

Given that this caters to a more niche use-case (projects that need very fast image encoding/decoding), I think this should be a third-party add-on in the asset library instead of core. Most users only need an efficient format for lossless/lossy, and don't mind waiting a fraction of a second for the image to be imported. There's also the KTX2 / Basis Universal format (recently merged #76572) if you need fast conversion to GPU texture formats.

EDIT: As discussed below, movie maker mode would be a good use case that could justify this feature.

@pidogs
Copy link
Contributor Author

pidogs commented Aug 20, 2023

Just to set any misconceptions to rest I use took an image set1 (1.2 GB PNG) and re-encoded it imagemagick2 and here are the results.

Type Size (bytes) % of png size Real Time User Time sys Time png / real time
bmp 4815411200 434.78 % 0m39.311s 0m37.656s 0m11.409s 677.94 %
gif 463912960 041.89 % 4m31.981s 5m30.570s 1m19.085s 097.99 %
jpg 346152960 031.25 % 0m45.587s 0m47.107s 0m11.979s 584.61 %
png 1107558400 100.00 % 4m26.505s 5m59.869s 0m16.139s 100.00 %
qoi 1368483840 123.56 % 0m47.160s 0m49.133s 0m10.547s 565.11 %
TGA 4733992960 427.43 % 0m56.646s 0m57.254s 0m11.774s 470.47 %
tif 4712478720 425.48 % 0m40.355s 0m39.593s 0m12.360s 660.40 %

QOI is very fast and has very good compression when compared to bmp. Compared to png it falls just a little behind but it has very quick encoding.

Script used to encode images

ext=$1
mkdir $ext
find "./Art" -type f -name "*.png" -print0 | while read -r -d '' i ; do
	mkdir -p "./$ext/${i%/*}"
	convert "$i" "./$ext/${i::-4}.$ext"
done

Footnotes

  1. Image data set.
    https://opengameart.org/content/roguelikerpg-pack-1700-tiles
    https://opengameart.org/content/platformer-art-complete-pack-often-updated
    https://opengameart.org/content/dungeon-crawl-32x32-tiles
    https://qoiformat.org/benchmark/

  2. $ convert -version
    Version: ImageMagick 7.1.1-16 (Beta) Q16-HDRI x86_64 f5abb90e2:20230819 https://imagemagick.org
    Copyright: (C) 1999 ImageMagick Studio LLC
    License: https://imagemagick.org/script/license.php
    Features: Cipher DPC HDRI Modules OpenMP(4.5)
    Delegates (built-in): bzlib djvu heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr png tiff webp x xml zlib
    Compiler: gcc (11.4)

@Calinou
Copy link
Member

Calinou commented Aug 20, 2023

Just to set any misconceptions to rest I use took an image set1 (1.2 GB PNG) and re-encoded it imagemagick2 and here are the results.

Encode times actually look quite competitive compared to BMP (and especially TGA), so it may be worth having this for Movie Maker mode (where saving uncompressed data may be unreasonably large, e.g. for 4K video). The only issue is making sure you have a FFmpeg version that can read QOI files so you can reencode them into a video format (this is the case since FFmpeg 5.1).

That said, having support for Movie Maker mode would require implementing QOI encoding, not decoding. This is almost guaranteed to be a better tradeoff than #74324 when you don't need compatibility with software that can't read QOI files.

<return type="int" enum="Error" />
<param index="0" name="buffer" type="PackedByteArray" />
<description>
Loads an image from the binary contents of a QOI file.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Loads an image from the binary contents of a QOI file.
Loads an image from the binary contents of a [url=https://qoiformat.org/]QOI[/url] file.

Copy link
Member

@Calinou Calinou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the benchmarking data, I think there's a good argument for having this in core, but only if encoding is also implemented. (This is my opinion; this PR isn't guaranteed to be merged if encoding is implemented.)

@pidogs
Copy link
Contributor Author

pidogs commented Aug 20, 2023

I will take a look at implementing QOI encoding if there is interest in it.

@Chaosus Chaosus added this to the 4.x milestone Aug 21, 2023
@DeeJayLSP
Copy link
Contributor

May I suggest where QOI encoding could be useful?

Whenever an image gets imported into Godot, it gets internally converted into WebP.

When an image being imported is above the WebP limit (16383x16383) the importer converts it to PNG instead.

QOI claims to have a higher dimension limit when compared to PNG (although I doubt anyone would use an image that large).

Maybe QOI could be an alternative to that?

@Calinou
Copy link
Member

Calinou commented Sep 13, 2023

When an image being imported is above the WebP limit (16383x16383) the importer converts it to PNG instead.

QOI claims to have a higher dimension limit when compared to PNG (although I doubt anyone would use an image that large).

Maybe QOI could be an alternative to that?

Godot doesn't support images larger than 32768×32768, and desktop GPUs rarely support images larger than 16384×16384 (no mobile GPUs do). Displaying a single 16384×16384 image without VRAM compression also requires a lot of video memory 🙂

@DeeJayLSP
Copy link
Contributor

QOI is an opensource image format under the CC0 License.

Both in the website and the repository clearly state that QOI is under the MIT license. CC0 is only for the logo.

Unless...

Is this based on the format specification instead of the reference implementation? If so, then there's no problem (that's how public-domain single-file libraries like stb_image and minimp3 are made).

@pidogs
Copy link
Contributor Author

pidogs commented Sep 13, 2023

You are correct. I did not read the license correctly.
Some code is copied from the reference implementation, but the entire file is not used. It sounds like it can be implemented without royalties either way.

I am not a lawyer. I am just a programmer for fun.

@jonnyawsom3
Copy link

jonnyawsom3 commented Dec 24, 2023

  1. If JPEG-XL had an uncompressed option, it could perhaps do that, but I think it only provides lossless or lossy options.

@Calinou
I was rummaging around the internet and ended up here, thought I'd mention that JXL has a separate "Fast Lossless" code path that reaches 600 MP/s encode, although I'd still let the software mature for proper multithreading, ect (Or look at alternatives such as JXL-Oxide)

Either way, apologies for poking the dead thread, but thought it was worth mentioning

@DeeJayLSP
Copy link
Contributor

DeeJayLSP commented Dec 24, 2023

  1. If JPEG-XL had an uncompressed option, it could perhaps do that, but I think it only provides lossless or lossy options.

I'd like to point problem I see with JPEG XL: the less complex the image is (like a pixel art) compression tends to become less effective, sometimes losing even to PNG.

Take the following image from Kenney for example:

tinydungeon

  • Lossless WebP (slowest lossless preset): 20.7 KiB
  • PNG (oxipng max compression with zopfli): 23.1 KiB
  • JPEG XL (lossless, highest compression efforts): 26.4 KiB
  • QOI: 96.2 KiB

Since we're talking about fast saving, I should mention you can always use lowest effort, which kinda results in a lower size when compared to QOI (I haven't tested encoding time though).

@basicer
Copy link
Contributor

basicer commented Apr 27, 2024

Given the benchmarking data, I think there's a good argument for having this in core, but only if encoding is also implemented. (This is my opinion; this PR isn't guaranteed to be merged if encoding is implemented.)

I gave this a shot and the results look very promising.

Movie Maker mode enabled, recording movie at 60 FPS...
----------------
Done recording movie at path: c:\projects\capture\png\out.png
500 frames at 60 FPS (movie length: 00:00:08), recorded in 00:02:06 (6% of real-time speed).
CPU time: 0.92 seconds (average: 1.84 ms/frame)
GPU time: 10.41 seconds (average: 20.83 ms/frame)
----------------
Size: 583mb


Movie Maker mode enabled, recording movie at 60 FPS...
----------------
Done recording movie at path: c:\projects\capture\qoi\out.qoi
500 frames at 60 FPS (movie length: 00:00:08), recorded in 00:00:22 (36% of real-time speed).
CPU time: 0.86 seconds (average: 1.73 ms/frame)
GPU time: 1.65 seconds (average: 3.29 ms/frame)
----------------
Size: 632mb

A 6x speedup. Uses a bit more storage space, but that doesn't really matter if you goal is to just feed it into ffmpeg anyway.

@pidogs pidogs closed this Apr 27, 2024
@pidogs
Copy link
Contributor Author

pidogs commented Apr 27, 2024

That looks very promising.
What scene did you use? When I first started to test encoding I used https://github.com/Calinou/godot-movie-maker-demo as my test scene. Also, what are the specs of your computer?
(sorry I closed this pull request I don't know what key combination did it.)

@pidogs pidogs reopened this Apr 27, 2024
@basicer
Copy link
Contributor

basicer commented Apr 27, 2024

I let the editor convert Godot Lumberyard Bistro from 4.0 -> 4.3 and added a camera. i9-12900K, 3070Ti, 128GB ram. This is using the reference encoder.

Ill try that scene.

@basicer
Copy link
Contributor

basicer commented Apr 28, 2024

Similar results on @Calinou 's movie test scene

PS C:\projects\godot> .\bin\godot.windows.editor.x86_64.exe --write-movie c:\projects\capture\png\out.png --path C:\projects\godot-movie-maker-demo\ --quit-after 500
PS C:\projects\godot> Godot Engine v4.3.dev.custom_build.0478170ec (2024-04-26 05:43:14 UTC) - https://godotengine.org
Vulkan 1.3.277 - Forward+ - Using Device #0: NVIDIA - NVIDIA GeForce RTX 3070 Ti

Movie Maker mode enabled, recording movie at 60 FPS...
----------------
Done recording movie at path: c:\projects\capture\png\out.png
500 frames at 60 FPS (movie length: 00:00:08), recorded in 00:02:10 (6% of real-time speed).
CPU time: 0.38 seconds (average: 0.76 ms/frame)
GPU time: 14.99 seconds (average: 29.98 ms/frame)
----------------
Filesize: 553mb

PS C:\projects\godot> .\bin\godot.windows.editor.x86_64.exe --write-movie c:\projects\capture\qoi\out.qoi --path C:\projects\godot-movie-maker-demo\ --quit-after 500
PS C:\projects\godot> Godot Engine v4.3.dev.custom_build.0478170ec (2024-04-26 05:43:14 UTC) - https://godotengine.org
Vulkan 1.3.277 - Forward+ - Using Device #0: NVIDIA - NVIDIA GeForce RTX 3070 Ti

Movie Maker mode enabled, recording movie at 60 FPS...
----------------
Done recording movie at path: c:\projects\capture\qoi\out.qoi
500 frames at 60 FPS (movie length: 00:00:08), recorded in 00:00:21 (38% of real-time speed).
CPU time: 0.31 seconds (average: 0.62 ms/frame)
GPU time: 6.26 seconds (average: 12.52 ms/frame)
----------------
Filesize: 592mb

@basicer
Copy link
Contributor

basicer commented Apr 28, 2024

I pushed a draft of QOI Movie Maker support to #91263 if you want to try for yourself.

@AThousandShips AThousandShips changed the title Implemented QOI importing. Implement QOI importing. Apr 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support of QOI images
9 participants