Repository navigation
WebP - Improve performance #1786
Description
Now that WebP has been merged #1552 into master we need to polish the performance to bring it up to speed for the release of V2.
We're looking for community contribution here while we focus on bring the Drawing/Fonts libraries up to RC status and also work on the new Unamanaged Memory allocator #1730 so anyone who like to make things go fast please have a crack.
Benchmarks
The last benchmarks (June 2021) indicate that we're a good bit slower than the Magick.NET implementation. (There's a positive here, potentially lots of low hanging fruit)
Decode
Big improvements in .NET Core with some SIMD which is positive. Definitely some allocations to deal with but will require profiling to determine if something else is in play.
Intel Core i7-6700K CPU 4.00GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.1.202
[Host] : .NET Core 3.1.4 (CoreCLR 4.700.20.20201, CoreFX 4.700.20.22101), X64 RyuJIT
Job-AQFZAV : .NET Framework 4.8 (4.8.4180.0), X64 RyuJIT
Job-YCDAPQ : .NET Core 2.1.18 (CoreCLR 4.6.28801.04, CoreFX 4.6.28802.05), X64 RyuJIT
Job-WMTYOZ : .NET Core 3.1.4 (CoreCLR 4.700.20.20201, CoreFX 4.700.20.22101), X64 RyuJIT
IterationCount=3 LaunchCount=1 WarmupCount=3
| Method | Job | Runtime | TestImageLossy | TestImageLossless | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|--------------------------- |----------- |-------------- |---------------------- |------------------------- |-----------:|----------:|---------:|----------:|----------:|------:|------------:|
| 'Magick Lossy Webp' | Job-IERNAB | .NET 4.7.2 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 105.8 ms | 6.28 ms | 0.34 ms | - | - | - | 17.65 KB |
| 'ImageSharp Lossy Webp' | Job-IERNAB | .NET 4.7.2 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 1,145.0 ms | 110.82 ms | 6.07 ms | - | - | - | 2779.53 KB |
| 'Magick Lossless Webp' | Job-IERNAB | .NET 4.7.2 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 145.9 ms | 8.55 ms | 0.47 ms | - | - | - | 18.05 KB |
| 'ImageSharp Lossless Webp' | Job-IERNAB | .NET 4.7.2 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 1,694.1 ms | 55.09 ms | 3.02 ms | 4000.0000 | 1000.0000 | - | 30556.87 KB |
| 'Magick Lossy Webp' | Job-IMRAGJ | .NET Core 2.1 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 105.7 ms | 1.89 ms | 0.10 ms | - | - | - | 15.75 KB |
| 'ImageSharp Lossy Webp' | Job-IMRAGJ | .NET Core 2.1 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 741.6 ms | 21.45 ms | 1.18 ms | - | - | - | 2767.85 KB |
| 'Magick Lossless Webp' | Job-IMRAGJ | .NET Core 2.1 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 146.1 ms | 9.52 ms | 0.52 ms | - | - | - | 16.54 KB |
| 'ImageSharp Lossless Webp' | Job-IMRAGJ | .NET Core 2.1 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 522.5 ms | 21.15 ms | 1.16 ms | 4000.0000 | 1000.0000 | - | 22860.02 KB |
| 'Magick Lossy Webp' | Job-NAASQX | .NET Core 3.1 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 105.9 ms | 5.34 ms | 0.29 ms | - | - | - | 15.45 KB |
| 'ImageSharp Lossy Webp' | Job-NAASQX | .NET Core 3.1 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 748.8 ms | 290.47 ms | 15.92 ms | - | - | - | 2767.84 KB |
| 'Magick Lossless Webp' | Job-NAASQX | .NET Core 3.1 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 146.1 ms | 1.14 ms | 0.06 ms | - | - | - | 15.9 KB |
| 'ImageSharp Lossless Webp' | Job-NAASQX | .NET Core 3.1 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 480.7 ms | 25.25 ms | 1.38 ms | 4000.0000 | 1000.0000 | - | 22859.7 KB |
| 'Magick Lossy Webp' | Job-GLNACU | .NET Core 5.0 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 105.7 ms | 4.71 ms | 0.26 ms | - | - | - | 15.48 KB |
| 'ImageSharp Lossy Webp' | Job-GLNACU | .NET Core 5.0 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 645.7 ms | 61.00 ms | 3.34 ms | - | - | - | 2768.13 KB |
| 'Magick Lossless Webp' | Job-GLNACU | .NET Core 5.0 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 146.5 ms | 18.63 ms | 1.02 ms | - | - | - | 15.8 KB |
| 'ImageSharp Lossless Webp' | Job-GLNACU | .NET Core 5.0 | Webp/earth_lossy.webp | Webp/earth_lossless.webp | 306.7 ms | 32.31 ms | 1.77 ms | 4000.0000 | 1000.0000 | - | 22860.02 KB |
Encode
The current implementation suffers from heavy allocations but I suspect a lot of that is low hanging fruit which we can refactor to use our MemoryAllocator types. The lack of difference between NET 472 and NET 5 suggests we have some SIMD opportunities also.
BenchmarkDotNet = v0.12.1, OS = Windows 10.0.19041.630 (2004/?/20H1)
Intel Core i7-6700K CPU 4.00GHz(Skylake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK = 5.0.100
[Host] : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
Job-OUUGWL : .NET Framework 4.8 (4.8.4250.0), X64 RyuJIT
Job-GAIITM : .NET Core 2.1.23 (CoreCLR 4.6.29321.03, CoreFX 4.6.29321.01), X64 RyuJIT
Job-HWOBSO : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
| Method | Job | Runtime | TestImage | Mean | Error | StdDev | Ratio | RatioSD | Gen 0 | Gen 1 | Gen 2 | Allocated |
|--------------------------- |----------- |-------------- |------------- |----------:|-----------:|----------:|------:|--------:|-----------:|----------:|----------:|-------------:|
| 'Magick Webp Lossy' | Job-RYVNHD | .NET 4.7.2 | Png/Bike.png | 23.30 ms | 0.869 ms | 0.048 ms | 0.14 | 0.00 | - | - | - | 68.19 KB |
| 'ImageSharp Webp Lossy' | Job-RYVNHD | .NET 4.7.2 | Png/Bike.png | 68.22 ms | 16.454 ms | 0.902 ms | 0.42 | 0.01 | 6125.0000 | 125.0000 | - | 26359.49 KB |
| 'Magick Webp Lossless' | Job-RYVNHD | .NET 4.7.2 | Png/Bike.png | 161.96 ms | 9.879 ms | 0.541 ms | 1.00 | 0.00 | - | - | - | 520.28 KB |
| 'ImageSharp Webp Lossless' | Job-RYVNHD | .NET 4.7.2 | Png/Bike.png | 370.88 ms | 58.875 ms | 3.227 ms | 2.29 | 0.02 | 34000.0000 | 5000.0000 | 2000.0000 | 163177.15 KB |
| | | | | | | | | | | | | |
| 'Magick Webp Lossy' | Job-GOZXWU | .NET Core 2.1 | Png/Bike.png | 23.35 ms | 0.428 ms | 0.023 ms | 0.14 | 0.00 | - | - | - | 67.76 KB |
| 'ImageSharp Webp Lossy' | Job-GOZXWU | .NET Core 2.1 | Png/Bike.png | 43.95 ms | 2.850 ms | 0.156 ms | 0.27 | 0.00 | 6250.0000 | 250.0000 | 83.3333 | 26284.72 KB |
| 'Magick Webp Lossless' | Job-GOZXWU | .NET Core 2.1 | Png/Bike.png | 161.44 ms | 3.749 ms | 0.206 ms | 1.00 | 0.00 | - | - | - | 519.26 KB |
| 'ImageSharp Webp Lossless' | Job-GOZXWU | .NET Core 2.1 | Png/Bike.png | 335.78 ms | 78.666 ms | 4.312 ms | 2.08 | 0.03 | 34000.0000 | 5000.0000 | 2000.0000 | 162727.56 KB |
| | | | | | | | | | | | | |
| 'Magick Webp Lossy' | Job-VRDVKW | .NET Core 3.1 | Png/Bike.png | 23.48 ms | 4.325 ms | 0.237 ms | 0.15 | 0.00 | - | - | - | 67.66 KB |
| 'ImageSharp Webp Lossy' | Job-VRDVKW | .NET Core 3.1 | Png/Bike.png | 43.29 ms | 16.503 ms | 0.905 ms | 0.27 | 0.01 | 6272.7273 | 272.7273 | 90.9091 | 26284.86 KB |
| 'Magick Webp Lossless' | Job-VRDVKW | .NET Core 3.1 | Png/Bike.png | 161.81 ms | 10.693 ms | 0.586 ms | 1.00 | 0.00 | - | - | - | 523.25 KB |
| 'ImageSharp Webp Lossless' | Job-VRDVKW | .NET Core 3.1 | Png/Bike.png | 323.97 ms | 235.468 ms | 12.907 ms | 2.00 | 0.08 | 34000.0000 | 5000.0000 | 2000.0000 | 162724.84 KB |
| | | | | | | | | | | | | |
| 'Magick Webp Lossy' | Job-ZJRLRB | .NET Core 5.0 | Png/Bike.png | 23.36 ms | 0.448 ms | 0.025 ms | 0.14 | 0.00 | - | - | - | 67.66 KB |
| 'ImageSharp Webp Lossy' | Job-ZJRLRB | .NET Core 5.0 | Png/Bike.png | 40.11 ms | 2.465 ms | 0.135 ms | 0.25 | 0.00 | 6307.6923 | 230.7692 | 76.9231 | 26284.71 KB |
| 'Magick Webp Lossless' | Job-ZJRLRB | .NET Core 5.0 | Png/Bike.png | 161.55 ms | 6.662 ms | 0.365 ms | 1.00 | 0.00 | - | - | - | 518.84 KB |
| 'ImageSharp Webp Lossless' | Job-ZJRLRB | .NET Core 5.0 | Png/Bike.png | 298.73 ms | 17.953 ms | 0.984 ms | 1.85 | 0.01 | 34000.0000 | 5000.0000 | 2000.0000 | 162725.13 KB |
Activity
SebastianStehle commentedon Oct 26, 2021
Would some sponsoring help for this task? I would really like to see webp support.
JimBobSquarePants commentedon Oct 26, 2021
Sponsorship is always welcome but it’s really a matter of resources which can be provided by the community if willing. In the interim you can download preview builds from our MyGet feed.
SebastianStehle commentedon Oct 26, 2021
I am using imagesharp in a server environment. I am afraid to use it in production currently as it has the potential to bring the CPU down.
I will see if I find the time (and knowledge) to help, but I guess you are much more familiar with this low level code than must other developers. I don't know whether you work as a freelancer or employee but I guess if you would have the chance to work on this for 1-2 weeks the results would be amazing.
JimBobSquarePants commentedon Oct 26, 2021
Our MyGet build is exactly the same as our NuGet ones we just don't tag them. Things are subject to change but do go through 30,000+ unit tests. We've still got #1730 to complete before we can consider shipping V2 I'm afraid so we cannot ship a stable build until then.
Everyone contributing to Six Labors is doing so in their spare time - There's not been enough financial support from the software industry for me to be able to work full time on the projects (hopefully one day). I personally attempt to get some work done most nights between 11pm and 2am.
SebastianStehle commentedon Oct 26, 2021
Yes, I assumed that it works like this, therefore I asked if I can support the team financially by sponsoring this task, at least partially.
brianpopow commentedon Oct 30, 2021
@SebastianStehle:
Getting some donation for doing the work here would be very welcome. I think most people really underestimate how much work it really is.
It took me almost 2 years to get webp done. Of course this was most of the time in the same mode as James mentioned: nights and weekends.
I am not sure that we can achieve that much in that short time period to be honest.
Also keep in mind that encoding webp is a CPU intense task, no matter how much optimization we do (i assume that's your usecase).
I am currently working on reducing the allocations. I have an experimental branch,
which uses 1/20 of the allocations for lossy encoding and 1/2 for lossless.
I think the first step for improving the performance should be, that someone does
extensive profiling and spots places which would benefit the most from optimization.
That would already be pretty helpful.
As James mentioned, hardware intrinsics are the way to go for performance gains.
The original libwebp source contains a lot of hardware intrinsics usages.
They can be easily spotted, because the files have the suffix
sse2orsse41.I tried to keep the original function names as they are in the libwebp source.
I have ported already some of them, but there are plenty left.
8 remaining items
brianpopow commentedon Nov 2, 2021
Sorry for that closing and reopening again, same issue as before: just wanted to link an PR to this issue.
JimBobSquarePants commentedon Nov 2, 2021
@brianpopow Rather than manually linking (which always closes when PRs are merged into the default branch) you can just use the hash syntax without closing keywords.
brianpopow commentedon Nov 4, 2021
Unfortunately i noticed a big difference between the lossy encoding benchmarks posted here initially
and the current ones. The current ones look much worse, but after digging deeper into it,
it seems the original benchmarks from June 2021 were made without all features implemented.
The commit which introduced this is a368f41. I will recreate new benchmarks from the point
when we merged it into master, so we have a better baseline.
We also need to double check, if the reference decoder/encoder uses the same parameters as we are.
The results can differ very largely with different parameters, especially when encoding.
brianpopow commentedon Nov 8, 2021
I have re-run the benchmarks again for the commit: 853c40c from 23. October, which was the commit merging webp into master.
I assume we stay with the same settings proposed in #1808.
Encode:
Decode:
The biggest deviation to the original benchmarks from this thread is lossy encoding. The good news is though, I made good progress in improving that (see the open PR and there is more potential there).
brianpopow commentedon Nov 10, 2021
I wanted to share some updates on the improvements for webp.
Benchmarks from the current master branch:
Encode:
Decode:
Notable mentions:
Notes:
Its pretty complicated though. I am not sure if i can port that over from libwebp. Maybe someone with more experience SSE2/SSE4 is needed here.
cleared before using them. From profiling i would roughly guess this costs 5% performance. I still think its worth it.
CarlosTorrecillas commentedon Nov 22, 2021
Is that already available in the latest package? If so, I somehow get the UnknownImageFormatException and WEBP is still not listed there. Perhaps I'm doing something wrong but I'm not able to load a WEBP image just yet.
JimBobSquarePants commentedon Nov 22, 2021
You need to use the MyGet feed. We’ll be shipping it to NuGet with 2.0
brianpopow commentedon Dec 21, 2021
Some update in the current status.
Here are the benchmark results from the current master branch:
Encode:
Decode:
Notable mentions:
antonfirsov commentedon Feb 6, 2022
Is this general issue still valuable? = meaning shall we roll it to "Future", or close it?
brianpopow commentedon Feb 6, 2022
Since I do not see any big opportunities anymore to improve the performance further, I would vote for closing the issue.
JimBobSquarePants commentedon Feb 6, 2022
Yeah let's close it for now.