Convert avatars to WEBP during upload

[silverwind]

Feature Description

As discussed in #24248 (comment) and below, we should evaluate converting and optimizing all avatars to WEBP. Currently all input formats (WEBP,PNG,JPG,GIF) end up as PNG during the avatar processing which squares them.

Notably, animation via APNG or WEBP input would be nice to have as well because WEBP can do animation, so even APNG should be convertable. Currently this avatar convertion strips APNG animation, would be nice if it can be preserved.

Related old issue: #18907

[silverwind]

Also, trying to upload a animated WEBP image currently gives this error, likely because the conversion code can not deal with animated WEBP correctly:

[zeripath]

Squaring

hmm... as far as I can see the squaring process is unrelated to the choice of format.

If you take a look at the code for Prepare:

gitea/modules/avatar/avatar.go

Lines 47 to 86 in 23ae939

	func Prepare(data []byte) (*image.Image, error) {
	imgCfg, _, err := image.DecodeConfig(bytes.NewReader(data))
	if err != nil {
	return nil, fmt.Errorf("DecodeConfig: %w", err)
	}
	if imgCfg.Width > setting.Avatar.MaxWidth {
	return nil, fmt.Errorf("Image width is too large: %d > %d", imgCfg.Width, setting.Avatar.MaxWidth)
	}
	if imgCfg.Height > setting.Avatar.MaxHeight {
	return nil, fmt.Errorf("Image height is too large: %d > %d", imgCfg.Height, setting.Avatar.MaxHeight)
	}
	img, _, err := image.Decode(bytes.NewReader(data))
	if err != nil {
	return nil, fmt.Errorf("Decode: %w", err)
	}
	if imgCfg.Width != imgCfg.Height {
	var newSize, ax, ay int
	if imgCfg.Width > imgCfg.Height {
	newSize = imgCfg.Height
	ax = (imgCfg.Width - imgCfg.Height) / 2
	} else {
	newSize = imgCfg.Width
	ay = (imgCfg.Height - imgCfg.Width) / 2
	}
	img, err = cutter.Crop(img, cutter.Config{
	Width: newSize,
	Height: newSize,
	Anchor: image.Point{ax, ay},
	})
	if err != nil {
	return nil, err
	}
	}
	img = resize.Resize(AvatarSize, AvatarSize, img, resize.Bilinear)
	return &img, nil
	}

The squaring happens because of:

gitea/modules/avatar/avatar.go

Lines 64 to 84 in 23ae939

	if imgCfg.Width != imgCfg.Height {
	var newSize, ax, ay int
	if imgCfg.Width > imgCfg.Height {
	newSize = imgCfg.Height
	ax = (imgCfg.Width - imgCfg.Height) / 2
	} else {
	newSize = imgCfg.Width
	ay = (imgCfg.Height - imgCfg.Width) / 2
	}
	img, err = cutter.Crop(img, cutter.Config{
	Width: newSize,
	Height: newSize,
	Anchor: image.Point{ax, ay},
	})
	if err != nil {
	return nil, err
	}
	}
	img = resize.Resize(AvatarSize, AvatarSize, img, resize.Bilinear)

Scaling up

Similarly the increase in size occurs because of the call to resize.

gitea/modules/avatar/avatar.go

Line 84 in 23ae939

img = resize.Resize(AvatarSize, AvatarSize, img, resize.Bilinear)

Exporting as png

Exporting as png appears to occur in two places:

gitea/services/repository/avatar.go

Line 50 in 23ae939

if err := png.Encode(w, *m); err != nil {

gitea/services/user/user.go

Line 265 in 23ae939

if err := png.Encode(w, *m); err != nil {

These would both need to change to use webp.

We could probably change the code in generateRandomAvatar to use webp too:

gitea/models/repo/avatar.go

Line 51 in 23ae939

if err := png.Encode(w, img); err != nil {

(It does not look like:

gitea/tests/integration/user_avatar_test.go

Line 55 in 23ae939

if err := png.Encode(imgData, img); err != nil {

would need to change)

Animation

Unfortunately the golang x/image/png library does not support animation. There has been a feature request:

golang/go#53364

but no action has been taken.

There are other libraries that use cgo to bind to libwebp and should provide animation support but we would need to change from using image.Decode to some other function and it would need to detect if there is animation present.

Gitea at present will probably just take the first frame of an animated gif. So I think animation is going to take a bit of work.

Issues with Scaling and Resize

As far as I can see resize.Resize (and in some cases cutter.Crop) will convert the avatar into image.RGBA which decompresses the image and may make it quite large - taking up memory. This does not seem quite ideal.

So I think what we need to do is to rethink about what the avatar code is supposed to do.

1. Should we be doing any of this resizing and cropping at all? Should we simply be enforcing our max height and width and storing like that?
2. If we do resizing/cropping should we only resize/crop when the image is bigger than the avatar size?
3. Should we store as webp?
4. Should we look at being able to store animations?

[silverwind]

Should we be doing any of this resizing and cropping at all? Should we simply be enforcing our max height and width and storing like that?

I don't think we need to square or crop. Browser rendering can and should take care of letterboxing the image. We can certainly ensure this in our rendering. If other sites that embed gitea avatars assume our avatars are always square, they will need to fix it themselves.

Should we store as webp?

Yes, browser support is good enough that we can store a well-compressed webp.

Should we look at being able to store animations?

Animations are nice to have, but not a requirement for me. Happy to defer this to golang/go#53364.

[sosasees]

Should we look at being able to store animations?

this is not needed.
animated avatars can be fun on a casual website,
but on a serious website like Gitea they would be just annoying.

[silverwind]

Should we be doing any of this resizing

I guess resizing could be considered above a certain pixel count. Let's say a lazy user directly uploads a photo taken from a digital camera which is let's say 50 Megapixels. We'd definitely want to downsize that to not destroy rendering performance and bandwidth consumption for other users.

I would say we could use a Megapixel constant above which to downsize the image to something more suitable for web consumption. Maybe make it 1 Megapixel, which is a 1000x1000 image that is suitable for fine rendering on 4dppx devices down to 250px width/height, which I think is close to avatars rendered on user pages.

[zeripath]

OK looking at encoding directly to webp we can't use x/image/webp and would need to use something that wraps the libwebp C library or write a PR to x/image/webp to do the encoding.

The first option I came across: https://github.com/kolesa-team/go-webp requires that I install libwebp-dev on my system - this would likely cause a significant issue with compiling on Windows.

PATCH for this option - would need changes to builders to install libwebp-dev though

diff --git a/go.mod b/go.mod
index 24765df18..e9e71a27e 100644
--- a/go.mod
+++ b/go.mod
@@ -218,6 +218,7 @@ require (
 	github.com/josharian/intern v1.0.0 // indirect
 	github.com/kevinburke/ssh_config v1.2.0 // indirect
 	github.com/klauspost/pgzip v1.2.6 // indirect
+	github.com/kolesa-team/go-webp v1.0.4 // indirect
 	github.com/kr/pretty v0.3.1 // indirect
 	github.com/kr/text v0.2.0 // indirect
 	github.com/libdns/libdns v0.2.1 // indirect
diff --git a/go.sum b/go.sum
index 62c01c690..269d46d55 100644
--- a/go.sum
+++ b/go.sum
@@ -792,6 +792,8 @@ github.com/klauspost/pgzip v1.2.5/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQ
 github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU=
 github.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
 github.com/kljensen/snowball v0.6.0/go.mod h1:27N7E8fVU5H68RlUmnWwZCfxgt4POBJfENGMvNRhldw=
+github.com/kolesa-team/go-webp v1.0.4 h1:wQvU4PLG/X7RS0vAeyhiivhLRoxfLVRlDq4I3frdxIQ=
+github.com/kolesa-team/go-webp v1.0.4/go.mod h1:oMvdivD6K+Q5qIIkVC2w4k2ZUnI1H+MyP7inwgWq9aA=
 github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
 github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
 github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
@@ -1334,6 +1336,7 @@ golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EH
 golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
 golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
 golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
+golang.org/x/image v0.0.0-20210628002857-a66eb6448b8d/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM=
 golang.org/x/image v0.7.0 h1:gzS29xtG1J5ybQlv0PuyfE3nmc6R4qB73m6LUUmvFuw=
 golang.org/x/image v0.7.0/go.mod h1:nd/q4ef1AKKYl/4kft7g+6UyGbdiqWqTP1ZAbRoV7Rg=
 golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
diff --git a/models/repo/avatar.go b/models/repo/avatar.go
index a76a94926..ac961989a 100644
--- a/models/repo/avatar.go
+++ b/models/repo/avatar.go
@@ -6,7 +6,6 @@ package repo
 import (
 	"context"
 	"fmt"
-	"image/png"
 	"io"
 	"net/url"
 	"strings"
@@ -16,6 +15,9 @@ import (
 	"code.gitea.io/gitea/modules/log"
 	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/storage"
+
+	"github.com/kolesa-team/go-webp/encoder"
+	"github.com/kolesa-team/go-webp/webp"
 )
 
 // CustomAvatarRelativePath returns repository custom avatar file path.
@@ -48,7 +50,12 @@ func generateRandomAvatar(ctx context.Context, repo *Repository) error {
 	repo.Avatar = idToString
 
 	if err := storage.SaveFrom(storage.RepoAvatars, repo.CustomAvatarRelativePath(), func(w io.Writer) error {
-		if err := png.Encode(w, img); err != nil {
+		options, err := encoder.NewLossyEncoderOptions(encoder.PresetDefault, 75)
+		if err != nil {
+			log.Error("Unable to create encoder options: %v", err)
+			return err
+		}
+		if err := webp.Encode(w, img, options); err != nil {
 			log.Error("Encode: %v", err)
 		}
 		return err
diff --git a/services/repository/avatar.go b/services/repository/avatar.go
index 74e5de877..9c6dd777a 100644
--- a/services/repository/avatar.go
+++ b/services/repository/avatar.go
@@ -6,7 +6,6 @@ package repository
 import (
 	"context"
 	"fmt"
-	"image/png"
 	"io"
 	"strconv"
 	"strings"
@@ -16,6 +15,9 @@ import (
 	"code.gitea.io/gitea/modules/avatar"
 	"code.gitea.io/gitea/modules/log"
 	"code.gitea.io/gitea/modules/storage"
+
+	"github.com/kolesa-team/go-webp/encoder"
+	"github.com/kolesa-team/go-webp/webp"
 )
 
 // UploadAvatar saves custom avatar for repository.
@@ -47,7 +49,12 @@ func UploadAvatar(ctx context.Context, repo *repo_model.Repository, data []byte)
 	}
 
 	if err := storage.SaveFrom(storage.RepoAvatars, repo.CustomAvatarRelativePath(), func(w io.Writer) error {
-		if err := png.Encode(w, *m); err != nil {
+		options, err := encoder.NewLossyEncoderOptions(encoder.PresetDefault, 75)
+		if err != nil {
+			log.Error("Unable to create encoder options: %v", err)
+			return err
+		}
+		if err := webp.Encode(w, *m, options); err != nil {
 			log.Error("Encode: %v", err)
 		}
 		return err
diff --git a/services/user/user.go b/services/user/user.go
index d52a2f404..1b379c025 100644
--- a/services/user/user.go
+++ b/services/user/user.go
@@ -6,7 +6,6 @@ package user
 import (
 	"context"
 	"fmt"
-	"image/png"
 	"io"
 	"time"
 
@@ -25,6 +24,9 @@ import (
 	"code.gitea.io/gitea/modules/storage"
 	"code.gitea.io/gitea/modules/util"
 	"code.gitea.io/gitea/services/packages"
+
+	"github.com/kolesa-team/go-webp/encoder"
+	"github.com/kolesa-team/go-webp/webp"
 )
 
 // RenameUser renames a user
@@ -262,7 +264,12 @@ func UploadAvatar(u *user_model.User, data []byte) error {
 	}
 
 	if err := storage.SaveFrom(storage.Avatars, u.CustomAvatarRelativePath(), func(w io.Writer) error {
-		if err := png.Encode(w, *m); err != nil {
+		options, err := encoder.NewLossyEncoderOptions(encoder.PresetDefault, 75)
+		if err != nil {
+			log.Error("Unable to create encoder options: %v", err)
+			return err
+		}
+		if err := webp.Encode(w, *m, options); err != nil {
 			log.Error("Encode: %v", err)
 		}
 		return err

https://github.com/chai2010/webp appears to wholesale import the libwebp source - which should work but we'd have to keep on top of ensuring that it's up to date.

https://git.sr.ht/~jackmordaunt/go-libwebp is a Ccgo port of libwebp to go - which could work without cgo - however, the library is marked experimental so is questionable.

[silverwind]

https://github.com/chai2010/webp sounds like the best bet to me, but it still needs CGO, right? I thought we could get rid of CGO eventually, so I doubt that adding a new CGO dependency will be worth it.

I guess as a first step, we can go and alter the resizing/cropping mechanism as discussed while still outputting well compressed PNGs, which in turn would solve #8972.