Refactor embedded assets and drop unnecessary dependencies

[wxiaoguang]

Benefits:

1. smaller binary size (reduces more than 1MB)
2. better control of the assets details
3. fewer unmaintained dependencies
4. faster startup if the assets are not needed
5. won't hang up editors when open "bindata.go" by accident

[silverwind]

How does this work on a high level? Put gzipped assets into go:embed?

Where does the size benefit come from?

[wxiaoguang]

How does this work on a high level? Put gzipped assets into go:embed?

See assetfs.GenerateEmbedBindata

Where does the size benefit come from?

Same as above, now:

1. we use gzip's "best compression" (just saw that you also submitted the "best compression" PR in upstream)
2. we use our own bindata format, which is more efficient
3. no duplicate generated code, all bindata packages share the same "assetfs" code

[silverwind]

What's the benefit over using something more established like https://github.com/vearutop/statigz? I guess we would need a webpack gzip plugin, but might still be worth it.

Also, I wonder whether we could eliminate the go:generate step completely somehow.

[wxiaoguang]

What's the benefit over using something more established like https://github.com/vearutop/statigz? I guess we would need a webpack gzip plugin, but might still be worth it.

It will bloat the binary about 20MB+

Also, I wonder whether we could eliminate the go:generate step completely somehow.

Not this PR's scope.

[silverwind]

It will bloat the binary about 20MB+

If we make webpack write the .gz files and delete the original output files, I guess output size should be pretty similar.

[wxiaoguang]

It will bloat the binary about 20MB+

If we make webpack write the .gz files and delete the original assets, I guess output size should be pretty similar.

There is no feasible plan at the moment, see the discussions in #26533

[wxiaoguang]

Any other questions?

[wxiaoguang]

$ ./gitea embedded extract --dest-dir ./test templates/install.tmpl
Extracting to ./test:
test/templates/install.tmpl

$ cat test/templates/install.tmpl
{{template "base/head" .}}
<div role="main" aria-label="{{.Title}}" class="page-content install">
	<div class="ui grid install-config-container">

[hiifong]

Please ignore my comment, the command parameters I executed are incorrect

[wxiaoguang]

Hmm, found another problem with **, will fix in a new commit.

[wxiaoguang]

Hmm, found another problem with **, will fix in a new commit.

Made more fixes in 9981619

[silverwind]

We could consider replacing gzip with zstd for more size savings and better page loading performance. 71% of browsers globally support content-encoding: zstd.

zstd encoding currently needs a third-party module, but it's very likely that it will be added in golang stdlib at some point: golang/go#62513.

[wxiaoguang]

We could consider replacing gzip with zstd for more size savings and better page loading performance.

Then the key point is not here, but the "gzip middleware"

71% of browsers globally support content-encoding: zstd.

IMO it is still new, the supported browsers were released in recent year

zstd encoding currently needs a third-party module, but it's very likely that it will be added in golang stdlib at some point: golang/go#62513.

TBH I don't have any expectations for Golang's feature, when it is ready, we use it; if it isn't ready, we use others.

[silverwind]

Then the key point is not here, but the "gzip middleware"

In any case, now that we have all this compression code in first-party, it should be easy to replace the encoding.

IMO it is still new, the supported browsers were released in recent year

I think the value will be > 80% by the time 1.25 releases.

TBH I don't have any expectations for Golang's feature, when it is ready, we use it; if it isn't ready, we use others.

Me neither, I expect the go feature earliest in a year or two. But it's still good to know that we can eventually eliminate the third-party dependency.

[silverwind]

I think the value will be > 80% by the time 1.25 releases.

Maybe not, given that Safari lacks support: https://caniuse.com/zstd. But I guess the usage share of Safari users of gitea may be lower then the global average 😆.