move lfs to S3 (minio)

[projectDaemon]

It would be nice to have minio backend for lfs

[sapk]

Is it to store lfs data via gitea in minio or use minio as external lfs backend ?

I haven't tested but external lfs backend can be set manually via :

git config -f .lfsconfig lfs.url https://my_other_server.example.com/foo/bar/info/lfs
git add .lfsconfig

[lunny]

I think he means store lfs files to s3 storage server but not local disk.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

[segevfiner]

Rather than the minio library, which only supports S3 like providers, we can choose a library that supports multiple different cloud storage services (And S3 clones), wrapping it's configuration in Gitea's configuration. And if that library also supports local files, even better, as there will be no need to maintain two implementations. The local file implementation will have to scale properly, that is, support splitting the objects into subdirectories as currently done. And there is also the issue of migrating existing data if needed.

This needs to be something that the Gitea developers are comfortable using and having a proper license. Here are a few options:

• https://github.com/google/go-cloud
• https://github.com/graymeta/stow
• https://github.com/lytics/cloudstorage

(There may be more)

The configuration format for this is highly affected by the chosen library.

There is also.the issue of what key format to use. Cloud storage doesn't need the subdirectory splitting trick, though it probably doesn't hurt them either, and still using it there might allow easier migration using a simple recursive cp to the cloud storage provider.

I would love to hear Gitea's maintainers take on this.

[clarfonthey]

I agree that having some sort of object storage support for LFS would be really nice. S3-compatible APIs are one option, but really, having a generic API that allows us to plug in any cloud provider would be better.

[i0x71]

Was there any progress on this ?
This would be a killer feature.
If you could store all git data on minio that would be even better.
Gitea combined with minio distributed amd fault tolerant setup would be awesome.

[jolheiser]

I believe this would be taken care of by #9567

[i0x71]

Awesome, thanks

[colorfulberry]

How about this going? Any plan to merge or not

[lunny]

See my PR #11387

[colorfulberry]

@lunny Cool, Thanks