Backup (core) of NewPipe's organization

[litetex]

I recently noticed that we don't have any kind of backup of our organization in case GitHub takes them down (#youtube-dl).

Also related to #1499.

What should be backed up

I think the following things should be backed up:

Repo	Git	Issues
https://github.com/TeamNewPipe/NewPipe	✔️	✔️
https://github.com/TeamNewPipe/NewPipeExtractor	✔️	✔️
https://github.com/TeamNewPipe/nanojson	✔️	✖️
https://github.com/TeamNewPipe/website	✔️	✔️
https://github.com/TeamNewPipe/documentation	✔️	✖️ (nobody maintains these issues)
https://github.com/TeamNewPipe/team (private)	✖️	✔️
https://github.com/TeamNewPipe/legal (private)	✖️	✔️

Backup destination

For now I would target GitLab.com (→ https://gitlab.com/TeamNewPipe)

Backing up Git

Idea: Run once a week a full backup (with GH actions?) and push it to gitlab (override everything that is already there)

See also:

• https://dev.to/brunorobert/github-and-gitlab-sync-44mn

Backing up Issues

Idea: https://github.com/MichaelMure/git-bug

Requires a new repository.

Add 2 bridges:

• One for GitHub - used for polling
• One for GitLab - used for pushing

Sync changes e.g. once in a week

[ktprograms]

Why not use Codeberg or another Gitea instance?

IIRC gitlab.com uses GitLab EE, which isn't open source.

[litetex]

Status update:

• git-bug doesn't allow to sync changes from one git hosting solution to another (e.g. I can't sync issues from GitHub to GitLab) - So it's off the list. It also stores data inside git itself which might be problematic when exporting it to other platforms. Maybe it's useable as a "secondary" backup.
• The problem with Codeberg, Gitea and also GitLab is that they also don't allow syncing of issues and PRs.

→ Therefore the most likely solution will be a script that periodically imports the GitHub repo (+issues and PR) into Gitlab.

Why not use Codeberg or another Gitea instance?

• Regarding Codeberg: Not my first choice as they are have mirroring turned off and have not as much market penetration as GitLab - Could be an option in the future.
• Regarding Gitea: If I would have access to an instance this would also be possible. However setting up and maintaining an instance requires an not negligible amount of work and I don't think someone here has the will or can guarantee to do so for the next x years.

IIRC gitlab.com uses GitLab EE, which isn't open source.

As I'm just using the parts of GitLab that are also available on the CE edition, I see no harm here.

In case GitHub goes down and we want to use our own GitLab instance, we can still export the data from GitLab to our own instance or import it into Codeberg, Gitea, ...

[litetex]

Run an full import of everything last night into a test repository:
https://gitlab.com/n3490

Looks pretty good.
However I encountered some issues today:

• Some people got mails despite the repo being archived.
  Sorry for the inconvenience → I disabled notifications inside the group now explictly:
• Some issues are missing (see below)
• The import of the NewPipe repository took forever (at least 1-2h)

Project import status:

GH	GL	Git -Import	Issues -Import	PR -Import
https://github.com/TeamNewPipe/NewPipe	https://gitlab.com/n3490/NewPipe/2022-03-18	✔️	❌ ~700 issues are missing	✔️
https://github.com/TeamNewPipe/NewPipeExtractor	https://gitlab.com/n3490/NewPipeExtractor/2022-03-18	✔️	✔️	✔️
https://github.com/TeamNewPipe/nanojson	https://gitlab.com/n3490/nanojson/2022-03-18	✔️	✔️	✔️
https://github.com/TeamNewPipe/website	https://gitlab.com/n3490/website/2022-03-18	✔️	✔️	✔️
https://github.com/TeamNewPipe/documentation	https://gitlab.com/n3490/documentation/2022-03-18	✔️	✔️	✔️
https://github.com/TeamNewPipe/team (private)	https://gitlab.com/n3490/team/2022-03-18	✔️	✔️	✔️
https://github.com/TeamNewPipe/legal (private)	https://gitlab.com/n3490/legal/2022-03-18	✔️	✔️	✔️

[litetex]

Did another import this week and there are even more issues missing (https://gitlab.com/n3490/newpipe/2022-03-21).

Not sure what is happening there exactly...

Related: https://gitlab.com/gitlab-org/gitlab/-/issues/32026