Migrations for the filesystem repository of Kubo IPFS nodes
These are migrations for the filesystem repository of Kubo IPFS nodes. Each migration builds a separate binary that converts a repository to the next version. The fs-repo-migrations is a tool that downloads individual migrations from the ipfs distribution site and applies them in sequence to migrate the ipfs repository to the target version. This tool is written in Go, and developed alongside kubo.
make installWhen you want to upgrade Kubo to a new version, you may need to migrate.
Here is the table showing which repo version corresponds to which Kubo version:
| ipfs repo version | Kubo versions |
|---|---|
| 1 | 0.0.0 - 0.2.3. |
| 2 | 0.3.0 - 0.3.11 |
| 3 | 0.4.0 - 0.4.2 |
| 4 | 0.4.3 - 0.4.5 |
| 5 | 0.4.6 - 0.4.10 |
| 6 | 0.4.11 - 0.4.15 |
| 7 | 0.4.16 - 0.4.23 |
| 8 | 0.5.0 - 0.6.0 |
| 9 | 0.5.0 - 0.6.0 |
| 10 | 0.6.0 - 0.7.0 |
| 11 | 0.8.0 - 0.11.0 |
| 12 | 0.12.0 - 0.17.0 |
| 13 | 0.18.0 - 0.20.0 |
| 14 | 0.21.0 - 0.22.0 |
| 15 | 0.23.0 - 0.29.0 |
| 16 | 0.30.0 - current |
Please see the migration run guide here.
Migrations are one of those things that can be extremely painful on users. At the end of the day, we want users never to have to think about it. The process should be:
- SAFE. No data lost. Ever.
- Revertible. Tools must implement forward and backward migrations.
- Frozen. After the tool is written, all code must be frozen and vendored.
- To Spec. The tools must conform to the spec.
To create a new migration, create a go module in a directory named fs-repo-X-to-Y, where X is the repo "from" version and Y the repo "to" version. Vendor the module's dependencies. The build tooling will find this module and build the migration binary.
If the migration directory contains a subdirectory named sharness, tests contained in it are run using the sharness test tool. Tests must be named tNNNN-*.sh, where NNNN is a 4-digit sequence number.
After the migration is merged into the main repo branch, create a version tag for it. This is necessary for versioning individual migrations within the repo.
git tag <migration>/v<version>
git push origin <migration>/v<version>Example:
git tag fs-repo-99-to-100/v1.0.1
git push origin fs-repo-99-to-100/v1.0.1We "archive" legacy migrations by adding them to ignored-migrations file.
This keeps CI fast, and removes issues caused by old, with unsupported go code.
Dependencies must be vendored independently for each migration. Each migration is a separate go module with its own vendor directory (created with go mod vendor for that migration). All migrations are built using go build -mod=vendor to ensure dependencies come from the module's vendor directory.
If IPFS plugins were used to operate your IPFS datastore, such as the ipfs-ds-s3 plugin, then migration may require building a custom migration with the plugin built into it. There is a script to assist with the process: build-plugin.sh.
This requires that you have Go installed.
First clone the fs-repo-migrations github repo:
git clone https://github.com/ipfs/fs-repo-migrations.gitThen run the build-plugin.sh script, supplying the necessary arguments (run with -h for help).
After the custom migration with plugin(s) has built successfully, change to the migration directory and run the migration binary. You can also copy the migration binary into your PATH if you want it to be run by ipfs-update or by the fs-repo-migrations tool.
Feel free to join in. All welcome. Open an issue!
This repository falls under the IPFS Code of Conduct.
MIT