Migration framework for versioning of JS IPFS Repo
This package takes inspiration from similar tool used by Go-IPFS: fs-repo-migrations
???
As JS-IPFS evolves and new technologies, algorithms and data structures are being incorporated it is necessary to enable users easy transition between versions. Different versions of JS-IPFS may expect different structure or content of the IPFS repo (see: IPFS repo spec, JS implementation ). For that reason IPFS repo is versioned and this package provides framework to create migrations which transits one version of IPFS repo into next/previous one.
This framework provides:
- Handles locking/unlocking of repository
- Define migrations API
- Executes and reports migrations in both direction: forward and backward
- Simplify creation of new migrations
> npm install ipfs-repo-migrationsconst migrations = require('ipfs-repo-migrations')const migrations = require('ipfs-repo-migrations')Example:
const migrations = require('ipfs-repo-migrations')
const getVersion = require('ipfs-repo-migrations/repo/version')
const repoPath = 'some/repo/path'
const repoVersion = await getVersion(repoPath)
if(repoVersion < migrations.getLatestMigrationVersion()){
// Old repo! Lets migrate to latest version!
await migrations.migrate(repoPath)
}All migrations are placed in /migrations folder. Each folder there represents one migration that behaves as stand-alone
package that has its own package.json file that states migration's dependencies, has its own tests and follows migration
API.
All migrations are collected in /migrations/index.js, which should not be edited manually is it is regenerated on
every run of jsipfs-migrations add (or the manual changes should follow same style of modifications).
The order of migrations is important and the migrations have to be sorted in the growing order.
Each migration has to follow this API. It has to export object in its index.js that has following properties:
version(int) - Number that represents the version into which will be the repo migrated to (eq.migration-8will move the repo into version 8).description(string) - Brief description of what the migrations does.reversible(bool) - Specify if it is possible to revert this migration.migrate(function) - Function that on execution will perform the migration, see signature of this function bellow.revert(function) - Ifreversible == Truethen this function will be used to revert the migration to previous version.
Do not confuse this function with the require('ipfs-repo-migrations').migrate() function that drives the whole migration process!
Arguments:
repoPath(string) - absolute path to the root of the repooptions(object, optional) - object containingIPFSRepooptions, that should be used to construct datastore instance.isBrowser(bool) - indicates if the migration is run in browser environment in oppose to NodeJS
Do not confuse this function with the require('ipfs-repo-migrations').revert() function that drives the whole backward migration process!
Arguments:
repoPath(string) - path to the root of the repooptions(object, optional) - object containingIPFSRepooptions, that should be used to construct datastore instance.isBrowser(bool) - indicates if the migration is run in browser environment in oppose to NodeJS
Migration might need to distinguish in what environment it runs (browser vs. NodeJS), for this reason there is the argument
isBrowser passed to migrations functions. But with simple migrations it should not be necessary to distinguish between
these environments as datastore implementation will handle the main differences.
There are currently two main datastore implementations:
datastore-fsthat is backed by file system and is used mainly in NodeJS environmentdatastore-levelthat is backed by LevelDB and is used mainly in browser environment
Both implementations share same API and hence are interchangeable.
When the migration is run in browser environment the datastore-fs is automatically replaced with datastore-level even
when it is directly imported (require('datastore-fs') will return datastore-level in browser). Because of this mechanism
with simple migrations you should not worry about difference between datastore-fs and datastore-level.
The recommended way to write a new migration is to first bootstrap an dummy migration using the CLI:
> jsipfs-migrations addAfterwards new folder is created with bootstrapped migration. You can then simply fill in the required fields and write the rest of migration!
Don't forget to update the Migration matrix in the README.md!
The node_modules of the migration should be committed to the repo to ensure that the dependencies are resolved even in
far future, when the package might be removed from registry.
If migration affects working of any of the following functionality, it has to provide tests for the following functions to work under the version of the repo that it migrates to:
/src/repo/version.js:getVersion()- retrieving repository's version/src/repo/lock.js:lock()- locking repository that uses file system/src/repo/lock-memory.js:lock()- locking repository that uses memory
Migration has to have a test coverage. Tests for migration should be placed in /test/migrations/ folder. Most probably
you will have to plug the tests into browser.js/node.js if they require specific bootstrap on each platform.
For inter-operable reasons with Go-IPFS it might be necessary just to bump a version of repo without any actual modification as there might not be any changes needed in JS implementation. For that you can create "empty migration".
The easiest way is to use the CLI for that:
> jsipfs-migrations add --emptyThis will create empty migration with next version in line.
| IPFS repo version | JS IPFS version |
|---|---|
| 7 | v0.0.0 - v0.35.0 |
| 8 | v0.35.0 - latest |
Executes forward migration to specific version or if not specified to the latest version.
Arguments:
path(string, mandatory) - path to the repo to be migratedtoVersion(int, optional) - version to which the repo should be migrated to. If left out the version of latest migration is used.ignoreLock(bool, optional) - if true won't lock the repo for applying the migrations. Use with caution.options(object, optional) - options that are passed to migrations, that use them to correctly construct datastore. Options are same like for IPFSRepo.progressCb(function, optional) - callback that is called after finishing execution of each migration to report progress.isDryRun(bool, optional) - flag that indicates if it is a dry run that should imitate running migration without actually any change.
Signature of the progress callback.
Arguments:
migration(object) - object of migration that just successfully finished running. See Architecture of migrations for details.counter(int) - current number of migration in the planned migrations streak.totalMigrations(int) - total count of migrations that are planned to be run.
Executes backward migration to specific version.
Arguments:
path(string, mandatory) - path to the repo to be revertedtoVersion(int, mandatory) - version to which the repo should be reverted to.ignoreLock(bool, optional) - if true won't lock the repo for applying the migrations. Use with caution.options(object, optional) - options that are passed to migrations, that use them to correctly construct datastore. Options are same like for IPFSRepo.progressCb(function, optional) - callback that is called after finishing execution of each migration to report progress.isDryRun(bool, optional) - flag that indicates if it is a dry run that should imitate running migration without actually any change.
Return the version of latest migration.
The package comes also with CLI that is exposed as NodeJS binary with name jsipfs-repo-migrations.
It has several commands:
migrate- performs forward migration to specific or latest version.revert- performs backward migration to specific version.status- check repo for migrations that should be run.add- bootstraps new migration.
For further details see the --help pages.
There are some ways you can make this module better:
- Consult our open issues and take on one of them
- Help our tests reach 100% coverage!
This repository falls under the IPFS Code of Conduct.
