Safety of ToCatchup?

[yonderblue]

In the commit a8bcd23 a number of years ago it was added to fill in the holes of steps that were not run even though the last migration is past already. How is this safe considering migration steps are not independent?

[rubenv]

It does a down step first on the later migrations.

…

On Tue, Jun 22, 2021, 16:53 Yonder Blue ***@***.***> wrote: In the commit a8bcd23 <a8bcd23> a number of years ago it was added to fill in the holes of steps that were not run even though the last migration is past already. How is this safe considering migration steps are not independent? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#196>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAKPGHJIIF5VJEILEPM7S3TUCPX5ANCNFSM47D3QF3A> .

[yonderblue]

I don't see that, would you mind pointing me to a spot in the code? Looks like PlanMigration() only has Up steps in the tests?

[rubenv]

It's right in the commit you linked to :-) Check the unit tests.

…

On Tue, Jun 22, 2021, 17:09 Yonder Blue ***@***.***> wrote: I don't see that, would you mind pointing me to a spot in the code? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#196 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAKPGA4KRDMZPFTMICRKCTTUCRUNANCNFSM47D3QF3A> .

[yonderblue]

For the case of calling PlanMigration with Up and 0, I see:

// apply all the missing migrations
	plannedMigrations, _, err := PlanMigration(s.Db, "sqlite3", migrations, Up, 0)
	c.Assert(err, IsNil)
	c.Assert(plannedMigrations, HasLen, 3)
	c.Assert(plannedMigrations[0].Migration.Id, Equals, "2")
	c.Assert(plannedMigrations[0].Queries[0], Equals, up)
	c.Assert(plannedMigrations[1].Migration.Id, Equals, "4")
	c.Assert(plannedMigrations[1].Queries[0], Equals, up)
	c.Assert(plannedMigrations[2].Migration.Id, Equals, "5")
	c.Assert(plannedMigrations[2].Queries[0], Equals, up)

which only outputs the holes going up. I don't see it going down unless you ask it to go down in the other two cases in that test?

[alexdavid]

IMO this should be an option since running down migrations to the holes isn't always safe (suppose a column is added and has real production data, the down + up migrations will lose that data).

I'm happy to submit a PR to make this functionality optional.

[rubenv]

IMO this should be an option since running down migrations to the holes isn't always safe

This can really only happen if you don't have a deployment process in place. I understand your concern, but normally this will only apply in development (where multiple people may contribute migrations at the same time).

If your production environment is being deployed to from multiple sources, with multiple different versions (as in: not a linear history), well then you've got much bigger problems on your plate.

I might be overlooking something though, so happy to be proven wrong!

[alexdavid]

Multiple environments does not solve the problem. It can potentially make the problem less likely to show up, but it is still entirely possible for a feature that introduces a migration to sit in PR for longer than a deploy cycle. If it's not impactful enough to raise red flags when migrating on pre-production environments it could blow out real data in production.

The ability to blow up on out of order migrations is a feature to inform the team that a deploy isn't business as usual and should at the very least be manually investigated.