What is the preferred way of doing migrations/schema changes?

[karlbohlmark]

I have started building an application on top of nohm, and recently needed to add an index to a model. This of course leaves all the entities already added of this model type unindexed.

What is your workflow for doing this, or other model changes on existing data? Maybe something to mention in the readme?

[maritz]

I've previously done it by manually loading each record and saving it again. This might work depending on your changes.

There is currently no code to handle invalid data structures in the db, thus you should probably test this stuff on a test db first.

I'll think about what I can do to handle this a bit better in the future.

[yuchi]

Sorry to cut in.

In Nohm the schemas are runtime builded models, and many pieces of it are very dynamic (eg. functions for validations and defaults). But I think Nohm could store a representational state of the properties and their types along with the records. When loading it could retrieve the state, check for differences and act on the records if told to do so. I can imagine a Nohm.migrateChanges to control this behaviour.

BTW I personally have a script which loads everything inside a big JSON and then stores it again, I can hook inside it and act on properties (when needed). It's horribly written, but I can build a better one and publish it.

[karlbohlmark]

I've also thought of building something of the kind and I asked the question to see if there already was smth in place.

Maybe smth like this (from the top of my head, possibly stupid):

• nohm:schemas is a set of links to schemas, example: nohm:schema:User
• nohm:schema:User is a list of links to schema revisions, example: nohm:schema:User:1
• nohm:schema:User:1 is a representation of the revision 1 of the "User"-schema
• nohm entities contains a reference to the schema revision
• then build migrations on top of this

what do you think?

[yuchi]

It would be awesome, but...

1. We don't have such a thing as revision in Nohm, should be done manually
2. The code necessary to do this can be huge
3. Rails migration are possible because when you build the sqls you actually do (read: rake) something. In Nohm there's no such a moment where we could hook into.

That said, a serialized state of the properties is an easy thing (a smart, ordered, JSON.strinfigy), and there are also some cool diff/patch methods out there that could help a lot (see @bruth's awesome jsonpatch, I'm sure there's somewhere a jsondiff which matches the standard).

Once the diffs are calculated building a simple migration script could also be trivial. Another idea is to use the ordered serialization output to build an hash, and use it as a fast diff matcher.

[maritz]

There is already a kind of schema system that was supposed to be used by nohm admin. That could be used but I don't know if it actually still works because there are no tests for it. I have not started doing any new work on nohm admin yet. I think once v0.9 is done I'll be able to do more on that front.
Maybe the migration stuff should be in there too.

[maritz]

There has been some work by me on something that'll maybe help. You can take a look at it over at nohm-admin. However that is still in development (although temporarily paused for at least another week) and should not be used on any database that holds serious data yet. To install, just clone it, npm install and change the config.js for your needs. Then once it is running register a new user and go to http://yourip/host:port/REST/User/give/me/admin to get admin rights. (DO NOTE: that route is there for anyone right now - for development purposes - and thus grants theoretically complete access to any db that doesn't require auth)