Model state format

[dmasad]

At the moment, the state of the model at any given time is captured by the model object itself, and by the agent objects under it. These are all dynamic, and change with every activation. The data collector stores a subset of the model state values, and the server generates a static data structure that's tied to the specific visualization.

An idea for an alternative approach is defining a static model state data structure. This could be as simple as a JSON-type object which stores (by default) all model- and agent-level variable values at the time its called. This could supplement the current DataCollector, and potentially provide a standardized data format to send the client side of the visualization. It would also make it easier to log and store an entire run for later analysis, or for scrubbing forward and back video-style. The downside is that JSON specifically is heavy with lots of redundancies (especially if we store each agent's unique_id with hundreds+ of agents and hundreds+ of steps). That might make sending data to the browser slower. Even a more efficient format might slow models down by collecting and storing a lot of redundant or unnecessary data.

[jackiekazil]

This crossed my mind too, when I was commenting on the other ticket, but I just saw this. I feel like this is something that should evaluated and then clearly compared. Right now I am not sure I see how all the different approaches work without making it confusing to users why things might exist.

[Corvince]

I came across this yesterday when I thought about how to store my various model runs. While a truly static data structure would be quite nice, I think it will require quite some effort and I am not sure how to approach this.

An intermediate and/or complementary solution could be to simply store the model state as a pickle. I played around a little and it seems to be ridiculously efficient. The actual overhead seems to be about the same we had before merging #576 , but of course it doesn't just store a bit of data, but the whole model.

So we could for example change the visualization server to keep a state dictionary like state[step] = pickle.dumps(model) and then scrub forward and backward in time by simply de-serializing the model state at a step and sending the relevant attributes. What would be really interesting about this that we could do things like going back in time, change some parameters and see how the model would have evolved. I think this is something I haven't seen in any ABM framework.

[quaquel]

With #2291 closed, this is now trivial to implement if desired.