Save Without References

[pcen]

builds on #423

Discussions

• saves: C7 Save Structure Proposal #377
• entity IDs: Object ID System #210

There are numerous gameplay features that can be implemented, but are blocked or otherwise unnecessarily difficult to implement due to the lack of a workable save file format. Included in this set of features are #102, #103, and #148, which are the remaining gameplay features for Carthage. These features all requiring adding new information to the save file in order to implement. Implementing these features is difficult because the current save file format is unwieldy for the following reasons:

• a basic new game save file is large: ~500k lines of json
• references between C# objects serialised to JSON are generated automatically, so it's unclear where in the structure the actual definition of the object exists
• objects have ad-hoc IDs (ie. Players have string GUIDS) but other objects usually hold references to them in C#, making the GUID meaningless in the save file

Another future potential downsides of the naive JSON serialisation approach is for save editors and modding. For save editors, the GameData class is a huge blob of mutable state, and any editor function must ensure that the changes it applies will permeate the entire GameData class. This already must be the case for things that can happen in game (ie. when a new city is built, we already must ensure it is added to the Tile, the Player's cities, etc.) but to add new features to the editor would also need to modify all of GameData correctly (the implementation overhead is O(the amount of state GameData holds)).

I've followed the principles @QuintillusCFC laid out here: #377. Save files are directly deserialised into SaveGame which in turn creates a GameData. GameData is first converted into a SaveGame and then serialised directly to the save file - there are no C# references in serialisation or deserialisation.

1. e pluribus unum: the SaveGame class has the same structure as the JSON file it serialises to, and no classes contained in SaveGame references any other classes in SaveGame - it is a tree of objects. Any cyclic references are stored as an ID
2. easy to understand for human readers: everything in the c7-static-map-save.json is human readable, references are incrementing integers prefixed with human readable strings.
3. polymorphic classes - not solved in this PR, but since SaveGame is still being serialised to JSON any solution that could be applied to the current save format can also be implemented for SaveGame

This save format also addresses another issue that requires manual work whenever new fields are added to GameData - the save needs to be regenerated from a .sav file, and any existing c7 save files cannot be salvaged due to the serialised C# references, and the fact that there is no good way to handle newly added or removed fields when using dotnet or newtonsoft json. With a value-based SaveGame format, the backwards compatibility is much easier to solve for existing save games. Since everything in the SaveGame format is "plain old data", if a new field is added, old save file formats can still be parsed, and then upgraded to the newer save format by an UpgradeSaveGameFromVersionAToVersionB (this process can be chained indefinitely for a c7 save that is n versions older than the current, by sequentially running the SaveGame through n UpgradeSaveGame... functions), while the actual GameData definition is not tied to any backward compatibility concerns.

[Sean-Brown]

indentation should be preserved

[Sean-Brown]

Why is this variable necessary/how is the variable intended to be used? Wouldn't the game just see if the player has unfortified/unskipped units with movement left and, if not, just allow the player to end turn?

[pcen]

hasPlayedCurrentTurn may be true when a player has remaining movement points but ends their turn early, which I don't think can be inferred from other state and needs to be stored directly

[Sean-Brown]

I see, such as when hitting "shift+enter"

[Sean-Brown]

does this pass? I checked out the branch and tried loading the first Conquest and it throws an exception (ERROR: System.NullReferenceException: Object reference not set to an instance of an object.), which makes the scen not load

[pcen]

I'll look into this and see if it's passing locally

[Sean-Brown]

The changes look logical to me, however I can't load any of the conquests or scenarios like the MPTournament scen... maybe that's a "me" issue?

[QuintillusCFC]

Thoughts (which may become obsolete as I learn more):

• I like the concept, separating it out and making it updateable
• The new JSON format indeed reads easily
• How will the TilePath work with serialization? (not that important but TBD)
• Should the TODO in CreateGame.cs be its own issue (or does the next PR address it?)
• Have reviewed the save manager, SaveGame.cs, and all the small files (not yet SaveMap, SavePlayer, etc.). I'm increasingly confident this is both the way to go and that the code itself will work.

Going to look at a bit more once my battery isn't near-empty, and see if it can be merged into the Godot 4 branch.

[pcen]

@QuintillusCFC I checked the other PR I have that branches off of this branch, and it looks like I did not address the TODO in CreateGame.cs, so it should be a separate issue.