Most savegames can not be loaded and crash the game

[ManDay]

With the latest release (from source) this applies to a good 80% of the saves I have made. To reproduce:

• Start the game
• Select "Load Game"
• Select one of the provided examples
• Click "Load"

Observed:
After a few moments, a message appears in the lower right corner "This saved game cannot be loaded because it contains errors" for a split second, then the game closes with a message "Info: Pi shutting down"

Additionally (maybe a separate bug?), to reproduce:

• Start the game
• Select "Options"
• Click "Debug info"
• Select "Load Game"
• Select one of the provided examples
• Click "Load"

Observed:
The game segfaults.

Additional information: There is a clock skew in the system time (CMOS battery dead), I don't expect this to be related.

savefiles.zip

[DynV]

Perhaps #6047 is related. I can't play like that and I reverted to previous official version.

[Gliese852]

So, when loading the game, an exception is thrown here:

pioneer/src/lua/LuaSerializer.cpp

Lines 292 to 293 in 89d503a

	if (lua_isnil(l, -1))
	throw SavedGameCorruptException();

when the serializer reads this element from savegame

                  "utility_s1_1",
                  {
                    "lua_class": "Equipment.BodyScannerType",
                    "ref": 93950602298560
                  },

Since this is an empty ref, the object was supposed to be defined earlier, but this did not happen. This object is defined in the ScanManager component, but it will be processed later. If remove this piece from the savegame, it is successfully loaded.

Thus, the necessary "FIFO" condition is violated somewhere while serializing such objects. Perhaps the unordered nature of the Lua tables is involved there. It fired when the component became 3 - CargoManager, ScanManager and EquipSet.

I think there are 2 solutions - either restore "FIFO", or remove the "FIFO" requirement.

@DynV Yes, the savegame "tmp13" from #6047 does not work for the same reason.

stacktrace on exception:

#0  __cxxabiv1::__cxa_throw (obj=0x55556da08a10, tinfo=0x555555e835a0 <typeinfo for SavedGameCorruptException>, dest=0x0) at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:80
#1  0x0000555555a955bb in LuaSerializer::unpickle_json (l=0x555556f564b0, value=...) at ~/src/pioneer/src/lua/LuaSerializer.cpp:293
#2  0x0000555555a9549e in LuaSerializer::unpickle_json (l=0x555556f564b0, value=...) at ~/src/pioneer/src/lua/LuaSerializer.cpp:281
#3  0x0000555555a9549e in LuaSerializer::unpickle_json (l=0x555556f564b0, value=...) at ~/src/pioneer/src/lua/LuaSerializer.cpp:281
#4  0x0000555555a55bba in LuaObjectBase::DeserializeComponents (object=0x555563e9c670, obj=...) at ~/src/pioneer/src/lua/LuaObject.cpp:815
#5  0x0000555555a96c05 in LuaSerializer::LoadComponents (this=0x555557167040, jsonObj=..., space=0x55555e4348e0) at ~/src/pioneer/src/lua/LuaSerializer.cpp:597
#6  0x00005555558458bb in Game::Game (this=0x555560b0d6c0, jsonObj=...) at ~/src/pioneer/src/Game.cpp:207
#7  0x000055555564a07b in SaveGameManager::LoadGame (name="tmp13") at ~/src/pioneer/src/SaveGameManager.cpp:100
#8  0x0000555555a304c7 in l_game_load_game (l=0x555556f564b0) at ~/src/pioneer/src/lua/LuaGame.cpp:213
#9  0x0000555555ac69b0 in luaD_precall (L=0x555556f564b0, func=<optimized out>, nresults=<optimized out>) at ~/src/pioneer/contrib/lua/src/ldo.c:318
#10 0x0000555555ac6f35 in luaD_call (L=0x555556f564b0, func=0x555555e835a0 <typeinfo for SavedGameCorruptException>, nResults=0, allowyield=0) at ~/src/pioneer/contrib/lua/src/ldo.c:394
#11 0x0000555555ac641b in luaD_rawrunprotected (L=L@entry=0x555556f564b0, f=0x555555ac03d0 <f_call>, ud=ud@entry=0x7fffffffcfd8) at ~/src/pioneer/contrib/lua/src/ldo.c:131
#12 0x0000555555ac7760 in luaD_pcall (L=0x55556da08a10, L@entry=0x555556f564b0, func=0x555555e835a0 <typeinfo for SavedGameCorruptException>, u=0x0, u@entry=0x7fffffffcfd8, old_top=0, ef=7) at ~/src/pioneer/contrib/lua/src/ldo.c:595
#13 0x0000555555ac02a7 in lua_pcallk (L=L@entry=0x555556f564b0, nargs=<optimized out>, nresults=nresults@entry=-1, errfunc=errfunc@entry=0, ctx=ctx@entry=0, k=0x555555ac3ac0 <pcallcont>) at ~/src/pioneer/contrib/lua/src/lapi.c:949
#14 0x0000555555ac3217 in luaB_pcall (L=L@entry=0x555556f564b0) at ~/src/pioneer/contrib/lua/src/lbaselib.c:393
#15 0x0000555555ac69b0 in luaD_precall (L=0x555556f564b0, func=<optimized out>, nresults=<optimized out>) at ~/src/pioneer/contrib/lua/src/ldo.c:318
#16 0x0000555555ad73fa in luaV_execute (L=L@entry=0x555556f564b0) at ~/src/pioneer/contrib/lua/src/lvm.c:709
#17 0x0000555555ac6f50 in luaD_call (L=0x555556f564b0, func=<optimized out>, nResults=<optimized out>, allowyield=0) at ~/src/pioneer/contrib/lua/src/ldo.c:395
#18 0x0000555555ac641b in luaD_rawrunprotected (L=L@entry=0x555556f564b0, f=0x555555ac03d0 <f_call>, ud=ud@entry=0x7fffffffd248) at ~/src/pioneer/contrib/lua/src/ldo.c:131
#19 0x0000555555ac7760 in luaD_pcall (L=0x55556da08a10, L@entry=0x555556f564b0, func=0x555555e835a0 <typeinfo for SavedGameCorruptException>, u=0x0, u@entry=0x7fffffffd248, old_top=0, ef=7) at ~/src/pioneer/contrib/lua/src/ldo.c:595
#20 0x0000555555ac02a7 in lua_pcallk (L=0x555556f564b0, nargs=<optimized out>, nresults=1, errfunc=5, ctx=0, k=0x0) at ~/src/pioneer/contrib/lua/src/lapi.c:949
#21 0x0000555555abcc89 in pi_lua_protected_call (L=0x555556f564b0, nargs=1, nresults=1) at ~/src/pioneer/src/lua/core/Sandbox.cpp:290
#22 0x0000555555763dbb in LuaTable::Call<bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, double> (this=0x7fffffffd388, key="mainMenu", args=@0x7fffffffd3a8: 0.0043108658865094185) at ~/src/pioneer/src/lua/LuaTable.h:437
#23 0x0000555555763660 in PiGui::RunHandler (delta=0.0043108658865094185, handler="mainMenu") at ~/src/pioneer/src/pigui/LuaPiGui.cpp:78
#24 0x00005555556298ba in MainMenu::Update (this=0x555555f5cda0, deltaTime=0.00431086589) at ~/src/pioneer/src/Pi.cpp:699
#25 0x00005555555ca2ea in Application::Run (this=0x555555f5ce50) at ~/src/pioneer/src/core/Application.cpp:193
#26 0x00005555555c567a in main (argc=1, argv=0x7fffffffda48) at ~/src/pioneer/src/main.cpp:183

[markdascher]

Confirmed that I can "fix" my own corrupt saves by running savegamedump and then editing the JSON, but only by moving the table up to the first empty Equipment.BodyScannerType reference, basically taking the definition out of ScanManager and moving it into EquipSet.

What's interesting is that swapping the order of the ScanManager and EquipSet sections doesn't work, even though that also corrects the FIFO issue in the save itself. It's as though the game re-sorts the keys alphabetically after reading the save file.

[Gliese852]

@markdascher

What's interesting is that swapping the order of the ScanManager and EquipSet sections doesn't work, even though that also corrects the FIFO issue in the save itself. It's as though the game re-sorts the keys alphabetically after reading the save file.

Yes, apparently this is due to the implementation of the JSON library used

pioneer/contrib/json/json.hpp

Lines 11543 to 11546 in 89d503a

	- Internally, name/value pairs are stored in lexicographical order of the
	names. Objects will also be serialized (see @ref dump) in this order.
	For instance, `{"b": 1, "a": 2}` and `{"a": 2, "b": 1}` will be stored
	and serialized as `{"a": 2, "b": 1}`.

[fluffyfreak]

output (1).txt

alice_wu_020.zip
alice_wu_021.zip

as mentioned in this comment on the PR bug fix:

I've just tried this PR with a savegame reported on Discord https://discord.com/channels/705212721666326618/705542778716684368/1342498734855946334 but without success, i.e. it does not load a savegame that has already been "corrupted".
In their case user pebblegarden has deduced that the problem started after buying the surface scanner.

attaching the zipped savefiles for further testing

[pebblegarden]

Based on Markdascher's remarks above, I realized my load crashes started happening when I fitted a surface scanner (not the orbital one). So as a test, I sold my surface scanner, and now my savegames appear to be working fine. Not a fix, but at least it might be a temporary workaround.