World Partition Support and Changes between 5.4 and 5.5 Breaking Game Saves

[ac-branislav]

I already had a quick chat with @sinbad about this, so I'm mostly putting it up here in case anyone ever takes a look at it or has any ideas or suggestions how to address this.

Upgrading a project between 5.4 to 5.5 it appears that all the SPUD data using world partition will completely break mostly because it appears they changed the way they compute the world partition cell names and all of the data inside for LevelActors is stored inside a TMap using a level name such as 5.4:

[+] Content of Cell MAP_NAME_MainGrid_0_0_0_5 (B3PX6MHSG1U9219QVBOZ06GCS)
|- Actor Count: 47
|- [+] /Game/Maps/MAP_NAME/UEDPIE_0_MAP_NAME.MAP_NAME:PersistentLevel.BP_CutscenePlayer_C_UAID_C87F54545F47343102_1648680475
| |- Package: /Game/ExternalActors/Maps/MAP_NAME/MAP_NAME/1/O2/IO0IC8WN7TG9OOVB7GVUEK
| |- Editor Only: 0
| |- Instance Guid: 01EBD95142BB97D54DCC5F83C22A4CF2
| |- [+] Container:
| | |- ID: 00000000000000000000000000000000
| | |- Transform: 0.000000,0.000000,0.000000|0.000000,0.000000,-0.000000|1.000000,1.000000,1.000000

5.5:
[+] Content of Cell MAP_NAME_MainGrid_L5_X0_Y0_Z0 (1Z4A0V9YX0RUMJEE926CYL2QE)
|- Actor Count: 47
|- Content Bounds: IsValid=true, Min=(X=0.000 Y=450.000 Z=0.000), Max=(X=352629.658 Y=183118.232 Z=55469.770)
|- Cell Bounds: IsValid=true, Min=(X=0.000 Y=0.000 Z=0.000), Max=(X=409600.000 Y=409600.000 Z=409600.000)
|- Is 2D: False
|- [+] /Game/Maps/MAP_NAME/UEDPIE_0_MAP_NAME.MAP_NAME:PersistentLevel.BP_CutscenePlayer_C_UAID_C87F54545F47343102_1648680475
| |- Package: /Game/ExternalActors/Maps/MAP_NAME/MAP_NAME/1/O2/IO0IC8WN7TG9OOVB7GVUEK
| |- Editor Only: 0
| |- Instance Guid: 01EBD95142BB97D54DCC5F83C22A4CF2
| |- [+] Container:
| | |- ID: 00000000000000000000000000000000
| | |- Transform: 0.000000,0.000000,0.000000|0.000000,0.000000,-0.000000|1.000000,1.000000,1.000000

So whenever a streaming cell loads and than we iterate actors inside of it we use the level data to fetch it from the spud data however if epic changes how the cell names are computed OR the object someone ended up inside a different cell for whatever reason, SPUD will fail.

I had some quick thoughts on how to potentially fix this but currently spud level data is stored as TMap<FString, TLevelDataPtr> LevelDataMap
with a comment that all FNames are unique inside Level Name, while this is true the chance of you getting the same actor UAID in 2 maps is almost impossible UNLESS you duplicate your maps. Unreal does not generate new UAID when doing that, so you are stuck.

One solution is that I thought of is that when we tried to restore the actor, if we find that ActorData is null returned from ActorData = GetLevelActorData(...) than iterate the map and load the other sublevels data until you find another sublevel that has actors with the same UAID and than load that data in using a new LevelPtr data and than on next save it will fix its self. This will work unless you duplicated your maps and have same UAID across 2 maps. (Potential solution to the duplicate UAID problem is to actually store LevelDataMap not as TMap<FString, TLevelDataPtr> but rather as TMap<RootLevelName, TMap<FString, TLevelDataPtr>> so that you can search the map for the parent persistent level and than the sub levels under it

I think at least for us I'm going to attempt to solve this using the approach I did above but I would be real curious to see if anyone ran into this and how they approached it. It definitly seems like there is a future where epic changes these again and our game saves will break again.

Some other random idea I had was if it was possible to generate the 5.4 guid inside the 5.5 version than you can try the look up using the old hash as well but Im kind of out of ideas on this

[ac-branislav]

I spent a bit more time just trying to figure out exactly how bad the world partition case is:
UWorldPartitionRuntimeHashSet::GetCellUniqueId(...) is the function which is responsible for creating these GUIDS such as
B3PX6MHSG1U9219QVBOZ06GCS or 1Z4A0V9YX0RUMJEE926CYL2QE.

and its very clear that indeed in 5.4 they were using
MAP_NAME_MainGrid_0_0_0_5

and in 5.5 they are using
MAP_NAME_MainGrid_L5_X0_Y0_Z0

So due to this change, they regenerated the GUIDS which will break the game save.

Interesting enough, there is more than just the grid ID used for the hash, they are also using the data layer id's hash added which means that if anyone simple assigns a new data layer to an actor inside a grid cell, since they hash ALL of the data layers inside that cell, the save data will break.

Ultimately, I don't see a solution to this problem without changing the LevelDataMap to actually just be more of something that can be more centered around a hash of actor UAID to data or something along those lines

[cyphusx]

I've been thinking about this for a while, and the conclusion I've come to is that I'm going to break the link between SPUD level data and the actual WP-generated levels. Instead, the SPUD 'levels' will form their own independent grid which spans the world, and the serialized actors will be stored in the grid based on their location / bounds. When a WP level is loaded, the bounds of the loaded level will then determine which SPUD 'levels' need to be loaded, and then be available for restoring the state of the loaded level actors.

This way SPUD is no longer dependent on the way that the engine cuts up the world. You're still potentially left with the issue of how to uniquely identify an actor when you've got duplicated level instances - you'd need something like your root-level-name proposal for that.

It's not something I've dived into the implementation of yet, though I have already done something similar for storing runtime-spawned objects (before SPUD had the functionality to do this), and it's something I'm going to have to start looking at very soon as we aim to migrate to 5.5 and beyond.

[ac-branislav]

I spent a bunch of time talking to EPIC and thinking about this as well and as far as the root level name proposal, not only does it need to store the root level, we also need to store it using the actor paths and NOT using guids because you will still have the issue of doing multiple level instances inside a root map if you want to store SPUD data onto them, BUT I think this is highly doable just needs work. In our case, we have basically

On the topic of a separate GRID to split it from world partition, I was also thinking of the same thing because essentially the way world partition is implemented the dynamic level hashes are not to be used for anything outside of runtime data which is the equivalent of given a position give me the cell ID its in for reverse look up. In our game, I already implemented a grid like you suggest for something else so I can say at least that approach would work just need to keep in mind whether we would allow people to change the resolution of this grid. If we allow the grid resolution change, than we need a versioning number that can be bumped and the data migrated across game saves. BUT honestly spud data is not really that memory intensive, that even 256m grid cell across the world boundary would work (I think we would need to define the bounds somewhere) aligned to 0,0,0

Now if we have a separate grid from world partition, that than becomes what do you do with runtime actors..? because those are moving world partition cells. Either way a lot of things to think about...