World storage / phase improvements

[Cervator]

This issue is to cover a GSOC-level feature described in more detail in the forum.

As a quick summary we lack a stage in world generation where you can consider larger scale objects before ever getting into chunks. Stuff like continent-wide trade routes, world or region maps, long rivers, interactions with height maps and other 2d layers (climate simulation etc). We don't have a ideal construct to involve with that - you are either dealing with a global store (world entities) or chunks which are very local. We also don't have support yet for multiple independent worlds in one game instance.

There are both short term and long term goals related to this, detailed below. The ultimate goal would be supporting multi-world as well as a non-chunk based layer within a given world, usable for larger scale world generation and simulation logic.

Short term targets

These could be done before or unrelated to GSOC, and/or may be prerequisites to working the main item.

• ArrayIndexOutOfBoundsException in NearestSortingList #2742 - bug related to chunk sorting. May be hugely overhauled by this item depending on how it is done
• Race condition / bad initialization order during game start vs chunk generation #2385 - poor initialization order (chunks can start generating too early?)
• 3D Faceted world generation #2458 - improvements to world generation to better consider things like surfaces or large 2D maps (world / region maps). Maybe also related comments in the forum
• Finishing PR Reorder menu screens #2074 - reordering the world creation steps from menu to in-game
• Finishing the CoreRegistry -> Context overhaul, noted some in the forum - possibly needed to allow independent entity stores / parallel simulation logic?
• Implementation of the extracted gestalt-entity-system (MovingBlocks/gestalt - still pending finalization and merge) in favor of the current embedded ES. Ties this item strongly to being worked in a v2.0.0 branch with that integrated, goal being to merge hopefully sometime in the second half of 2017, after GSOC (if the item gets picked)

GSOC level targets

Exactly what would be considered in-scope as part of a GSOC proposal would have to be worked out in a student application in close collaboration with our mentors. Am excited to hear about different approaches!

• Reorganizing the phases the engine goes through as a game is set up. Related to Reorder menu screens #2074 and Race condition / bad initialization order during game start vs chunk generation #2385 (since you could prepare all block registration etc before generating any chunks at all). This can be done without fully implementing each step/phase, so that at least we have each step conceptually prepared. Example: a better world configuration screen system that can support different stages, such as step 1 being picking a world shape (infinite vs finite, flat vs round vs torus..); step 2 generating a primary height map for the main surface showing a preview, then taking options to further customize the world; step 3 showing some of those options like cities pre-placed on the world map, maybe allowing custom naming and letting the player choose a city to spawn in. That could even be hit as part of joining a server with an existing world (picking spawn point before fully joining the game)
• Introducing the Sector concept allowing for independent entity stores for different disconnected parts of the world, and parallel processing of activities in said sectors, including stuff like climate simulation, population simulation, and so on
• Introducing the Surface concept to better support multiple surfaces within the same world, like a sky world, different cave layers deep below the primary surface, etc. Improve Pathfinding to understand the connections between surfaces for long distance navigation (for instance trade between a surface city and a cloud city, including the possibility for raids intercepting trade)
• Adding support for true multi-world game instances. As part of GCI we now have examples of "fake" dimensions simply using deep (or very high) alternative world layers, which for all intents and purposes could be independent sectors as-is - now turn them into entirely independent worlds!
• Thoughts on hosting different worlds or sectors on different server nodes - ability to run multiple server instances together to power a single game server (way out of scope for GSOC but worth considering implications on the architecture)

[Cervator]

Addition: If focusing on improving the world gen stages while in the main menu the stalling while reloading the module environment needs a bit of extra consideration.

• Consider running the env switch in a background thread so it doesn't stall the entire menu. Hopefully this will also allow music and such to keep running while the process is running
• Possible improvements to gestalt-asset-core could probably be made to speed up asset reloading and when it actually needs to happen. This should cut down on the time it takes to reload an environment (entirely different topic)

See #2429 and #2509 for some related observations, some of which were fixed in #2815

As an overall note on this issue it could probably be approached for GSOC from several different angles:

• Rework the world gen setup in the main menu - should be doable entirely without messing with world architecture
• Rework the world gen phases to allow for better seggregation and cleaner initialization
• Rework world storage architecture to be able to split the world into sectors/surfaces for keeping each segment more lean and to qualify for multi-threading simulation
• Rework world types to more properly support multi-world and/or different world shapes (flat infinite, round, cubic planet, torus, etc)

Any one of them could probably be worked into GSOC scope without involving the others, although more likely a combination of them would ideal.

[Cervator]

Initial implementation of Sectors, Zones ("surfaces"), and improved preview has been done as part of GSOC 2017 by @Vizaxo yay! Leaving this issue open for now though as it doesn't have a bounty to close out, and there are so many more good things to continue on. Heck there might even be enough for another round of GSOC in the future. Mainly we need to do a whole bunch of organizing and write-ups :-)

[Cervator]

Partially involving this issue in @TheFlash98's GSOC item for 2018. Might not be entirely within scope but some parts are likely affected so consider this issue somewhat active / occupied for the summer :-)