Feature request: multiplayer game with server API and more features

[CrsiX]

Hi, we really like the game but the multiplayer functionality just sucks, sorry. (And the server is pretty ... hacky. Maybe a plain FTP server would have been a better choice).

Therefore, we would like to put our hands on a new multiplayer service. We essentially build our own server (in Rust) while also hooking the relevant client functionality. Additionally, we integrate feature requests such as the following:

• Feature request: Multiplayer Game #8476 (yes)
• Feature request: Chatting in multiplayer game #8063 (definitively)
• Feature request: Multiplayer Server Listing, Pinging, and implementing "Choose Best Server" for Server Groups #7745 (pretty far ahead the roadmap, but a nice idea)
• Also, we want to completely overhaul the API, like using WebSockets/HTTP2 instead of polling on intervals and much more intuitive user experience.

However, we would like some help on integrating some of the stuff in the client app. For example, we build the server and the API and all of the network-related code, while someone else could jump in to tweak the user interface to make use of the newly available features (e.g. chatting or username/password instead of UUIDs).

Side question: how many people are currently using your public multiplayer service? A rough approximation (20 a day or 20k a day)?

[yairm210]

That's a question for @touhidurrr, he built the server at https://github.com/touhidurrr/UncivServer.xyz
Given the '3M requests per day' I assume we're talking more in the 20K range

I have no problem with overhauling the API (current API was basically a MVP for migrating away from Dropbox, hence the FTP-ness)
But we need to let existing users retain their current games.

This means one of the following:

• Changing the implementation together with uncivserver.xyz
• Adding an alternate implementation alongside the existing one

I would obviously prefer the former but @touhidurrr also has his own free time restrictions.

@GGGuenni has been leading the recent changes for passwords &c so he's more relevant to this than I am
As I have stated many times in the past, this is super outside of the roadmap for the project - but if you have people willing to work on it from both sides, then why not

Starting with the basics - 'create multiplayer game and allow others to join'.
We'll want to start with a really stripped-down POC:

• One player creates the game with set number of players, game options, etc
• All initial players (except the owner?) are AIs. Each player who 'joins' the game effectively replaces one of the AIs, so we can start the game at any point with the same amount of players
• Uploads to server (GameSetupInfo, including a new field: "creator ID" for the person who created and can start this game), which registers this as a 'lobby game
• Other players can retrieve 'lobby games' and ask to join.
• When a player joins:
  • If there is an AI he can replace - does so
  • If not: error message to user
• Creator gets updates of new players joining the game and can decide at any point to 'start the game'
  • Server gets message 'game starting' to disallow other players from joining and to remove from 'lobby'
  • After the new game is created, it is uploaded to server

That's already a lot of work, and this doesn't include issues like

• Closing inactive lobby games after X amount of time
• 'leaving' games before they have started
• Someone creates a game and then closes the app - how does he get it back

[touhidurrr]

why not websocket?
i think current fileserver is good enough, but if we want to implement all those features you are talking about,websocket might be better.
also, i would be free after a week or so, at the time my mid term examination finishes. jobless now.

[yairm210]

I'm not against, do whatever feels best to y'all

[touhidurrr]

Also due to some issues, i am actually rewriting UncivServer.xyz.

[myOmikron]

Hey guys, I'm with @CrsiX,

nice to hear that you're open to improvements in this regard :)

But we need to let existing users retain their current games.

I'd personally would go for a new implementation, let's call it v2 for now. It makes it easier to make design choices if you don't try to reuse the existing implementation. So I think it would be best to add another option to the multiplayer settings for this, like it is now for dropbox vs uncivserver.xyz

We're also open for any contributions, regardless if it is code or suggestions regarding API design @touhidurrr and @GGGuenni.

We'll want to start with a really stripped-down POC:

The suggested scope sounds reasonable for a first prototype

---

why not websocket?
i think current fileserver is good enough, but if we want to implement all those features you are talking about,websocket might be better.
also, i would be free after a week or so, at the time my mid term examination finishes. jobless now.

I think websockets would greatly improve the load on the server, as there's no "is there any new data?" like requests anymore.

I would use an API for the lobby mangement and go for websockets for exchanging the actual game data (and chatting in the later implementation).

[touhidurrr]

Ok, or let's split this into separate parts and implement them one by one:
I would like to work on the lobby and chat feature first. Preferably in-game chat.
The concept of in-game chat is pretty simple. You would join via WebSocket and any message you send will be broadcasted to other players. There would be no chat logs stored on the server, and if you go out of the game then you lose the chats. This is actually pretty common and examples are chat in google wrokspace products (meet, docs, sheets etc.), IRC chats etc.
I think this can be done within a week or less.

[CrsiX]

As far as I understand you, @touhidurrr, you "just" want to improve the current multiplayer handling with a few new features (and a server re-write on your side). However, "our" current approach is completely overhauling the whole multiplayer handling. That's why I said earlier:

we build the server and the API and all of the network-related code

This includes both server and client. Therefore, the question: do you want to participate in designing and implementing the new architecture? Because splitting the work only makes sense if we pull together.

Our server implementation work already started here: https://github.com/hopfenspace/runciv

[touhidurrr]

However, "our" current approach is completely overhauling the whole multiplayer handling.

Oh, I think I misunderstood the context of the issue. Good work!

the question: do you want to participate in designing and implementing the new architecture?

I don't know rust, so no. However, I would like to participate on the design aspect of it. I already have many design concepts in my mind for many features like Lobby, Chat, etc. But not as a REST API but as a WebSocket server. Obviously, some endpoints can be REST but overall it is better to avoid REST as a whole to make the mp experience feel more seamless. As polling is not really a pleasant experience for anyone. I guess it is time to write them down and take opinions and start working on them. I guess we can share our design choices and discuss among us to build the best Multiplayer server ever.
Also, the focus should not be just rust but a good standard that can improve the Multiplayer experience as a whole.

[touhidurrr]

Side question: how many people are currently using your public multiplayer service? A rough approximation (20 a day or 20k a day)?

I think this is the data you are looking for:

total_requests_2023-03-05T19_03_02.562Z.csv
unique_visitors_2023-03-05T19_03_02.564Z.csv

Also see:
https://docs.google.com/spreadsheets/d/e/2PACX-1vSPk2pf0_SGVm08pXLmrJjStlVh33ItHweVOYFdqqP9Ghtec1rvTuz9GmfP-zQIWfiB2TGtJU7kn6Lu/pubchart?oid=2039282213&format=image

[CrsiX]

Thanks for the insight. Around 100k users is a good value for this game, but no stressful amount for a good server as well.

Do you want to join us at the discussion page? :)

[touhidurrr]

Thanks for the insight. Around 100k users is a good value for this game, but no stressful amount for a good server as well.

This is monthly statistics, but yes, that is not considered as high volume traffic.

Do you want to join us at the discussion page? :)

Yes. I read your rust code and there is some things that I would like to talk about. However, I would prefer if for Unciv Multiplayer v2, we discuss a list of features that we would like to implement, open a new issue with a checklist in this repo and then start working on that. Also, @yairm210 from you post #8817 (comment), I am not sure if you want to maintain UncivServer.jar and it would be good if you could clarify that, but regardless I think it would be good if we open a new branch to and start writing proper docs UncivServer API and possibly implementation for UncivServer.jar (given that many players from time to time want to start a server locally) and the client.

[yairm210]

Unciv.jar can die as far as I'm concerned. It was just a temporary hack to demonstrate the API so people who wanted to stop using Dropbox could.

[yairm210]

Like with everything, I'm in favor of burning all the old stuff to the ground - provided that we have a replacement that's been tested and that we can phase out the old gradually.
Which is why it's important for us to maintain compatibility to the current server, at least until such a time that the current server uses the new format, or - alternatively - all of our users successfully migrate to the new one.