[FEATURE] ISteamUserStats:SetUserStatsForGame support

[blaberry]

Description

Currently, change suggestions to neotokyo has been a consensus driven process, where the judgement of players have been leveraged heavily with regards to making changes to core gameplay. This has been done without the use of data analysis and statistics tracking, making quantification of whether a change has had the desired effect or not very difficult.

Example

As an illustrative example, if you were to suggest a damage boost to the SRM-S, you'd want to track:

Indicators for improved weapon variety

1. The number of times the SRM-S is picked - if this rises to within a given tolerance band, that's a indicator of a good balance change
2. The number of kills made by the SRM-S - This should rise, but you should define a ceiling to the number of kills. You probably also will want to normalize the value against how often it's picked, but this should be done in calculation, not during logging.

Counter-indicators that may demonstrate too much of a balance change:

1. Average recon KDR - If this rises too sharply, you have probably boosted damage too much.
2. Class selection - If people who play assault or support start picking recon to counterstrat or take advantage of the gun after the change, your change was definitely too harsh. This will be a trailing, longer term indicator.

Proposed Solution

• As a change suggester, I would like to be able to point to tracked metrics to define whether my suggested change is successful at achieving it's intended effect or not. I'd like these stats to be available through the ISteamGameServerStats::GetGameServerPlayerStatsForGame interface, as it allows me or someone with data visualization skills to create a dashboard where the community can monitor the results of my experiment.

The easiest way to achieve this would be through a new server side logging library. The library exposes an interface for logging metrics to steam, which can be added in the appropriate places into the game engine. The library checks whether sv_neo_comp is 1 and whether a sv_publisher_api_key is set and if true, the library updates the metrics at round conclusion or match end.

The ISteamUserStats::SetUserStatsForGame interface can be used for sending the statistics to steamworks.

Alternatives Considered

A sourcemod plugin could be written for tracking metrics and a separate statistics server could be hosted. But since steam provides free centralized storage, that seems unnecessarily burdensome to maintain.

Additional Context

The interface definition for the REST call is available here: https://steamapi.xpaw.me/#ISteamGameServerStats, but these interfaces are better available through the steamclient library https://partner.steamgames.com/doc/api/ISteamGameServerStats

[Nbc66]

this can not be done using the gameserverStats that interface is only used to set achievements on the server side so that people don't cheat to get the achievements. if you wanted to track the stats of how many times a weapon is picked you need to create a separate API that can track user stats and send that info to a database trough an API call

[blaberry]

It's true that the stats are recorded server side and that the API call requires a publisher API key, meaning community servers (ie, any who is just downloading and running the dedicated server) couldn't automatically report stats. We'd need to have an onboarding process for hosts if they want to report stats, where we create an API key for them. It's not ideal to have this split between "official" and "community" servers, but it is a requirement for matchmaking as well, because we'll need to manually add IP(range)s for servers that will accept steam lobby groups.

You're right that we can't use gameserverstats, I was confusing it for SetUserStatsForGame, which lets you record any sort of counter:

Updated the description to match

The slight issue I see with the steamworks interface is the default low cardinality. There's no way for us to add metadata fields to the counters, so we can't have a single "kills" counter than we then populate metadata fields on like "weapon:mx" and "class:recon" that we can slice by after the fact. We will have to record a stat called "deaths:recon" along with "deaths:total", "deaths:support", etc. This leads to a bit of a cardinality explosion: we'll have to record kills:mx, assists:mx, deaths:mx and at the same time kills:recon, assists:recon, deaths:recon for the same events.

The small consolation is that we don't have to have a kills:all_weapons or kills:all_classes counter, we can just add them together them after the fact.

Storing this duplicate data is very inexpensive for steam however, and doesn't introduce any overhead on the clients, all the data can be sent at the end of a match or during warmup the next round to avoid any network overhead server side as well.

Sidenote: If steamworks would let us add metadata/decorators to basic counters, it would let us create some amazing statistics without having to parse demos or perfom some more advanced queries, if we added "weapon:mx", "class:assault". "ghost:close", "team:koba" and "round:start"(or "round-time:163") as fields to the "dropped_weapon" counter, we'd be able to create a query like "how many times did an assault on koba drop an mx for an early round ghost pickup?". Doing that without metadata fields on counters is sadly infeasible, but we can still get like 80% of the value with the counters steamworks does provide, by adding just one level of cardinality and duplicating some of the counters.

[Nbc66]

Again it would be way easier and better to just make our own stats api the game can pull from snd send data to. You could handle server onboarxing by providing server with a custom api key they need to set in the game and just verify it trough the api

This would make it easy to track and show stats and access em

Making a simple FastAPI python app would be sufficient enough

[blaberry]

That would be one way to do it, yes, but I don't think we can match steam in terms of availability and cost. There's also some concerns about fragmenting the data if the statistics can be sent anywhere, which would make drawing useful conclusions much more difficult.