Implementing the "expected winrate" formula and the corresponding aggregated stats

[alocate]

The goal is to be able to produce the same "expected winrate" statistic derived as modelled by the glicko2 rating system.

See: https://warcraft3.info/articles/359 for an introduction. I am not talking about fitting the bias parameter over the dataset.

Ideally, anywhere there is an "actual winrate", there should be an "expected winrate" equivalent.

For example:
-https://www.w3champions.com/OverallStatistics/winrates-per-race-and-map
-https://www.w3champions.com/OverallStatistics/heroes-winrates
-players' profiles so we can show the best and worst matchups based on that!

This would, later on, involve frontend work (by someone else most likely?).

Contact Toxi for exact formula.

[mourasman]

@abeninski @modmoto I've been working on this for the past couple of weeks and I've had a lot of back and forth with the conceptualization of the solution for this so I could use your help here.

So, in order to be able to achieve an accurate Expected Winrate (for all time, not just seasonal) we will need to calculate the MMR of each player based on their entire match history and save that value somewhere so it can then be used to calculate the Expected Winrates.

Currently, I'm trying to solve this by having the Matchmaking Service handle the "Real" MMR calculations (literally the same way it does for "Normal" MMR except that it doesn't get reset each season) and then with every Match Result, it sends both the "before" and "after" to the statistics service (although, we really just need the before for Expected Winrate purposes). Then, on the Statistics Service, we would update the Expected Winrate on every "MatchFinishedEvent" basically.
In order to achieve this, we would need a new admin endpoint both on the Matchmaking Service and Statistics Service to recalculate both the Real MMR and Expected Winrates (hopefully, we only need to do this once so we can probably drop these admin endpoints...? thoughts?)

How would you guys do this?

[modmoto]

I dont get the notion of expected winrates to be honest. What would it show you? With 2k mmr you should have a winrate of 60%, but with 1k only 20%? Should it not be 50% all the time? I also dont know if we should show this winrate to people. I would see it more as a monitoring tool to see if the matchmaking algorithms are working well. But in the wc3 scene people often mistake Winrate with skill, where it is rather an indicator of a bad matchmaking system.

I also would not touch the old Match Finished events. The Match Finished events are records of past events. If we did not have it then, leave it like that. If it is really necessary (which i doubt), we can create a script for mongodb.

[alocate]

expected winrate means in each single match, what were the win probabilities for either player. Say 2200 MMR vs 2000 MMR, then the first player should win 70% of the time. so expected winrate will be 0.7 for him, 0.3 for the opponent.

once you aggregate this for all matches of a particular player, for example vs a certain race, then you can compare his expected winrate vs that race (which matches the strength of players he met) to his actual winrate. So you can tell people, hey, here are the matchups where you overperform/underperform compared to your MMR and the strength of the opponents you met. Of course with perfect matchmaking and infinite number of players, this goes to 50%.

For example vs orc I have say, 70% winrate, but is it because I only meet lower level orcs since there are so few good orcs, or is it because I am particularly good at the matchup?

I think there is a lot of value in that feature for players who train on ladder.

[modmoto]

ah ok, that makes sense. Jeah this seems cool. And as this would be a new readmodel, we can actually change the events, if we can calculate that from previous matches. It should only be one more number on the match events, right?

[mourasman]

Yes, exactly. It would be one extra property on the "match" object of the "MatchFinishedEvent" that tells you the players' Real MMR at the beginning of the match.

[abeninski]

I'm really not sure if this is worth the effort. Sounds like a lot of data juggling :/

[mourasman]

Indeed, it is. If you guys feel like it, I can always put this back on the backlog. We probably have other higher prio stuff right now.

[modmoto]

Calculating it from now on forward should be good enough, imo. If someone has the time or wants to do it, i think it is a cool feature.

[Cepheid-UK]

I think this is a great feature but it does sound complex to implement. I'd vote for it with an "if its not to hard".

I guess you could put it in the python service for testing as that's what you used for the w3info article?

[mourasman]

@alocate do you think we could introduce this with season 4 using the actual MMR instead of recalculating everything for the real MMR? would that be useful? that way, we lay the groundwork for some of the things and we can then improve upon it in the future iteratively rather than doing everything in one go.

[alocate]

what if we first did the migration to compute it using MMR at game time first, for all games since season 1? on top of that, also make the change to compute it for games as they complete.

that should be the easiest part to start getting work done on the frontend. once that's done, we can always re-do the more complex part of computing shadow MMRs and expected win rates for those. it feels like that very part is the bottleneck atm, so yeah we can definitely bypass it.

[mourasman]

yes, that seems reasonable. I'll do that, then! 👍

[zygissss]

done