MMR for TSL (and eventually, other parts of the zone) Planning Thread. Nerds welcome.

[Cape]

I have nothing to contribute other than my interest.
God speed sir.

[Riverside]

I can temporarily move the !reg command (show the names of all players registered) to being available to everyone. It won't show in !help, though, so it's just a hidden command.

Right now there are 10 signups, which is technically enough. Doesn't hurt to get registered to be able to test in the future, though.

Open access to !reg would be appreciated.

I'll post the names here: if someone has a problim with this, please let me know, and I'll remove you from this Publically Posted List.

I think we can get to 20 players who ?go mmr to !signup within the next couple weeks.

[qan]

I can temporarily move the !reg command (show the names of all players registered) to being available to everyone. It won't show in !help, though, so it's just a hidden command.

Right now there are 10 signups, which is technically enough. Doesn't hurt to get registered to be able to test in the future, though.

[Riverside]

qan, with the aim to get more !signups in ?go mmr

Cud we plz (automatically if possibil) allow the playirs who !signup to be listid on forums, similar to what haf been done with TWDT/TWEL signups?

[qan]

As Turban says, wins are the only thing that matter, yes. No other data is taken into consideration. A few MMR systems attempt to adjust slightly based on performance, but arguably this is a mistake.

If you kill 20 people in battle but your army dies at the end, you didn't win. You're dead. In fact, in some cases it's possible that the reason you were able to amass such an impressive list of victims is simply because your team was supporting you properly, but in your excitement, you didn't do the same for them.

Those who will do best in MMR are those who know how to play well as part of a team, and do so as skillfully as possible, bringing wins for their team. Theoretically, you could actually make no kills but still be at the top of the leaderboard. (Extremely unlikely, but perhaps you've developed some nefarious bait scheme that allows your team to pull in the wins.)

We'll also be able to run teams of different skill levels against one another and award points fairly, much like how TWD team scores function. Matchmaking is not strictly a necessary component. Queuing with a small group of friends or a full team might also be possible, with work.

Just had a side-thought as well: we could look at both casual and comp. Casual is exactly the same but hides MMR score, allowing you to feel a bit freer to experiment while still getting the benefits of an MMR system.

[Turban]

In a true MMR system only winning matters.

[Exalt]

That'd be pretty cool, but no, it would be difficult to make use of that older data unless it was done to make a sort of simulated MMR with your past self vs all other players, adjusting each person's simulated MMR over time. Interesting idea, actually, considering the huge amount of match data we have available. It would even allow simulated MMR to be tracked over time, so you could see who was the very best in every single age (at least from TWD data) ... But as fascinating as that might be, it's a different project altogether.

It'll just work as most MMR systems do: getting closer to estimating your current actual skill level as you play more and more games. Nothing fancy, just: did you win, or did you lose? It's an algorithm that takes the human guesswork out, which is often the step (at least without rigorous testing and reiterating) that mucks up just about any kind of ratings system.

And what !signup does is just allow me some test data to use to make sure things are working properly. It'll also make it easier to start testing. Everything will be wiped before the real go-live, so no worries.

Quick question: is the mmr going to favor wins over record? Will it have have a bigger weight for kills or KOs, etc? I only ask this because the answer can directly impact how someone plays.

If I play for the win, record doesn't matter to me in most cases. I'll take risks to keep momentum, give up tons of deaths to get the enemy 9s, setup kills for others, or just die to keep high death teammates alive. All of these things are huge parts to winning games (especially with teammates that are not top players), yet do not show up on stat sheets for the most part.

If I play for record, I can just sit back not giving a fuck and get a better record in a probable loss, unless I'm surrounded by better players who will hold their own regardless of what I do. Since that's rare unless I join a better squad (there are too few already), record gets put on the backburner a lot.

Maybe it's being on Rocket for so long or not being a 10* that makes me have this perspective over people on stacked squads, but I literally have to play different depending on my teammates and what their capabilities are just to win games. I compare it to a PG in basketball being pass-first compared to score-first depending on whose around them. This game doesn't record assists so wins are the only thing that can reflect the difference. Winning 'more' than otherwise would be possible with lesser-skilled players due to doing those untracked things is difficult, but doable. Hypothetically, MMR should eventually recognize that as well?

[Riverside]

And what !signup does is just allow me some test data to use to make sure things are working properly. It'll also make it easier to start testing. Everything will be wiped before the real go-live, so no worries.

I appreciate the clarification.

Signing up sounds like a true 0 commitment case with this ask.

Meanwhile, it sounds like the consequence cud b impactful for the Volunteer Dev Staff/teams to help move this project, and other surrounding projects, forward.

Will vs !signup upon entry 2 TW, ty qan u fuking beast

[qan]

That'd be pretty cool, but no, it would be difficult to make use of that older data unless it was done to make a sort of simulated MMR with your past self vs all other players, adjusting each person's simulated MMR over time. Interesting idea, actually, considering the huge amount of match data we have available. It would even allow simulated MMR to be tracked over time, so you could see who was the very best in every single age (at least from TWD data) ... But as fascinating as that might be, it's a different project altogether.

It'll just work as most MMR systems do: getting closer to estimating your current actual skill level as you play more and more games. Nothing fancy, just: did you win, or did you lose? It's an algorithm that takes the human guesswork out, which is often the step (at least without rigorous testing and reiterating) that mucks up just about any kind of ratings system.

And what !signup does is just allow me some test data to use to make sure things are working properly. It'll also make it easier to start testing. Everything will be wiped before the real go-live, so no worries.

[Riverside]

If you'd like to help with the development effort, please ?go mmr and PM !signup to the bot. This doesn't commit you to anything. I just need more records in the database for testing purposes.

Fuking qan n biet ty 2 u n da teams 4 dis



To clarify:

If I sign up in ?go mmr, and pm the bot !signup, what happens?

Does me doing that give permission to allow past, present, future TWD games to be collected and used to improve the accuracy of the MMR system/bot?

Excitid 2 vs this $$$

[qan]

Some MMR updates:

• Bot has been created (forked from TSLBot). Signup, initial and individual league ratings, qualifications, etc., are all looking good.
• To simplify things, for the initial run, we're going to work on the easier case of just D/J at the moment. BIET needed to pause on the base team choosing algorithm for RL reasons. Base is still happening, but it's much more complicated, so it needs to come later down the line.
• Still hammering out exact values for base factor and how quickly uncertainty changes based on games played (see above for details), but getting close. Once this is nailed down, the initial formulas will be a go.
• Hoping to possibly have some kind of very basic alpha version to test ratings changes by next weekend.

If you'd like to help with the development effort, please ?go mmr and PM !signup to the bot. This doesn't commit you to anything. I just need more records in the database for testing purposes.

[Zeebu]

The only issue I have with other games' MMR matchmaking experiences are that in many cases, the way they force 50-50 matchups is to put the high-MMR person in with a bunch of scrubs against a team of balanced MMR players, assuming this is a balanced matchup. It almost never truly is though. In games where teamwork is essential and hard-carrying is extremely difficult to do (Overwatch 1 for example), this was a huge detriment and ultimately helped kill the game. It also tended to put high MMR players in each medal bracket against each other to create 50-50 matchups, but what that ended up doing was making it so that instead of just dominating a bunch of gold players and jumping ranks into higher tiers asap, a new player who belongs in a much higher bracket will be stuck playing other players in gold who belong in a much higher bracket. Smurfs playing smurfs essentially. They do eventually rank up, but it forces things to take much longer. Instead of dominating people and quickly moving up, they end up being forced into 50% win/loss games for that bracket. If a high MMR player can't be found for the other team, it would add a bunch of scrubs that belong in a lower bracket but haven't downranked yet as teammates.

I don't actually think that applies much to TW due to the lack of players, gameplay that allows for far more individual skill to shine through, and a lack of SSR tiers on top of hidden MMR affecting games. I just wonder how badly games will set against certain players. If a 10* WB is playing, and the other side doesn't have a 10*, will that 10* then be stuck with four 5* players against a bunch of 8* and 9*s? That's an unwinnable situation for them all just to achieve 50-50 matchmaking. That game cannot be fun for anyone on either team really.

This all said, it will be interesting to find out how things shake up regardless. Maybe things go much different than the above examples.

This is another reason I made the suggestion about trying it with captains. 2 knowledgeable people should be able to pick more even lines within the pool of the 8 remaining players.

[Vehicle]

I'm not good at math or statistics so I'm just going to put some general observations about how we would apply an MMR system here:

- Require a 24-72-hour signup cooldown to prevent smurfing. If someone wants to sign up during the season, force them to wait out a timer to hopefully discourage smurfing.

- "Leaver penalties" probably need to be built in. Meaning someone leaving a 5v5 would lose a bunch of points and the losing team would lose less or no points.

- This is going to be integrated into basing as well, correct? Because of its variability in basing lineups, I think it would be smart to still assign captains who !add. What I'm picturing is everyone who wants to base does !signup, the bot makes groups of 16 and assigns two caps per group who alternate adding players from that pool. I do not think letting people pick their ships will lead to engaging play.

[Exalt]

Based on my extensive matchmaking experience in games using an elevated starting point like you have, I think your MMR change should come out to around 25-30 a game in an even matchup (post-calibration). Whatever base factor is required to make that happen should be fine.



In each case I've seen, your confidence is very high (fixed, reduced K) by the time you've played a dozen post-placement matches. From that point you need to grind out a bunch of games if you were placed far off your true rating. Keep in mind there is always a hard cap on how high you can be placed. You should never come out of the placements at a top tier, those spots need to be earned via grinding.

These are the 3 variations I've seen:

• Low confidence throughout placement matches that continues into official ranked play until you start to win roughly 50% of even matchups
• Low confidence (or very high K value) throughout placements, followed immediately by maximum confidence (or a lower, fixed K value) during official ranked play.
• No placements, no uncertainty factor, just one fixed K value from beginning to end (lazy, but not necessarily bad)

I have seen matchmakers do some strange things when you get streaky, such as throwing you into 1 or 2 games way outside of your skill bracket. But I'm not going to delve deep into that as I don't want to make this any more complicated. Nor do I think we have the player count for that. As for an uncertainty algorithm, I can't help you there, but I'll keep my eyes open.

Ultimately it's going to be hard to go wrong and this will be a success as long as you control smurfing and there is enough activity.

Other suggestions:

• If this is how players will be ranked in TSL, consider hiding MMR (at least for everyone outside of the top 10) and then doing a full numeric reveal at the end of the season. For some players, most of their journey will be a straight decline from the starting point until they reach their true rating. It might be disheartening for them to see, especially when some of these individuals (Jessup) somehow believe they should be able to compete for a top spot. I want these people to keep playing, learn how they can create more value for their team, and improve. This isn't TWL or TWDT. You are against yourself.
• Because of the above, be very generous with rewards to encourage our lesser talented players. Tw kiddies like pixel medals. Maybe different medals for top 1, 3, 5, 10, 25..... 50?

Really appreciate you trying to make something good happen here.

The only issue I have with other games' MMR matchmaking experiences are that in many cases, the way they force 50-50 matchups is to put the high-MMR person in with a bunch of scrubs against a team of balanced MMR players, assuming this is a balanced matchup. It almost never truly is though. In games where teamwork is essential and hard-carrying is extremely difficult to do (Overwatch 1 for example), this was a huge detriment and ultimately helped kill the game. It also tended to put high MMR players in each medal bracket against each other to create 50-50 matchups, but what that ended up doing was making it so that instead of just dominating a bunch of gold players and jumping ranks into higher tiers asap, a new player who belongs in a much higher bracket will be stuck playing other players in gold who belong in a much higher bracket. Smurfs playing smurfs essentially. They do eventually rank up, but it forces things to take much longer. Instead of dominating people and quickly moving up, they end up being forced into 50% win/loss games for that bracket. If a high MMR player can't be found for the other team, it would add a bunch of scrubs that belong in a lower bracket but haven't downranked yet as teammates.

I don't actually think that applies much to TW due to the lack of players, gameplay that allows for far more individual skill to shine through, and a lack of SSR tiers on top of hidden MMR affecting games. I just wonder how badly games will set against certain players. If a 10* WB is playing, and the other side doesn't have a 10*, will that 10* then be stuck with four 5* players against a bunch of 8* and 9*s? That's an unwinnable situation for them all just to achieve 50-50 matchmaking. That game cannot be fun for anyone on either team really.

This all said, it will be interesting to find out how things shake up regardless. Maybe things go much different than the above examples.

[saiyan]

• What is the base factor for ratings changes?

Based on my extensive matchmaking experience in games using an elevated starting point like you have, I think your MMR change should come out to around 25-30 a game in an even matchup (post-calibration). Whatever base factor is required to make that happen should be fine.

• How quickly does uncertainty lower based on games played?

In each case I've seen, your confidence is very high (fixed, reduced K) by the time you've played a dozen post-placement matches. From that point you need to grind out a bunch of games if you were placed far off your true rating. Keep in mind there is always a hard cap on how high you can be placed. You should never come out of the placements at a top tier, those spots need to be earned via grinding.

These are the 3 variations I've seen:

• Low confidence throughout placement matches that continues into official ranked play until you start to win roughly 50% of even matchups
• Low confidence (or very high K value) throughout placements, followed immediately by maximum confidence (or a lower, fixed K value) during official ranked play.
• No placements, no uncertainty factor, just one fixed K value from beginning to end (lazy, but not necessarily bad)

I have seen matchmakers do some strange things when you get streaky, such as throwing you into 1 or 2 games way outside of your skill bracket. But I'm not going to delve deep into that as I don't want to make this any more complicated. Nor do I think we have the player count for that. As for an uncertainty algorithm, I can't help you there, but I'll keep my eyes open.

Ultimately it's going to be hard to go wrong and this will be a success as long as you control smurfing and there is enough activity.

Other suggestions:

• If this is how players will be ranked in TSL, consider hiding MMR (at least for everyone outside of the top 10) and then doing a full numeric reveal at the end of the season. For some players, most of their journey will be a straight decline from the starting point until they reach their true rating. It might be disheartening for them to see, especially when some of these individuals (Jessup) somehow believe they should be able to compete for a top spot. I want these people to keep playing, learn how they can create more value for their team, and improve. This isn't TWL or TWDT. You are against yourself.
• Because of the above, be very generous with rewards to encourage our lesser talented players. Tw kiddies like pixel medals. Maybe different medals for top 1, 3, 5, 10, 25..... 50?

Really appreciate you trying to make something good happen here.