MMR Matchmaking Planning Thread. Nerds welcome.

[Riverside]

Imagine signing up for MMR test game. The bot has no clue what to even do with you and it just makes you a sub. (Riverside)

You've been a sub for a number of TWL seasons and games, haven't you champ?

I guess being a sub is something you can relate to.

Bench buddies!! Gr8est 2d spaceships teammate (geio) alert!!!

[Geio]

I just love trolling you because I actually used to fawks with you. Now you are just a fucking cornball 🤣🤣

[Riverside]

Lol I enjoy the occasional trolling too

Are you playing in this coming TWL playoffs mr geio, or is that a 2028 goal for ya?

[qan]

Note that scores with a handful of single-round games played are so close to meaningless as to be not worth mentioning. Literally.

With this in mind, I've put in a minimum games played requirement of 20 before scores start displaying on the site. At that point, ratings will have stopped jumping around so wildly after each game, and the scores will be slightly less meaningless. Uncertainty in a rating will still be about 85% at that point. An MMR system needs games, games, games to improve accuracy, and you're not going to see truly accurate results until 100+ GP.

Also worth noting: the sub list has nothing to do with skill. The bot attempts to make teams 1000 times, constantly swapping players around and checking if the swap improved the balance. Anyone on the sub list is just the odd person (or people) after swap #1000. A bit like a cakewalk. Also, if you were benched in the previous game, you have a much higher chance of being able to play the next game because the bot prioritizes you getting played.

Leaving for a vacation for about a week, and I'll only be checking in sporadically. All staff now have access to instructions for how to host. Please get in some games so we can make sure this is all functioning properly after months of planning and work. Appreciate you!

[Rab]

100 games is like a year for eu lol

[qan]

100 games is like a year for eu lol

With one round and lines pre-decided, you can probably get in 3-4 per hour. Accurate rating does take time, regardless, and it needs a lot of data to do it. I'd suggest asking for games via ?help on weekends. As more people begin to better understand the concept, there will be more opportunities to play. Eventually, it should be the easiest and fastest way to get a game, and a balanced one at that.

[Geio]

Lol I enjoy the occasional trolling too

Are you playing in this coming TWL playoffs mr geio, or is that a 2028 goal for ya?

Yeah probably. Remember on rapid with creature we could have went further but you were literally hot brain dead dumpster dog fire water. That one one season you played well! Hopefully you can catch lightning in the bottle again! Would be great for Rocket bub.

[Exalt]

Yeah probably. Remember on rapid with creature we could have went further but you were literally hot brain dead dumpster dog fire water. That one one season you played well! Hopefully you can catch lightning in the bottle again! Would be great for Rocket bub.

Hey, Rocket needs all the help it can get right now. You don't know the meaning of hot brain dead dumpster dog fire water until you've played a few hundred games with Rocketeers and experience their race to dying out in 5 minutes or less.

[Riverside]

Remember on rapid with creature we could have went further but you were literally hot brain dead dumpster dog fire water. That one one season you played well!

Im glad the shared rapid season existed.

Its about the only season you've participated as a player, instead of coording from F11 (yellow chat or on discord).

Rooting for your playtime, bud. This may finally be a season you can pew.

[RageRitual]

Wall of text thoughts incoming:

Played some MMR matches this week, won (well got carried) almost all of them, but they didn't feel like fair matches. Still felt better than TSL though. People usually only complain when they are losing, honestly I care more about balanced matches than winning or losing because they are the generally the most fun matches.

One of the major unbalances that I notice is when we have 10*'s on one team but not the other. Not sure how the best way to go about balancing that would be, but when you have 2 10* on one team and none on the other, the solution there is obvious to humans but not the bot. Right now I have a huge mmr in javelin compared to everyone else I look up in the arena, but I am complete garbage at the ship. I just keep getting carried by having the more 10* on my team than the other. If I have the highest mmr in the arena, why is the bot putting me on the same team with all the 10* players? One possible solution could be have all the 10* (could even add 9.5 ones too) to a list and have the bot check the lines vs the list to prioritize placing them on different teams if possible.

We could also have a baseline where when the season ends or whatever, people aren't fully reset, just partially. So at the end of the season instead of being reset to 3000 or something, they get reset to some pre-decided number like half the rating difference or something, so someone that was 5000 ends up at 4000, or someone that was 2400 ends up 2700.

I used to play league of legends a lot (made it to diamond!) and getting stuck with really bad RANDOM teams can feel very tilting so It's great that these are 1 round matches, if they end up being unbalanced it's 10-15 minutes and you're done. Instead of being stuck with crap team for an hour or whatever. We should make base when it's ready to 10 minute instead of 15 too. Speaking of base, there is a problem with switches right? We don't need switches honestly, especially if we have shorter rounds and allow people to pick their ships ahead of time. We could change the separate signup command for to include which ships people are willing to play to avoid people getting stuck in ships they don't want to play. Command could look like !signup 1 2 3 5 8 or something (1 and 2 being dd and jd please no specials in base), or even allow them to change it every time with the !p 3 5 8.

Thinking ahead here, would it be possible for matchbots in TWD arenas to pull the prediction data from MMR to just show how balanced caps are making the lines in TWD? I feel like the best way to make balanced lines are somewhere in between people's opinions and bots calculations.

Anyways, like many here I appreciate the work that goes into this project, which I assume is a pain the ass to do and a lot of fiddle-fucking around with balancing.

[saiyan]

deleted

[qan]

I would say when we're playing these early games, since we have human knowledge of the capabilities of almost all players, let's just try to balance the teams manually. A host can just start an MMR-enabled game, then maybe assign the two best players as opposing caps and let them add lines? We could also enable pick by turn in an alternate rules files for this kind of mode. This makes playing MMR games actually far easier because you wouldn't need to change arenas.

The two downfalls to this approach:
1) you're not guaranteed to have enough to play.
2) no guarantee that everyone added has done !signup. Currently, a player not signed up will crash the bot. (No sanity checks. A solution to that would be to just auto-signup anyone who was in the past DT.)

Both of these can be fixed by just opening matchmaking in MMRBot but ignoring the assigned teams and picking from the pool of players provided, including subs.

So, basically, don't worry about the matchmaking quite yet; humans will be far superior to the bot for the present moment. Straight MMR is better when we don't have this wealth of knowledge to draw upon. Eventually, the bot should surpass human abilities, however.

One thought: We could also preload ratings based on stars, but we're not yet sure what MMR rating corresponds to what star rating, especially with lower-rated players.

---


Thank you both for taking the time to think about this and post some comments and suggestions. I'm still on vacation (just did some 10* humpback whale watching, so close you could almost touch the buggers) and won't be back for a bit, but I'll take this into consideration and reply in more detail after I'm back at home.

[Exalt]

Personally I would scrap the uncertainty and consider adding hidden mmr to all relevant arenas. It's free testing that doesn't require people to opt-in.

I feel like this would just kill the entire point of MMR if this was done. It's supposed to be something different from all the current things going on. The biggest issue as I see it is that this would just encourage stacking rather than equal teams in other arenas. It also doesn't really help even if the bot knows for instance that one team is stacked and has a 90% chance to win, since the team with the 10% chance still loses 90% of the time and everyone on that side gets a huge net loss of MMR. This is inherently going to hurt anyone not on a stacked squad for instance. There are no consolation prizes, because even being someone that pulls out a win in an extremely improbable way, the MMR won once every 10 games won't counterbalance the other 9 losses due to teammates simply being outmatched by everyone else. It's far less probable you'll get shit teammates every single game against a stacked team in pure MMR play than by playing on Rocket or Chaos Lord against Evil in TWD for instance.

As it stands, MMR only works as a concept when teams are equally matched, and they can only be equally matched by starting from scratch and going from there over time. It would be one thing to add it to wbduel or something where squads aren't set already and the only mitigating factor are captains being competent enough to choose equal teams.

I think the real solution is to keep pushing MMR until people play it regularly. Matches will settle down over time if people did this rather than play TWD games for instance.

[Rab]

Yeah, just need to get the games played. It's a bit of a downer for me when people start playing wb because the arena has a res limit. So I'd say pick a division and play a load of games of it rather than flip-flopping. Would appreciate some sort of schedule so people know when it's happening.

I also noticed that when you view a player's recent games on the twd website, these mmr games appear, and they don't display properly as they're not real twd games. Dunno if tsl games have the same issue. Wonder if they count towards twd stats. Borrowed games still don't afaik, but should.

[Dwopple]

I feel like this would just kill the entire point of MMR if this was done. It's supposed to be something different from all the current things going on. The biggest issue as I see it is that this would just encourage stacking rather than equal teams in other arenas. It also doesn't really help even if the bot knows for instance that one team is stacked and has a 90% chance to win, since the team with the 10% chance still loses 90% of the time and everyone on that side gets a huge net loss of MMR. This is inherently going to hurt anyone not on a stacked squad for instance. There are no consolation prizes, because even being someone that pulls out a win in an extremely improbable way, the MMR won once every 10 games won't counterbalance the other 9 losses due to teammates simply being outmatched by everyone else. It's far less probable you'll get shit teammates every single game against a stacked team in pure MMR play than by playing on Rocket or Chaos Lord against Evil in TWD for instance.

FWIW, in a situation like this where the win probability is heavily skewed, then so is the net change in points. e.g., If the team w/ 90% win probability wins they'll only net a 10% increase in their MMR from that round and the same goes for the losing team's MMR. It'd like be close to a single digit change after ~20 or so matches, but I may be wrong. If the team w/ a 10% chance to win does win, then there will be a more sizeable MMR points swing up and down for each team. That said, those matches would still likely be a snooze-fest which is what the MMRBot is being tuned to prevent and auto-balance team creation.