MMR Matchmaking Planning Thread. Nerds welcome.

[Exalt]

FWIW, in a situation like this where the win probability is heavily skewed, then so is the net change in points. e.g., If the team w/ 90% win probability wins they'll only net a 10% increase in their MMR from that round and the same goes for the losing team's MMR. It'd like be close to a single digit change after ~20 or so matches, but I may be wrong. If the team w/ a 10% chance to win does win, then there will be a more sizeable MMR points swing up and down for each team. That said, those matches would still likely be a snooze-fest which is what the MMRBot is being tuned to prevent and auto-balance team creation.

IMO the best way to auto-balance without requiring it manually to be done would be to somehow artificially raise the 10* players to where their impact level is immediately known by the MMR bot. Everyone else is mostly not a huge deal. 10* players can swing games hard where it's not as big of an issue with the rest.

[Zeebu]

IMO the best way to auto-balance without requiring it manually to be done would be to somehow artificially raise the 10* players to where their impact level is immediately known by the MMR bot. Everyone else is mostly not a huge deal. 10* players can swing games hard where it's not as big of an issue with the rest.

qan's suggestion seems the best at the moment, just have the caps pick as even of teams as possible from the MMR pool for your particular match. Until the bot system itself is in place, it seems like an inefficient use of already slammed programming resources. Unless you mean something like just forcing played games into the system for certain players? Knowing nothing else on how this is set up, i would imagine that is doable, but it doesnt seem necessary. It's still just manually forcing something a certain way without data. I would definitely like to see something automated to address variance after a reset at some point, but currently it doesnt seem super valuable yet.

[Rab]

Would be good if we could run mmr games without needing staff... e.g. javduel and just signup people when they're added

[qan]

That's the eventual goal, to have MMR games run automatically. We're still in the testing phase, though. Having a staff member involved allows adverts, and the option to run matchmaking via MMRBot if desired. Auto-signup isn't that difficult to do.

However, as the jav/wbduel codebase is entirely separate from MatchBot (where MMR is implemented), it wouldn't work out of the gate to run in those arenas, other than by using a MatchBot there. I don't have the code in front of me, but I'm not sure if there's a method set up to allow players to auto-start games and claim caps in MatchBot.



Starting known 10* a bit higher would be possible, but I'd hesitate to do it without having a clear idea of what their ratings would be in MMR. MMR isn't linear, but rather statistical in nature, so the difference between 3000 and 3500 and the difference between 3000 and 4000 is not just "double" the difference. We could possibly just pre-load a slight boost to ratings of the highest-rated players without adversely affecting it, especially considering the small population and small number of games played. Looks like nobody has hit the 20 required to show up on the leaderboard, for instance.

We really do need to get more games going, though. My availability to dedicate larger amounts of time to this is running short.

[Riverside]

Would be good if we could run mmr games without needing staff... e.g. javduel and just signup people when they're added

If this is ez and quick 2 add 4 coding team, my assumption is that this would increase participation a fair amount

[Rab]

I was here when Dwopple won MMR

[Exalt]

I was here when Dwopple won MMR

I appreciate Dwop hosting these games, but last I checked, it was almost all jav games and he stopped as soon as he hit 20 games for himself. Is there nobody else on staff that knows how to run them? MMR won't take off if nobody ever hosts it, and when I asked a few days back, I was told that nobody knew how to do it.

[qan]

I appreciate Dwop hosting these games, but last I checked, it was almost all jav games and he stopped as soon as he hit 20 games for himself. Is there nobody else on staff that knows how to run them? MMR won't take off if nobody ever hosts it, and when I asked a few days back, I was told that nobody knew how to do it.

Thanks for the feedback. It's still a pretty new process. I posted a guide in a subchannel on our Slack, but it's an opt-in channel and many staffers may not have seen it. I've posted a link to it on the main staff channel in the hope that more will step up to host.

[Dwopple]

I appreciate Dwop hosting these games, but last I checked, it was almost all jav games and he stopped as soon as he hit 20 games for himself. Is there nobody else on staff that knows how to run them? MMR won't take off if nobody ever hosts it, and when I asked a few days back, I was told that nobody knew how to do it.

I'll be hosting some more today, but yes it'll be better with additional hosts. I had to stop hosting Friday so I could have a dinner date night with my gf and now that college football is back I won't be around much on Saturday's. Hopefully today I can get another staffer or 2 comfortable running matches.

[Exalt]

I'll be hosting some more today, but yes it'll be better with additional hosts. I had to stop hosting Friday so I could have a dinner date night with my gf and now that college football is back I won't be around much on Saturday's. Hopefully today I can get another staffer or 2 comfortable running matches.

No need to apologize or explain why you couldn't keep going! I was just pointing out that only you were hosting, and nobody else knew how to do it. Hookecho was doing it a bit last night, but he said he wasn't allowed to make subs to even out teams. It ended up working out alright, but it was a bit weird that he wasn't allowed to move 10* guys to opposite teams.

[qan]

No need to apologize or explain why you couldn't keep going! I was just pointing out that only you were hosting, and nobody else knew how to do it. Hookecho was doing it a bit last night, but he said he wasn't allowed to make subs to even out teams. It ended up working out alright, but it was a bit weird that he wasn't allowed to move 10* guys to opposite teams.

Probably was a misunderstanding, as the shortened guide says "no subs." But hosts can mix teams up as much as they'd like ahead of time in an attempt to even things out.

A few big improvements went in today that should make the process smoother. Hosts are now able to auto-add all players using matchmaking from MMRBot inside MatchBot. MMRBot can also now sit in the arena being hosted so that players don't have to move between arenas. Running back-to-back matches should be greatly simplified.

We're also looking at the possibility of starting players at different MMR ratings based on their DT star rating. We don't have a perfect mapping of stars to MMR, but even a bit of difference should help keep teams more even while people play their first few matches. Without fully-automated matches and a large playerbase, it's hard to get in enough games to have MMR start working well in a short amount of time, so even if this is slightly inaccurate, you shouldn't have starting 10* (and 6*) players throwing off the balance nearly as much in their first games. More information on the way on this.

[Exalt]

Probably was a misunderstanding, as the shortened guide says "no subs." But hosts can mix teams up as much as they'd like ahead of time in an attempt to even things out.

A few big improvements went in today that should make the process smoother. Hosts are now able to auto-add all players using matchmaking from MMRBot inside MatchBot. MMRBot can also now sit in the arena being hosted so that players don't have to move between arenas. Running back-to-back matches should be greatly simplified.

We're also looking at the possibility of starting players at different MMR ratings based on their DT star rating. We don't have a perfect mapping of stars to MMR, but even a bit of difference should help keep teams more even while people play their first few matches. Without fully-automated matches and a large playerbase, it's hard to get in enough games to have MMR start working well in a short amount of time, so even if this is slightly inaccurate, you shouldn't have starting 10* (and 6*) players throwing off the balance nearly as much in their first games. More information on the way on this.

What's been weird tbh is that in some of the MMR games, teams that should obviously win are still losing to significantly lower "star" teams. Games are not crazily blowouts like you would think. I'm not sure I like the idea of adjusting ratings to TWDT ratings for the simple reason that it kinda ruins the point of MMR in my opinion. Isn't the point to show that Wins/Losses are all that matters and K/D ratio or other things do not? I think it may almost be a fallacy to automatically move players to opposite teams even as we were doing before, because of the games I saw where Hoochecho wouldn't do it, teams were fine, and the teams you'd never expect to win were actually winning through superior teamwork, positioning, etc that don't necessarily show up on the stat sheets.

Just my $0.02 but I'd like to see how it plays out before artificially moving things around. People should have to learn how to win with random players who may not be the usual cast they play with in leagues. MMR just makes it so the teams would be theoretically even in the process.

Here's two examples from the other night: https://twd.trenchwars.org/showgame/90124466 https://twd.trenchwars.org/showgame/90124470

In the first example: 10*s (Best/Mythril), 8* Booker, 7.5* Monkee, 7.5* Ravage (44*) against.. 8.5*.absurd 7.5* Exalt 6* halfgoat 8* JURASSIC, 8.5* Zapata (38.5*). The guy from pub literally went 0-10 and Team 2 still won.


Second example: 10* Best/Mythril, 7.5* Exalt, 7* rabbit!, 6.5* suav (41*) vs 7* Aselsan, 8.5* Clark Kentaro, 7* Klean-X, 7.5* monkee, 7.5* product (37.5*)
Team 2 won.

In TWDT, having teams with * numbers that much lower would be considered sacrilegious and whoever was the captain would be clowned on by everyone. They in theory should never win any games with that significant difference in * count, yet it happened. It's almost as if * ratings aren't always accurate representations of skill or ability to win and people only think these teams are uneven because they don't want to accept that. I'd also like to say that even though the guy in the first game went 0-10, he stayed alive longer then 5 other players in the game, including one of his 8.5* teammates. That contributes something clearly that helps win games even if the playerbase will only see the 0-10.

[qan]

Good to hear that there have been some upsets. I agree with the "0-10 but still (somehow) contributing" effect, which is almost completely obscured by a stats-based approach.

The problem we face with accurately rating MMR is that it takes quite a while to get games in, and in the meantime, there will be some rather unbalanced games that people may need to suffer through. If they only play a few games at a time, most of their experience of MMR may be of this unbalanced state, and they may start to believe (incorrectly) that that's what MMR is. Trying to compare its level of balance to DT or TSL right now is almost meaningless, as there's very little data to go on and the bot is essentially picking teams almost at random.

I'm hoping the improvements made yesterday will help get games in much more rapidly, back to back. But we still need players to play, and hosts to host.

Note that in many MMR systems, data from past seasons is fed in to the current season as a starting point for quals (usually with a reduced effect). This would be equivalent to using stars as a starting point, though of course the mapping would be not as direct, as it would be a conversion. Starting a 10* a bit higher than 3000 would mean that we're using past information about skill levels to help give the algorithm a nudge in the right direction as the player qualifies, equivalent to winning their first few games of MMR. If they lose despite this small boost at the start, then the algorithm would quickly knock them back down to 3000 anyhow. The point is not to drastically skew how it functions, but to use data we have gathered over years to make those first few matches a bit more likely to be balanced. If we had the luxury of hundreds of active players and games played every day, it wouldn't be needed. Managing public impressions of the system is still important with a smaller playerbase, especially as alternative gamemodes in which players can create relatively balanced teams already exist.

In an ideal world, though, with far more testing going on, I'd love to just watch it run its course from the start with no assistance from data gathered previously.

[Grasps]

I am not sure if this is the proper thread to post but I felt inclined to come on here and say thank you for all the hard work on this MMR stuff. Also, many thanks to Beam, Qan, LF and some of the other staffers on lastnight putting in work for our enjoyment hosting games. I hadn't had that much fun on this game in ages guys holy! I stayed up way too late and truthfully if I wasn't an old geezer with responsibilities would of easily pulled an all nighter playing MMR matches. Absolute blast!

This system is so much better than the matchmaking based off ratings that we were using a few years back for the shitty league (TSL). Lastnight game were mostly tight and blowouts were minimal although in regards to blowouts I was playing mainly warbird so can't speak much on javs.

Anyways. Great work Qan and co. Again I just wanted to say thanks and give a little feedback. Hopefully this was the appropriate thread. I'll say I think this could easily take over TWD games if this became automated and there was a website that logged all the data for us to view and compete on. Squad based games will still be appealing and I don't see those going away but I also think just being able to jump into and play games seamlessly against people without having to mess around with challenging and accepting borrows will be a welcomed change for many. Something like a wbduel/javduel 2.0. Either way, it's a blast and am looking forward to the future :>

[Rab]

Congratulations Exalt, King of the Res Limit Gamers.