Tweet
Huge first night of proper MMR testing last night. Thanks to everyone who participated, and to Dwopple for hosting. We had games going for a good couple of hours, back to back, including some very close ones. In the parlance of our times: HYPE!
I'll leave the planning thread up for those interested in the technical details, but wanted to start fresh now that we've moved into the beta-testing phase. Now's the time to get in on the ground floor and see what this is all about.
What is MMR?
MMR is an algorithm used to accurately rate player skill and create even matches, based on the statistical system developed by Arpad Elo. Almost every single modern online game uses some form of MMR to balance matches. It's the industry standard, basically, because it works. If you've played an online game with strangers in the last 5-10 years, you've played using an MMR system.
In MMR, the only thing that affects your rating is whether or not you win or lose. This puts all the focus on team play. Even if you go 6-10 regularly, if the way you play still contributes to your team's chances of success, your rating goes up. In some ways, this is the opposite of TSL, which is focused on personal performance and particularly on huge individual performances in single games. That's still an indicator of skill, but an unreliable one that doesn't account for playstyles that don't always put up huge numbers but still translate to a high win %. For MMR's purposes, you can go 21-10 or 2-10 and there's no difference. If you win the battle but lose the war, you lose rating.
Your rating changes rapidly in your first matches as the algorithm attempts to place you. Then, as it grows more confident of your rating, those changes become smaller. Rating starts out at 3000. 95% of all players will eventually fall within 800 points of that rating, from 2200-3800.
EDIT: Barebones rating ladder site hacked together in 15 minutes from the TSL site (credit to Rab for original site design): https://trenchwars.org/mmr/
What does it look like right now?
Right now, the bot is in ?go MMR, and games are played in TWD arenas (currently twdd3 and twjd3). MMRBot handles matchmaking to create teams as evenly as possible. (Note that it isn't strictly necessary to do this, as any two teams can play one another. Stronger teams earn less for wins over weaker teams, and take a bigger hit to ratings for losses.) Staff may decide to balance teams to create more even matches, especially for players who are known to be strong but are still unrated. We can use a bit of our human understanding of player skill levels to create higher-quality games even for early matches.
To play MMR right now, you'll need to wait for a staffer who can host it. Lately we've been testing in the US evening on weekday nights, but that's just when staff have been available to host so far, and there have been players around willing to try it.
When a host is on, ?go MMR and make sure to !signup (either in PM or public chat as of today). Currently the bot is based on TSL, so it has all the same quirks as TSLBot — but note that this is not "just" TSL and is intended to be able to work inside any match type, eventually.
Then queue up by typing !p. The bot attempts to make the most even possible match given the available pool of players (though games are likely to be a little unbalanced if the majority of players have only played a few matches). After matchmaking finishes, check for which team you've been assigned to. This will be in the arena message and PM'd to you. ?go to the requested arena. Then either staff or a selected captain will add the line, and the game starts. Team 1 will have a frequency starting with a 1, and Team 2 has a frequency starting with 2, as in TSL. Games are single-round, so the next match with completely new teams, adjusted based on the results of the last match, will be just around the corner.
If you win, your rating goes up. If you lose, it goes down. Currently this rating message appears before the scorebox -- not ideal, but we're still testing. In the future, ratings won't be shown until you've played a certain number of games in that league/ship (probably 5), as your rating will jump around a lot at this early stage in a way that doesn't reflect your actual skill. Once you've played 20+ games, you'll start to get much closer to your real skill range, and after 100, you'll have a pretty good idea of how you rate compared to other players, completely independent of your star level, TWD rating or any other measurement. In the end, this will be the single most reliable indicator of true skill in Trench Wars. That's the real leap forward here.
Future plans
Comments? Suggestions? Ideas?
All welcome! I'll be going on vacation at the start of September, but I'll review any comments and take them into consideration after returning. You can also contact Dwopple, as he's been involved in the process from the beginning, being instrumental in making sure it could reach this stage.
Thanks for your interest in TW's next big leap forward.
I'll leave the planning thread up for those interested in the technical details, but wanted to start fresh now that we've moved into the beta-testing phase. Now's the time to get in on the ground floor and see what this is all about.
What is MMR?
MMR is an algorithm used to accurately rate player skill and create even matches, based on the statistical system developed by Arpad Elo. Almost every single modern online game uses some form of MMR to balance matches. It's the industry standard, basically, because it works. If you've played an online game with strangers in the last 5-10 years, you've played using an MMR system.
In MMR, the only thing that affects your rating is whether or not you win or lose. This puts all the focus on team play. Even if you go 6-10 regularly, if the way you play still contributes to your team's chances of success, your rating goes up. In some ways, this is the opposite of TSL, which is focused on personal performance and particularly on huge individual performances in single games. That's still an indicator of skill, but an unreliable one that doesn't account for playstyles that don't always put up huge numbers but still translate to a high win %. For MMR's purposes, you can go 21-10 or 2-10 and there's no difference. If you win the battle but lose the war, you lose rating.
Your rating changes rapidly in your first matches as the algorithm attempts to place you. Then, as it grows more confident of your rating, those changes become smaller. Rating starts out at 3000. 95% of all players will eventually fall within 800 points of that rating, from 2200-3800.
EDIT: Barebones rating ladder site hacked together in 15 minutes from the TSL site (credit to Rab for original site design): https://trenchwars.org/mmr/
What does it look like right now?
Right now, the bot is in ?go MMR, and games are played in TWD arenas (currently twdd3 and twjd3). MMRBot handles matchmaking to create teams as evenly as possible. (Note that it isn't strictly necessary to do this, as any two teams can play one another. Stronger teams earn less for wins over weaker teams, and take a bigger hit to ratings for losses.) Staff may decide to balance teams to create more even matches, especially for players who are known to be strong but are still unrated. We can use a bit of our human understanding of player skill levels to create higher-quality games even for early matches.
To play MMR right now, you'll need to wait for a staffer who can host it. Lately we've been testing in the US evening on weekday nights, but that's just when staff have been available to host so far, and there have been players around willing to try it.
When a host is on, ?go MMR and make sure to !signup (either in PM or public chat as of today). Currently the bot is based on TSL, so it has all the same quirks as TSLBot — but note that this is not "just" TSL and is intended to be able to work inside any match type, eventually.
Then queue up by typing !p. The bot attempts to make the most even possible match given the available pool of players (though games are likely to be a little unbalanced if the majority of players have only played a few matches). After matchmaking finishes, check for which team you've been assigned to. This will be in the arena message and PM'd to you. ?go to the requested arena. Then either staff or a selected captain will add the line, and the game starts. Team 1 will have a frequency starting with a 1, and Team 2 has a frequency starting with 2, as in TSL. Games are single-round, so the next match with completely new teams, adjusted based on the results of the last match, will be just around the corner.
If you win, your rating goes up. If you lose, it goes down. Currently this rating message appears before the scorebox -- not ideal, but we're still testing. In the future, ratings won't be shown until you've played a certain number of games in that league/ship (probably 5), as your rating will jump around a lot at this early stage in a way that doesn't reflect your actual skill. Once you've played 20+ games, you'll start to get much closer to your real skill range, and after 100, you'll have a pretty good idea of how you rate compared to other players, completely independent of your star level, TWD rating or any other measurement. In the end, this will be the single most reliable indicator of true skill in Trench Wars. That's the real leap forward here.
Future plans
- The most important improvement will be to have matchmaking built right into MatchBot. You'll be able to queue up and automatically get added to a balanced game. No need to change arenas, no need to add lines, no need for staff to set anything up.
- Squads will also be able to use MMR. We already have an Elo-based system used in TWD squad rankings, but soon you'll be able to develop your individual rating over time on any lineup, as part of any squad, or as a borrow.
- Subs cause issues with MMR. This is a problem in any game using this system, because an important part of determining how much your rating changes after a W/L is down to the strength of the two teams in comparison to one another. I have a preliminary algorithm worked out for how to handle this, but it will be a bit difficult to implement. For now, subs are not enabled in MMR. Please play your matches until the end. They're single-round so even if things are a bit uneven, you'll be on to the next match before you know it.
- Basing currently is not MMR-ready. Selecting a balanced basing team is a challenge. Fortunately, Rab developed a working algorithm to do this, and BIET implemented it inside TSLBot. It needs more testing before we can be sure it's ready to go in MMR, though. The same issue that causes MMR problems with subs also plagues switches, too, so we'll need to have a working solution for subs/switches before basing is viable. Captaincy is also an important consideration, as managing a team well is essential to basing success. For now, MMR is being tested in WB/Jav only.
- There is currently no website set up for MMR. Once we have a better idea of what we want to display and how to do it, I'm sure we'll find a talented web developer interested in making a site to view ratings across all leagues and ships. For now, you can use !rating <name> on MMRBot (or just !rating/!r to check your own record).
Comments? Suggestions? Ideas?
All welcome! I'll be going on vacation at the start of September, but I'll review any comments and take them into consideration after returning. You can also contact Dwopple, as he's been involved in the process from the beginning, being instrumental in making sure it could reach this stage.
Thanks for your interest in TW's next big leap forward.
Comment