Boards›[ARCHIVED] Miscellaneous

Elo rating/MMR and why it's not right for LoL

DocSmoothJew·10/21/2013, 8:46:17 PM·20 votes·2,869 views

Hey guys. I've forwarded this to Support as well but I'd like to hear you guys' comments on it too. It's somewhat revised since the language feels a bit weird when addressed to fellow players compared with Riot themselves.

A little about myself just for some context, I've been with LoL since before it was actually a game. I was in IRC before the beta even went live and my American account from beta (this one) is account #96. I love this game and quite frankly think it's the best MOBA out there. Where I think the game is lacking is in assessing the average player's skill level. Of course pros are assessed by tournaments and the like so that's not where I'm here to bring up.

As most players would agree the forums are beset with idiotic rants about the league system and I think that's largely why there's no word about it being changed; there's very little thoughtful, productive feedback on it that doesn't get thumbs-downed into oblivion. So, all of the childish screaming aside, the system is not good. While most of the complaints about it are screaming they should at least bring it to Riot's attention that customers are unhappy with it and that it should be changed.

Don't get me wrong though, the system isn't all bad. I think it's a fine way to run things for a team based ladder (5v5s and 3v3s) but when it comes to solo, it's a nightmare. I said it back in beta and a number of times since and I want to help Riot improve it.

To brass tacks, the system in solo is flawed simply because this is a game based on teamwork and a player's individual performance in a solo queue shouldn't be judged based on how four other random players do. So, an individual rating is required. That said, I don't think Riot should use some out-of-the-box solution but I'm sure the Riot team can come up with something that specifically fits LoL.

One thing Riot is doing well is curbing toxic behavior. It's a large reason solo queue is horrible and Riot's attacking that problem. There's still ground to be covered on that and hopefully the Player Behavior team gets there (looking at the upcoming Team Builder and praying it makes it to draft modes - ranked and normal).

That said the other aspect of solo queue not being fun is that even all toxic players aside you still can get stuck with players well beneath your level and just keep getting dragged down. It's all fine and good to look at statistics and say that in the long run a good player will rise and a bad player will stagnate in the lower tiers but that's illogical. Why should a good player be forced to grind ranked games to get out of the pits? The simple answer: they shouldn't. For a start, placement games for solo queue should be looking at more than just wins vs losses as it's even more random as to who you get placed with.

I would like to believe it's not hard to see if you simply look at where Elo rating comes from: Chess. Chess is a purely 1v1 competition, so it's understandable to say that over a significant amount of games that the player's skill level will be accurately represented by purely wins and losses. It just doesn't make sense in LoL where you are solo but are also required to rely on allies to do well.

Thanks for reading!

EDIT: It's like if you took a really bad baseball team and just ignored how great a single player on the team was because the team as a whole is not good. That doesn't happen. It's why you such players get snatched up in a trade. It's unfortunate for the bad team losing their one player but the quality player should not be dragged down because of the rest of the team. This is exactly how it goes in LoL right now and has gone forever; a single player may be of excellent quality but constantly stagnates in bronze or silver because of allies that are not of the same skill level.

Also, another thing to clarify is that promo series' would be obsolete in a system with a skill rating. You would basically have everyone start the season and play and after a set amount of time, maybe after 'x' players had played at least 'y' games it would break everyone by skill rating into the 5 main tiers (Bronze through Diamond) and then as people played their first 10 games (can keep that number from placement matches now) they would get their skill rating evened out and be placed into whichever tier their skill rating fits. If their skill rating is above Diamond they start the Challenger tier for example. The way you rise would be by increasing your skill rating to match the players already in a certain tier and that's it; poof, done.

One thing that would be great with such a skill rating would be that demotion would be incredibly hard. Since the rating is based on skill, you cannot get yourself carried. You must perform well or your rating will reflect it. No Diamond buddy carrying you from silver to gold, etc.

62 Comments

Dgreentheawesome10/22/2013, 1:58:45 AM8 votes

I feel rather strongly about this issue. As a rather bad silver V player, I was unable to drop to Bronze 1 when I first started playing ranked, going on a losing streak. This lowered my MMR to abysmal depths.

Now, after watching a ton of youtube, lots of normals, and some 1v1's with my plat friend, I am winning 66% of my ranked games in the past 5 days, when I started seriously playing ranked again. However, due to my MMR, winning 66% of my games is enough to stay at exactly 0 lp. Seriously. I gain 10 - 11 lp per win, and lose 20 - 21. Any system that makes a player stay at the current rank with a 66% win rate is, IMO, broken. I think the idea is GREAT, but the execution and math behind it needs work.

TLDR; Silver V with dismal MMR stuck with 66% win rate, system needs work.

EDIT: After posting this, I went on a four win spree in ranked. Haven't lost yet, but I will report here what the loss is. I think having 2-4-19, 1-12-19 (not proud of that) 2-3-21, and 3-2-4 support games should raise my MMR a bit.

EDIT AGAIN: I think its important to note that my total ranked win rate is 35-56 or about 39%.

Kufkus10/22/2013, 1:18:36 PM3 votes

I disagree with your post in more than a few ways. First and foremost, your baseball analogy. In baseball, its a 162 game season, and if youre on a team, unless traded you are stuck with THOSE EXACT people. In league you get a new batch every single game. Thus meaning, the constant in all of your games is YOU. assuming match making does its job and pairs 10 people of similar mmr ( and you are better than the rest as some claim) then you should win more than you lose. And good players need to prove that they are good through rigorous tests, countless games. You cant just say "im good" and get gold. You have to showcase higher level mechanics than your opponents and better game knowledge on a consistent basis. No game is truly unwinnable is soloq. I've won countless 4v5 games, and games where we were down 20+ kills and down 2 inhibs. Being a leader for your team, maintaining confidence even when behind, and showcasing higher level mechanics ( and knowing what to do with the gold that results from it) always wins out in soloq. Any individual rating system that isnt somewhat related to wins and losses is fatally flawed.

TL;DR: OP makes a few good points, but the constant in soloq is YOU and YOU ONLY, you get new teammates every game, if you do belong higher than you are you will win more than you lose, thus climbing the ladder.

Solan Stonewhip10/21/2013, 9:20:33 PM2 votes

fascinating an interesting perspective.

i feel the same way. but i'm a lowly silver III. i decided i finally felt strong enough to try ranked half way through the season. and got placed there. despite having leavers/afk in 1 of my 10 placement matches and a toxic surrender in the 3rd of my losses.

it's a difficult problem to attack. the only thing i would have thought would make it better would be would be to take into account kda and cs (how you distiniguish that from non carry roles i don't know). and to use that number to compute some kind of off the board score. with enough statistics you could possibly do this. it would mean giving each and every possible KDA a specific value, or create some kind of range of values. then you'd have to apply some kind of formula to take into account the outcome of the game as well. maybe even a constant to include the experiences level of players involved? it sounds like a bloody night mare. a real math numb3rs guy could maybe come up with something.

the only other thing i could think of. is keeping the elo system. but having placement matches and promotion matches would be done differently. they would have to be reviewed by an independent minded group of summoners, who didn't know who was being promoted/placed and they would be graded on all the things that shows experiences and team work. pings. cs. warding. positioning. item purchases. ugh, way too complex.

maybe there needs to be a separate series of tests in addition to the placement and promotion series. like you have a certain number of points or something. and every time you miss a creep kill then you lose some points. and the waves keep coming, getting bigger and better until you run out of points. you're stuck at base items too. that could be one score. i like the idea of their being a duelist test as well. maybe another separate test for jungling?

TL;DR So bottom line - add further 'test' modes such as CSing as adc. duelling in all lanes and a jungler test. more placement matches which also take into account the role played and the KDA/cs level

i dunno, this is not an easy proposition

SmokingPuffin10/22/2013, 1:50:57 AM2 votes

There's a lot of text here and I don't particularly disagree. However, you didn't really propose anything better than what they're currently doing.

I am pretty scared of creating any system that rates players based on in-game performance other than wins and losses. This creates an incentive to make the play that makes your stats look good, rather than the play that maximizes your team's winning chances.

For example, if you decide to reward people based on their KDA, you'll find that nobody wants to play a split pushing top anymore, even in cases when that's the right strategy.

Alankevis10/22/2013, 1:53:53 PM2 votes

First of all, reading these kind of threads makes me feel like you are harassing Riot to be honest. Let's admit most players who complain about the ranking system are GOLD and below. Let's me talk about the first post first. If you are saying that the system is flawed, then what do you suggest in return? Let me say this clearly that YOU ARE just a player who enjoying the game, meanwhile Riot is the one who work on the game EVERY SINGLE DAY to make it better. How would you go about and judge their works based on your performance? tl;dr: If you want to make an argument, make sure you are able to point the flawed point as well as a solution for it. If you can't provide the solution, then your argument is not valid. Second of all, psychotically, people judge their own abilities higher than they actually do. Let's say you don't think you belong in this certain league, ask someone who is your friend at a league that you think you belong to and spar with them to asset your real skill. I'm not saying sparring is just 1v1 by all mean. There are many different ways to test your skill. Trust me when I say this, there are a big different from GOLD 5 to GOLD 1 and a big different from GOLD 1 to PLAT 4 and so on. Agree that there are trolls and afkers in this game, but if you are on a losing streak in ranked or normal, look at those game and tell me what is the one thing that is common on those games. It's YOU. If you can't see your weakness, I doubt that you will improve at all. That's just my 2c.

GundayMonday10/22/2013, 3:49:27 PM2 votes

I honestly don't think Riot should fix this. Yes matchmaking is frustrating when your teammates suck. But look at it from a Riot business perspective. This current system leads players at all skill levels to think:

"I am better than this. It's just X holding me back, not me. I just need to play more games."

If Riot fixes this, yes a few players will be happy that they're at their true skill level, but it will be interesting to see what happens to overall play time (My money's on it decreasing).

Just something to think about. Yes Riot wants to improve player experience, but at the cost of sabotaging their "success"?

Tiara Azmalan10/22/2013, 8:28:28 PM2 votes

My big issue with the MMR league system is that, as a support main in bronze tier it is almost impossible to carry yourself out of "The Pits" (aka elo hell.) My mantra has essentially become WIN LANE LOSE GAME because I'll work so hard to push the enemy team out of lane, let my Marksman get farm and a couple kills, push the tower down, and maybe even secure a dragon. Then we shift to teamfighting and I can no longer make plays, my teammates dont listen to my pings, and no matter how much I ward SOMEBODY will run off by themselves to facecheck the enemy jungle while the rest of us are defending base or trying to push a lane. It doesn't matter how well I do in lane, after that phase I can never assassinate the enemy carry, split push towers, or really do much of anything to effect the enemy team without a reliable partner.

I started at Bronze II and dropped like a stone to B5. Ive recently switched to mid/top and despite the fact that I do MUCH WORSE in the lane I win twice as many games as I used to just because I can reliably impact team fights. Kayle, carry me to gold!

DocSmoothJew10/23/2013, 6:32:44 PM1 votes

Please keep upvoting this so it gets to Riot's attention! :)

Thanks for everyone's feedback so far.

DocSmoothJew10/24/2013, 10:43:33 AM1 votes

A lot of doubt has been cast on how a skill rating could account for things like split pushing. Here's the thing, currently how do most people look at performance in game? Their K:D:A if you ask me. So, how does that even factor in split pushing? It's not like you're getting assists for split pushing. Tower stats aren't really looked at in the event your split actually takes a tower.

The more I think about it, the more I realize we simply don't currently evaluate a player's performance or overall skill accurately.

FofoLindo10/25/2013, 2:51:50 PM1 votes

Im sry for text wall. I also fear Im off topic. If that’s the case, pls say so and I will stop posting on your topic. When I said no better method will come out its because there's no literature/documentation of better methods available that I know off. But I might have missed it. Easily.

ELO and team skill measuring is an area im not an expert but I have some knowledge of it. If you do have some pointers to documentation on this subject, please send them to me, I always look forward to improve.

That said, when you need to select a system to make automated measures of skill in team based activities, you always end up in ELO system variations.

The variations came from what KPIs you consider on evaluation, what segmentation you use, the weighting of decaying factors, and some other factors that are too boring and technical to mention here.

ELO applied to teams, like any statistical method (and like ELO for individuals), will give a precise measure of skill only with infinite number of games played. ELO in 5x5 teams needs much more games than individual ELO to get a good measure of skill, and it gets worst when you consider the different roles and champions involved.

So, the challenge Riot faced is to find a method that gives you a reasonable measure of skill in team based activities in a reasonable number of games. A skill based system, like you propose, although very good in theory, has many problems that seem unsurmountable, some examples:

  • It has to be closely aligned with meta. If meta changes, or some team decides to troll, different measures must be considered…

  • How to measure a support skill? Number of wards, zoning, ADC protection, …

  • How to measure a jungler? Number and timing of ganks, blue buffs given/taken, enemy jungle CS, …

  • How to measure ADCs skill? CS ratio, kills, deaths, team fights positioning, …

Unreasonably complex. Completely unfeasible.

We cant really discuss how good Riot algorithms are because we dont know them. But the key here is “reasonable”. You have also to consider the queue times. If a system always aim perfect matches, expect a rather long queue time for extreme cases (challenger or diamond 1 for instance).

I would dare to say, without elaborating on that, Riot method is good. Might be better but seems to be doing its job.

Is it possible to improve? Sure. But how?

To better measure skill, a simple (in theory) solution might be to break down things and separate ranks by role and champion. But that would become unmanageable in no time and:

  • would not give you a single and unified measure of skill. If you are measured separately on mid role and support role how to merge those measures to get a single one???

  • would not handle standard meta-gaming deviations. All mid, jungle invasions, no jungling, lane swap, …

The problem of ladders in LoL does not come, IMHO, from MMR/ELO, but from other factors related to the mixin between a cosmetic league organization and a (de facto) ELO based matchmaking. These impedance mismatch gets worse because:

  • No demotions. In theory a good thing to prevent ladder anxiety, but caused a much worst problem.

  • Opacity of the system (Hidden ELO/MMR). Again with good intentions, to reduce toxic behavior but much more was lost than they gained…

  • Badly explained/documented LP clamping in promo zone. That relates heavily with previous point. Why am I being clamped?? MMR!

  • Promotion series (dont fit well in ELO/MMR system). They depend too much on luck/bad luck. It you got to 100 LPs playing with higher MMR, no need to play promos.

So, IMHO, Riot should really revise their system. On that I fully agree with you, it’s a broken system.

My preferred options are:

  1. Keep MMR

a. Add demotions: 0 points => demoted

b. Show MMR/ELO. Why was it hidden anyway?????

c. Show or document when a match is about to be created with less balance. Its understandable that to minimize queue times, the matchmaking system relaxes some parameters like balance. But make that visible and plain. This way, if I don’t want to play an unbalanced game, I just get out

Rokkugo10/23/2013, 5:00:55 AM1 votes

The change of ranked would take a rather large amount of time to implement but i agree needs to be done. I am tired of getting scores when I clearly tried to win and end with like 17/4/5 and end up losing because my team refuses to communicate or loves to do toxic and stupid stuff to make the game a lose. Having a skill based score would fix a lot of problems with ranked inducing forcing a lot of trolls and toxic players to play together and be stuck in "ELO hell with their kin". In short i hope RIOT switches to a skill based system such as Kill and assist totals and ratios play a role as well as deaths and CS score because all are very important parts of the game as any pro player will tell you.

General Elephant10/23/2013, 1:59:06 PM1 votes

Here is where fallacies come into play. I'm not going to go all "logic 101", but I am going to say that people seem to be missing the fact that players have a lot more than just a simple game of baseball to manage, and because of this, there are very large peaks and valleys in player skill. Just last night I saw an extreme case of this while watching my B1 brother play a ranked game. He was nami supporting a twitch because they synergize really well together. This twitch was farming like you wouldn't believe, but when it came to actually fighting, he fell flat and really burdened the team, even with a 100cs lead he was still behind in gold because he died, and never caught up to their adc even though my brother landed his qs, and the jungler laid out cc, the enemy would get away. They lost the game because twitch felt "i'm a hyper carry, I gotta farm all day erey day!" Which is false, you need both to be a good adc. Anyway, had he played a top lane champion like nasus, this strategy would have been much more suited, and they probably would have won the game.

TL;DR I guess my point is, player skill varies widely even in an individual player, so assigning a single value to that is somewhat falling flat.

My suggestion is to have a rating like that of a credit score. 3 seperate sections that take into account different factors, and maybe divide it up a bit more. I'm thinking a 2 tier, 3 section that comprises a composite score.

Ie: composite score=kinda mmr, which is determined by the PvP, objectivity, and w/l, each being normalized around 100 every game, and being dished out in various amounts upon completion, which then affects the cumulative composite score without diminishing return, as the current system has. I'm probably going to write a separate post to lay out this idea a bit more now that I think about it.

BaTe67110/22/2013, 10:19:49 AM1 votes

The solution would be simple: Make a 1v1 map for all the solo que players to rank up in. Sure League of Legends was made to be a team based game, but unless you have an actual "team," the game is more like toxic players raging, trolls trying to get attention, and real interested players leaving in frustration because they're the only ones on their whole team who is doing any real work. If Riot plan's on continuing with the elo system, at least give solo que players a ranked stage where they can advance based on their level of skill and not on the people they get paired with. The 5v5 and 3v3 maps will always be available for those looking for a challenge, but at least give the solo player an area where they can rise out of the toxic tiers and on the level where their skills will be much more appreciate.

Here's my take on in: just like the boot camp stage back in Call of Duty, Riot should implement a ranked system where solo que players can play 1v1 games at least to get out of Bronze and Silver. After that, they then can only play normal 1v1 games but not use it to rank up. That way, people cannot abuse the system to climb to challenger tier. Sure I see a lot of flaws with this idea but its just a suggestion. I hope someone will be able to see this and help those who really want to get out of the lower elos and play where the skill levels are really counted.

Because lets state the obvious here: Elo Hell is real. It causes many newcomers to the game to quit because lets face it, in their level they will never be able to get out of Bronze, maybe out of Silver, but not Bronze. All that time they invest playing to get to level 30 only to be placed in Bronze because he was paired with the wrong people during his qualifiers. I just don't think it's fair. There are people who can be really interested in the game, but the people who welcome them into Bronze are players who will cause them to quit. No one's going to start playing ranked from Gold unless they're really experienced.

All I'm saying is give the noobs a chance. Elo Hell exists, and it's holding back potential customers due to frustration. Do something about it or all you're gonna have are smurfs.

tl;dr Ranked 1v1 for solo quers to get out of Bronze. The Elo system is flawed.

ArisenHero10/22/2013, 12:52:08 PM1 votes

You have some good points about the elo system, it is not accurate for a team game. But, if riot implements what you suggest it will ruin the game. If people know that they are being rated as individuals it will encourage more selfish play. People will be less willing to jump in and save a teammate at the risk of their own life, and the number of ks's will probably skyrocket. Moreover, how do you compare the stats of a carry vs. a support or tank? Unfortunately what you propose is nearly impossible for riot to implement however I was reading a thread yesterday that may be of use to you:

http://community.na.leagueoflegends.com/c/ideas/RhXUKsq5-league-of-our-own-making-a-more-automated-system-for-connecting-with-players?comment=0001

This thread basically talks about having a guild system for league. If we had a guild system players would be more committed to learning the game and how to play better. You have the veteran players in the guild teaching the "noobs". Yesterday I played a ranked game and I played adc bot lane and my support (Janna) did not know what they were doing. I told them that I wanted to play safe until I hit 6 ( I was Ashe). They refused to take exhaust and took ignite, they pushed the lane by galing minions, and they refused to communicate with me. We lost the game real hard, and its not like I played perfect, but my biggest frustration was that this person had no reason to listen to me no system in place for encouraging them to do what they were told or learn how to play better. There was no reason to report them because it just appeared that they weren't that good. And apparently the hope of getting honor at the end of the game meant nothing. We need something else to ensure results on the battlefield. I don't know if guilds are the answer but we need something.

EDIT: I just looked at the stats on that game I played as ashe: Their team consisted of three bronze 1, and two silver 5 ok that's fine. Our team consisted of a silver 4, a silver 5 (me), two bronze 1 and a bronze 3!! I'm sorry but it is impossible to win a game at that elo trying to carry a bronze 3. The skill level of bronze three is so much lower to bronze one then compared to say silver three and silver one. Riot pls.

Fawlty10/22/2013, 2:18:37 AM1 votes

Actually like the OP said, I really hope (and think) this Team Builder thing is going to fix a ton of the issues. I still would rather see the Elo system back than the League System, but the League may not be so bad with Team Builder in place, though it would still need some serious work.