Ranked Systems - The Flaws in ELO

Frenzicles·Sep 29, 2015·2 votes·1,212 views

Having played League of Legends for over 5 years, I've decided I wanted to share some "thoughts." There are too many problems in the ranked game mode - I will try to be as concise as possible!

I recently ranked up to Gold 3, lost 3 out of 4 games, then got demoted to Gold 4 again. In my last 2 games, I went Ezreal adc. I did the most damage on my team, I had the most kills, and the least deaths. I lost both games, and was flamed at by my team when I offered some advice (play safer, buy wards, try not to get caught out, etc).

As a Physics Graduate, I like my numbers and formulae, and I can tell you that whatever equation is being used to judge rankings is entirely flawed, since it seemingly considers only wins/losses. There are many other different aspects to the game, and I guarantee that more could be done to benefit those players who perform well despite being on losing teams. I implore Riot to either change this themselves, or allow me to help!

Furthermore, the matchmaking is almost insulting. I have never been banned or restricted. I play with a good attitude and do my best for the team, rather than flaming and giving up like so many players do in this volatile community. Yet I still get matched with players who obviously have been banned/restricted before, and who receive reports on a regular basis. On behalf of all non-toxic players, this needs to change! Put the flamers and toxic players in one basket, and put everyone else in another! I would happily experience larger queue times if it meant a more positive gaming experience.

I ask you all, am I being crazy? Or are Riot simply not doing enough to make their game fair? What are your experiences with ranked?

NOTE: I have developed a draft formula which judges the amount of LP a player should receive after a game (relative to his team mates)! Send me a link to a match in your history so I can test it on multiple games please! I can tell you the results too :)

22 Comments

Matthias9119Sep 30, 20151 votes

As a Physics Graduate, I like my numbers and formulae, and I can tell you that whatever equation is being used to judge ELO is entirely flawed, since it seemingly considers only wins/losses. There are many other different aspects to the game, and I guarantee that more could be done to benefit those players who perform well despite being on losing teams. I implore Riot to either change this themselves,

Elo is someone's name, not an acronym. For a "Physics Graduate" you don't seem to do much research on things.
There are a number of issues with using individual stats to determine matchmaking in a team game. While this might get people closer to their correct rating more quickly in some cases, it can create perverse incentives in others, which is really bad. (Basically, sometimes it becomes better for someone to throw the game and preserve their personal stats.) It can also stifle creativity and evolution of the metagame, since "off-meta" play might generate poor statistical performances even if it is effective at winning matches.
Riot has spoken a number of times about why "Prisoner's Island" approaches are not effective for behaviors like raging. Most "toxic" behavior comes from players who are normally fine but have the occasional bad game or bad day. Taking those players and putting them in a horrible gameplay environment makes things worse overall.

Edit:

Not to say there couldn't be some value in using individual stats to weight MMR changes. But the simplistic approaches people usually suggest (like basing it on KDA or assists or gold earned) are badly flawed.

FrenziclesSep 30, 20151 votes

Fair, I've just heard/used the term with my friends and assumed I was using it right. I've changed the text now, so thank you!
I disagree. If you introduce a "stats per death" type system (e.g. damage dealt/taken per death, kills/assists per death, etc) I feel this discourages players to feed, whilst motivating them to play smarter and perform better. If they go 9/1, then afk to preserve their stats, they will simply be reported and punished through the tribunal (in theory). Of course, I would also suggest the inclusion of wards placed, damage to buildings, when considering a player's gain/loss of LP (these are off the top of my head, the benefit of using equations is that you can account for everything ). I wouldn't include gold earned in MMR calculations.
I think if someone has been banned/restricted, this suggests consistent abuse of the game. Maybe in some cases there are anomalies, so I accept that to be fair. Maybe if someone has been restricted/banned 2 or more times in a season?

Matthias9119Sep 30, 20152 votes

If they go 9/1, then afk to preserve their stats, they will simply be reported and punished through the tribunal (in theory).

The problem isn't that they would actually AFK. It's that if someone is personally doing well but their team is struggling, they might decide to play in a way that best preserves their stats (which will minimize their personal LP/MMR loss if they feel the game is, say, 90% likely to be a loss no matter what they do) rather than playing in the way that maximizes their team's chance to win. You have to be very careful about things like this.

Another issue (since you started going into more specifics there) is that the kinds of stats you're talking about inherently favor certain roles and champions. Assassins and ADCs will, with the same level of player skill, almost always have better KDAs and fewer deaths in general. (Not to mention you now massively discourage any kind of initiation or sacrificial play. Trading an assassin or tank for an ADC late game is usually a positive, but if you punish people for every death you're obviously "misgrading" some plays. In short, some kills and deaths are much better/worse than others.) If you reward warding, people will just drop wards on cooldown no matter where they are. Rewarding damage to buildings arbitrarily rewards split pushing as "better", and most AP champs do very little damage to structures late game, which is when most structures go down. (Ironically, gold earned/minute probably has a stronger correlation with winrate than any of the stats you mentioned.) Did I mention that this gets suggested a lot?

The stat you're trying to measure is something like "win/loss contribution" -- sort of like WAR in baseball SABRmetrics (which has an extremely complex formula). But it's much easier in a game like baseball to say "you won because X and Y got hits off pitcher Z" or "you lost because Q made a fielding error". There are some things that definitely correlate with winning in League, but the actual causation is much more complex. Any formula that actually captured it accurately would likely be extremely complex and opaque. And if it was based on machine learning of past games I still wouldn't want to weight it too heavily, since it might totally fail to recognize novel tactics that are actually effective.

FrenziclesSep 30, 20151 votes

I think you are incorrectly assuming that stat-whoring (for lack of a better phrase) will lead to players gaining more LP than if they played for the benefit of the team. For example, let's take your ward analogy. If people just drop wards on cooldown, yes they will gain a small bonus. However, if it means that their ward isn't well-placed, or well-timed, they are more likely to die from ganks and experience larger negative effects in the long term. This is the same with all methods of "self-boosting" - the long-term effects in-game will probably be negative.

I completely understand what you're saying about favouring ADCs and assassins. However, if you consider damage taken and assists, everything balances out. You could even add different weightings on each stat to level it out, e.g. damage taken could have more weighting than number of kills. (Not saying this would/wouldn't work, just an example). The reason I would reward damage to buildings is because split-pushing is an "off-meta" strategy that could have positive effects that need to be considered. But it means that player is less likely to get points from kills/assists. With careful consideration, everything can be balanced.

Win/loss contribution is exactly what I'm suggesting, yep! I'm not saying it's easy, and I'm not saying it's perfect. I just think over a large number of games, player rankings would give a more accurate representation of their skill level than the current system.

NOTE: Bear in mind you would of course still consider whether the player wins/loses when measuring their LP, so some tactics that may go less recognised will still be rewarded if they result in a win.

VirtualWolfeSep 30, 20152 votes

Hey there Artist, fellow Physics graduate here. I frequently visit the community boards and rarely comment but your post inspired me to login and write a reply. The concept for a win/loss contribution system is really interesting and could be a great addition to the matchmaking algorithms for game queues. However, it has been suggested many, many times by the community, in far more detail than you have suggested so far here. As the other person in this thread suggests it is a non-trivial augmentation to the MMR system deserving of far more in-depth discussion than that which falls out of "I think" or "maybe if" statements. That being said, you're somewhat on the right track to an improvement with your early ideas and I'd like to see you take some initiative with your talents and love of equations to put a concrete example together with real data. Riot provides a great API that you can pull some aggregate stats info from and play with.

FrenziclesSep 30, 20151 votes

Hey Wolfe, really appreciate the comment! I am currently in the process of recording all my matches and putting them into my algorithm to see what the results look like over many more games. I'm also tweaking it based on what I see/experience in-game, e.g. if a player does well but the algorithm suggests otherwise. The reason I'm mainly focusing on my match history is because I can see who does/doesn't play well, and compare this to my statistical analysis. I'd love to see other people's thoughts, e.g. post a link from your match history, tell me the order in which you thought players were (from best to worst), and I'll compare those with my results?

I know it has been suggested many times, I'm just trying to prove that it is actually possible! As you well know, you can do almost anything with equations!

Matthias9119Oct 1, 20151 votes

I think you are incorrectly assuming that stat-whoring (for lack of a better phrase) will lead to players gaining more LP than if they played for the benefit of the team. For example, let's take your ward analogy. If people just drop wards on cooldown, yes they will gain a small bonus. However, if it means that their ward isn't well-placed, or well-timed, they are more likely to die from ganks and experience larger negative effects in the long term. This is the same with all methods of "self-boosting" - the long-term effects in-game will probably be negative.

The point is that if you reward simply for "warding" (or any other purely mechanical action), you arbitrarily reward players for pushing a button. It only makes sense to reward players for actions/choices that actually cause their team to be more likely to win. And if warding helps you win, doing it well will increase your winrate relative to not doing it, which will cause you to gain more MMR and LP over time already.

The danger of this, IMO, is that in games that people think they're very likely to lose, they will essentially give up and just focus on anything they can do to pad their own stats. Like rather than defending their nexus (which would cause them to repeatedly die, lowering their stats!), go farm a bunch of minions/monsters, or knock down one of the enemy's undefended outer towers.

I completely understand what you're saying about favouring ADCs and assassins. However, if you consider damage taken and assists, everything balances out.

Properly weighing kills vs. assists is quite tricky. Some champions, for instance, have more access to AOE damage (or ways to buff allies) and will tend to get more assists in teamfights simply because of that. In some cases a player made a herculean effort to get the kill on a fleeing enemy, and in others they swept in at the last second and hardly did anything to actually secure the kill. Sometimes they only got those kill(s) because someone else drew off all of the CC and damage.

And taking damage is normally something you normally want to avoid. If you're suggesting something like "if you're a tank, taking damage gets you points, but it loses you points on an ADC", then 1) that's easy to game and 2) you're requiring the system to make quite detailed judgments about the metagame. Riot has generally taken pains not to enforce any sort of meta, and grading players on how they fill predetermined roles definitely does that.

The problem is that it's very difficult -- if not impossible -- to look at most stats in isolation and say what they're "worth". Sometimes you can definitely say that with all else equal, it's better to have more of something (kills, assists, CS, gold, objectives damaged/taken) or less of something (deaths). But those things are the product of complex decisions and gameplay situations, and so it's almost never the case that everything else is equal.

Probably the most common complaint I hear from players who want individual stats taken into account is that they "win lane but their team throws the game". And sometimes they really do have good KDA and CS, and clearly have the mechanical skills to dominate other players around their rank during laning. But those stats don't mean anything if they can't use that advantage to win games. In fact, sometimes what is happening is that they are picking lane bullies and laser-focusing on killing their opposing laner over and over and over (despite the diminishing returns from doing so). Those "great stats" they have in the early game are, in fact, exactly why they are losing in the mid and late game, because they are squandering their champion's power spikes padding their own stats rather than snowballing their team. No system that just looks at raw numbers and says "more kills = better!" can take something like that into account.

I just think over a large number of games, player rankings would give a more accurate representation of their skill level than the current system.

Arguably, rating players solely on the outcomes of games they were in is optimal over a large number of games. By definition, doing this takes into account all possible in-game decisions the player has made and cannot over- or under-weight any specific choices the player made. If MMR in solo queue is supposed to mean something like "how good you are at making your team win", the most direct way to measure that is by seeing how frequently you win (relative to the strength of the opponents you're going up against).

The reason you'd want to include other criteria is that game outcomes depend on far more than a single player's decisions -- namely, the nine other players in the match. It can take a lot of games for those "random" elements to even out and give a true picture of an individual player's ability. So if you can accurately and consistently figure out who bore the most responsibility for the wins and losses, you can increase/decrease players' ratings more rapidly, and get players at or near their correct rating more quickly.

But over a large number of games, I would not expect any fancy grading system to be able to do significantly better than the current MMR system. Over time, if you're better than your rating indicates you'll win >50% of your matches, and if you're worse than it indicates you'll lose >50%, and that will move you towards the most accurate rating for you.

FrenziclesOct 1, 20151 votes

The point is that if you reward simply for "warding" (or any other purely mechanical action), you arbitrarily reward players for pushing a button.

I think encouraging players to ward can only have (to different degrees) positive effects. However, it is still in the players' interests to ward appropriately, so long as you have an algorithm that does not "overpower" any spammable mechanics. Furthermore, defending the nexus may lead to more damage dealt/taken, and more kills/assists. If these are weighted appropriately, then the player will consequently suffer from taking actions against their team's best interests. Also, the system would still deduct points for losing, so players shouldn't be desensitised to defeat.

Properly weighing kills vs. assists is quite tricky.

Your points here are completely true, and it is difficult finding an accurate way of correlating stats to in-game performance. However, based on initial findings in the equation I've developed, I definitely thing it is possible (despite current flaws). Something I should note is that in my current formula, players are rated compared to their team median on all aspects (mean would be affected too much by outliers). The reason this is working so far is because of how champions in LoL are designed - less CC = more damage, or vice versa. So either a champion will get kills and deal damage, or get assists and take damage. So ADCs should be way above the median for kills, but way less for assists and damage taken (in theory). This can of course change depending on how good/bad the player is. I will keep fiddling until I have a system that accurately reflects player performances in ~90% of games. As for kill-stealing - this is why balancing the weighting between damage dealt, kills, and assists is key. Same thing for those players with sunfire capes and 1,000,000 assists simply for standing nearby. It links back to my earlier point, you can't give credit to players for potentially spammable mechanics . But by considering so many statistical aspects, this mostly solves the issue - spamming one thing will lead to a lack of another.

The problem is that it's very difficult -- if not impossible -- to look at most stats in isolation and say what they're "worth". Sometimes you can definitely say that with all else equal, it's better to have more of something (kills, assists, CS, gold, objectives damaged/taken) or less of something (deaths). But those things are the product of complex decisions and gameplay situations, and so it's almost never the case that everything else is equal.

I agree. Some stats have more behind them than others. However, in the vast majority of cases, kills are good, assists are good, deaths are bad (unless a player gets kills/assists before their death, which would be considered).

Probably the most common complaint I hear from players who want individual stats taken into account is that they "win lane but their team throws the game".

This is a problem. The scenario that I am torn on is something like this: Player goes 8/0 in lane Team-mates struggle, maybe die once or twice in lane Player doesn't use his advantage to help Team-fights are lost, but relatively this has little effect on the final rating of the 8/0 player

If it's not this player's fault that his team lost their lanes, should he take as much responsibility for the loss? Yes he could've done more, but in every game you can say "well this person could've done this better." The player will still lose LP for the loss, and repeated performances will lead to further LP loss unless they learn. However, they will just lose it at a slower rate - is that not fair?

Arguably, rating players solely on the outcomes of games they were in is optimal over a large number of games.

You would think! However, the fact that you start on 0LP after promotion makes a big difference. This is what spurred me to make this discussion - I was promoted, my following performances probably ranged between 6-8/10 (for my rank), yet I was almost immediately demoted. Above average performances do not necessarily lead to a higher rank, or if they do, it can be a far more painstaking process than it should be. If I win my next 5 games, get promoted, then lose the next 3, I haven't progressed in the ranking system, despite a 63% winrate! Surely a 20LP start would make more sense! This happens because the promo games do not contribute to LP, and Riot don't seem to have considered the effects of this.

But over a large number of games, I would not expect any fancy grading system to be able to do significantly better than the current MMR system.

It could definitely do better, having seen the results from my first draft algorithm, never mind with improvements! Like I said before, my target is to have a formula which is accurate in 90% of games. If it fails in 1/10, this is far better than Riot's current system which fails in probably 5/10 (my algorithm also re-allocates LP in winning teams depending on good/bad performances, e.g. if a bad player is in a winning team, they will get less LP).

JubbinautOct 1, 20151 votes

I don't want to get too into this discussion, since it seems to be moving well on its own, and I have a far weaker grasp of the underlying concepts of the MMR system than Matthias does. I do, however, feel that I should step in to point something out.

The last two paragraphs of your post, here:

You would think! However, the fact that you start on 0LP after promotion makes a big difference. This is what spurred me to make this discussion - I was promoted, my following performances probably ranged between 6-8/10 (for my rank), yet I was almost immediately demoted. Above average performances do not necessarily lead to a higher rank, or if they do, it can be a far more painstaking process than it should be. If I win my next 5 games, get promoted, then lose the next 3, I haven't progressed in the ranking system, despite a 63% winrate! Surely a 20LP start would make more sense! This happens because the promo games do not contribute to LP, and Riot don't seem to have considered the effects of this.

It could definitely do better, having seen the results from my first draft algorithm, never mind with improvements! Like I said before, my target is to have a formula which is accurate in 90% of games. If it fails in 1/10, this is far better than Riot's current system which fails in probably 5/10 (my algorithm also re-allocates LP in winning teams depending on good/bad performances, e.g. if a bad player is in a winning team, they will get less LP).

Indicate a potential point of confusion. Specifically, you seem to be conflating LP with MMR - they're two different systems. In fact, as I understand it, they have little to do with each other. To my knowledge, the only connection they have - and I've been told this is false - is that LP gain/loss is affected by one's MMR relative to the expected MMR of one's current rank (so, a player with Gold-level MMR who is currently ranked Silver due to issues with promos will gain more LP on a win, and lose less from a loss than a player who has Silver-level MMR, but is currently ranked Gold due to some lucky matches).

I point this out, because what Matthias is discussing is purely the MMR system (which is hidden). The functions of LP are very different, and not particularly relevant to his points.

FrenziclesOct 1, 20151 votes

Yeah fair point. From what I understand, MMR judges the amount of LP you win/lose per game, based on previous wins/losses and MMR of opponents. But with the algorithm I'm suggesting, you could essentially render MMR irrelevant and just take each game individually - if you play well in a game, you deserve more LP. However, if you have a low MMR, this won't happen. MMR, similar to LP, currently is heavily based on whether you win or lose, rather than your performance.

JubbinautOct 1, 20151 votes

[{quoted}](name=The Artist XCIII,realm=EUW,application-id=cIfEodbz,discussion-id=L8Rl8pXx,comment-id=00000000000000000001000000000000,timestamp=2015-10-01T21:41:28.603+0000)

But with the algorithm I'm suggesting, you could essentially render MMR irrelevant and just take each game individually - if you play well in a game, you deserve more LP.

But... see, this concerns me. Because LP - by itself - is meaningless. It's nothing more than a "visualization" of MMR - and frankly, not a very good one.

So, to me, that statement is not unlike saying you can get rid of all that pesky "Operating System" stuff, and just keep the pretty GUI - when the Operating System is what's doing all the work.

However, if you have a low MMR, this won't happen. MMR, similar to LP, currently is heavily based on whether you win or lose, rather than your performance.

It's based entirely on whether you win or lose (well, and your relative MMR to the other team's average, but that's kind of "duh").

Changing that... I mean, I guess it depends on what you want Ranked to measure - or what you determine "success" to be in the game. If you want to measure how consistently your actions can influence your team to win, the best and easiest way to do that is to measure your win/loss ratio.

If you want to use different metrics, you should know ahead of time that you're changing the goal of the game.

FrenziclesOct 2, 20151 votes

Yeah I've been thinking about this whole MMR thing (I realised I've got a variable in my algorithm which effectively equates to MMR, but is more performance-based than Riot's one). I think it could be useful for matching players of equal standard with/against each other.

But... see, this concerns me. Because LP - by itself - is meaningless. It's nothing more than a "visualization" of MMR - and frankly, not a very good one.

They're very similar - LP is just a transformation of MMR (+ win/loss points) into ranked points. It's the same in my algorithm, I just never realised until now (you inspired my epiphany! :p ). I basically made an equation to calculate MMR, then input this variable into another equation to transform it into LP. I assume Riot did something similar.

If you want to measure how consistently your actions can influence your team to win, the best and easiest way to do that is to measure your win/loss ratio.

This is untrue - you could be good 80% of the time, but win only 55% of the time. Surely this is an unfair reflection of someone's skill? Yes, on average they should increase their rank, but:

At a much slower pace than they deserve.
They may not increase their rank given that an "unlucky" run of games between promo games and just been promotion, will lead to immediate demotion.

Essentially, with a performance-based algorithm, you remove the "unlucky" or "lucky" aspects, and a player's skill is much more closely and accurately analysed. This seems like a fairer way to do things?

JubbinautOct 2, 20151 votes

[{quoted}](name=The Artist XCIII,realm=EUW,application-id=cIfEodbz,discussion-id=L8Rl8pXx,comment-id=0000000000000000000100000000000000000000,timestamp=2015-10-02T11:22:21.914+0000)

This is untrue - you could be good 80% of the time, but win only 55% of the time.

That strongly depends on how you define "be good."

For example, if you define "being good" as high gold income/high fight participation/low deaths, you could easily wind up with players who are "good" 100% of the time, but win 0% of the time.

Think about chess for a moment (since that's where Elo came from). If you develop a system that places some measure on every piece you lose, every piece you take, the number of moves you make, the number of times you move each piece in particular, and say, the number of times you are in check or put your opponent in check, you've created a fantastic system to measure a player's style and specific performance.

But that system would never be able to consistently predict how well a player can win - because that's a metric that's not measured.

And that's rather the point. The metrics you measure by become the goal of the game.

They may not increase their rank given that an "unlucky" run of games between promo games and just been promotion, will lead to immediate demotion.

Eh. I'm not a fan of the promo concept in the first place, so I don't want to get into that. Promos seem to require that a player either be lucky or massively over-skilled to get through consistently.

Essentially, with a performance-based algorithm, you remove the "unlucky" or "lucky" aspects, and a player's skill is much more closely and accurately analysed. This seems like a fairer way to do things?

If those aspects of performance are what wins games most consistently, yes. But they're not. Unless you're wanting to change the goal of the game, you'd have to create a system that fully analyzes every skill that correlates with W/L. Many of those (map awareness, decision-making, communication, coordination, etc) aren't quantifiable. Which makes them difficult to analyze.

But you can analyze the effects they have - y'know, winning games.

And in the short run, that can be - and often is - less accurate than may be wished. But in the long run, it's extremely accurate.

FrenziclesOct 2, 20151 votes

That strongly depends on how you define "be good."

When I say "good," I mean their general play was above average for their team. Of course, you need to consider map-awareness, communication, coordination, etc. You can't directly measure those, but you can measure the effects of these, like you said, except you jumped to winning games as the answer, which I think is incorrect.

Good map-awareness -> less deaths from ganks/more kills vs your lane or their jungler/more damage dealt to opponent in lane/more damage dealt to buildings, and many more. These are all measurable.

Good decision-making, communication AND coordination all probably correlate to all the above, which THEN leads to winning games.

in the long run, it's extremely accurate.

I disagree. This goes back to my point about playing well 80% of the time, but only winning 55% of the time. Of course, it is a team game, so I'm not saying you should have an 80% win rate, but I'm saying you also can't have a system which only considers wins/losses as a measure of performance because it's clearly inaccurate if you play well 80% of the time but only win 55% of the time.

Matthias9119Oct 3, 20151 votes

[{quoted}](name=The Artist XCIII,realm=EUW,application-id=cIfEodbz,discussion-id=L8Rl8pXx,comment-id=000000000000000000010000,timestamp=2015-10-01T19:43:21.444+0000) Something I should note is that in my current formula, players are rated compared to their team median on all aspects (mean would be affected too much by outliers). The reason this is working so far is because of how champions in LoL are designed - less CC = more damage, or vice versa. So either a champion will get kills and deal damage, or get assists and take damage. So ADCs should be way above the median for kills, but way less for assists and damage taken (in theory).

As for kill-stealing - this is why balancing the weighting between damage dealt, kills, and assists is key. Same thing for those players with sunfire capes and 1,000,000 assists simply for standing nearby. It links back to my earlier point, you can't give credit to players for potentially spammable mechanics . But by considering so many statistical aspects, this mostly solves the issue - spamming one thing will lead to a lack of another.

Hmm. That's an interesting approach, although I'm not convinced it will work in all cases or not be exploitable. But the idea of looking for players who exceed in multiple categories across their team might have something to it.

I agree. Some stats have more behind them than others. However, in the vast majority of cases, kills are good, assists are good, deaths are bad (unless a player gets kills/assists before their death, which would be considered).

I think the more fundamental problems you're going to find are:

The "value" of many stats can't accurately be determined in isolation. Deaths are almost always "bad", but sometimes dying is the least bad outcome (like maybe you died so the rest of your team could get away), and sometimes you're throwing the game.
Some stats are good at some times and bad at others. For example, you keep referencing "damage taken", perhaps considering it a proxy for something like "participation in team fights". But you might also just take a lot of damage because you're bad at trading or positioning. Even if you're a tank, with all else equal you'd want to take less damage.

This is a problem. The scenario that I am torn on is something like this: Player goes 8/0 in lane Team-mates struggle, maybe die once or twice in lane Player doesn't use his advantage to help Team-fights are lost, but relatively this has little effect on the final rating of the 8/0 player

If it's not this player's fault that his team lost their lanes, should he take as much responsibility for the loss? Yes he could've done more, but in every game you can say "well this person could've done this better."

You're explicitly proposing a system to dole out responsibility for wins and losses -- so yes, any "bonus" they're getting in rating should be based on how close they came to doing everything they could to help the team win. In extreme cases it might even make sense to give an extremely selfish player extra LP loss, even if they have a "pretty" stat line. If your system is incapable of doing that then it will definitely be exploitable.

The player will still lose LP for the loss, and repeated performances will lead to further LP loss unless they learn. However, they will just lose it at a slower rate - is that not fair?

If their "good stats" had no impact on the game -- yes, actually it is unfair in some sense to reward them for it. Doing that is effectively penalizing the person who took a late-game teamfight champ and got stomped into the ground because Mr. Lane Bully didn't help his team get to the late game like they should have.

Whether you phrase it as one player getting a bonus or another getting a penalty, you're effectively assigning more blame for the loss to one player than another. Keep in mind that rating gains/losses have to be balanced -- if you give one player more credit for a win/loss, you have to give another player less, or you'll break the rating system over time!

In a system like this it's beneficial to play early-game champs, or assassins/duelists, since if you're mechanically good you'll probably have a good stat line even when you lose. If you pick a team-oriented champ, and your team struggles around you or doesn't play effectively together, there isn't much you can do. Just that effect alone will screw with the meta in solo queue.

You would think! However, the fact that you start on 0LP after promotion makes a big difference. This is what spurred me to make this discussion - I was promoted, my following performances probably ranged between 6-8/10 (for my rank), yet I was almost immediately demoted. Above average performances do not necessarily lead to a higher rank, or if they do, it can be a far more painstaking process than it should be. If I win my next 5 games, get promoted, then lose the next 3, I haven't progressed in the ranking system, despite a 63% winrate! Surely a 20LP start would make more sense! This happens because the promo games do not contribute to LP, and Riot don't seem to have considered the effects of this.

The whole division system and the way promo series work is a separate thing. Some players definitely find it frustrating, although there are frustrating elements to almost any ranking system (mostly because almost everyone thinks they are better than their rank says they are.) Also, you do have protection from being demoted immediately after you promote to a new division, and a single loss at 0LP is not guaranteed to demote you.

If you maintain a 60%+ winrate over time, your MMR will keep rising, and eventually that will more or less force you into higher divisions until your winrate levels out. It may take longer than you'd like, but you'll get there.

It could definitely do better, having seen the results from my first draft algorithm, never mind with improvements! Like I said before, my target is to have a formula which is accurate in 90% of games. If it fails in 1/10, this is far better than Riot's current system which fails in probably 5/10 (my algorithm also re-allocates LP in winning teams depending on good/bad performances, e.g. if a bad player is in a winning team, they will get less LP).

What Riot currently does is very accurate over a large enough sample of games. (It seems like most people need 50-100 matches in Ranked to get close to where they should be on the ladder.)

The whole reason you'd want to have a system that tries to grade individual performance is that you can get better results in the short term. If you can do that very accurately, you can adjust players' ratings more accurately (and more quickly). But either way, eventually almost all players will end up in an equilibrium where they win ~50% of their matches.

The problem is that any sort of rating system based on individual stats *will distort how people play to some extent. For a system like that to be "worth it", it has to be extremely accurate. If you only use it to shade MMR gains/losses to a small degree (say, +/- 5%), then it's probably not worth the headaches you'll get from players trying to game it and screwing their teams over in the process. If you're going to use it to make big adjustments to rating gains/losses, it has to be pretty much impossible to abuse, and grade people correctly almost all the time -- even if they use novel tactics that you haven't seen before. That's a high bar to clear -- I'm not saying it cannot be done, but it's really hard.

FrenziclesOct 3, 20151 votes

The "value" of many stats can't accurately be determined in isolation. Deaths are almost always "bad", but sometimes dying is the least bad outcome (like maybe you died so the rest of your team could get away), and sometimes you're throwing the game.

Yeah that's true, although I'm not sure that happens frequently enough to have a massive effect on a player. Most of the time if one person in the team dies in that type of situation, it's because they got caught out rather than sacrificial reasons.

Some stats are good at some times and bad at others. For example, you keep referencing "damage taken", perhaps considering it a proxy for something like "participation in team fights". But you might also just take a lot of damage because you're bad at trading or positioning. Even if you're a tank, with all else equal you'd want to take less damage.

Hmm yeah, but I think the lack in mechanics would be reflected in any lack of damage dealt/kills/assists. So again, by taking into account multiple statistics you basically end up doing worse overall.

Whether you phrase it as one player getting a bonus or another getting a penalty, you're effectively assigning more blame for the loss to one player than another. Keep in mind that rating gains/losses have to be balanced -- if you give one player more credit for a win/loss, you have to give another player less, or you'll break the rating system over time!

Yeah that's why I compare each player to their team median for each statistic - there is a base amount of LP given/taken per game, then you add/deduct LP based on whether you did better/worse than the team median, so overall it balances out on the team.

In a system like this it's beneficial to play early-game champs, or assassins/duelists, since if you're mechanically good you'll probably have a good stat line even when you lose.

This is possibly true, but you will lose more than you win if you pick champions who aren't optimal for your team. Plus assassins/duelists tend to fall-off lategame unless they rack up gold early, so as the game progresses, other picks will start to give better overall stats than the assassin ones.

What Riot currently does is very accurate over a large enough sample of games. (It seems like most people need 50-100 matches in Ranked to get close to where they should be on the ladder.)

I was placed in Silver 5 at the beginning of the season. I am now at Gold 4-3 and still going up. This is after over 250 games. It has been a frustrating season to say the least. Maybe my experience is actually not very common, but I think something has to change.

The whole reason you'd want to have a system that tries to grade individual performance is that you can get better results in the short term.

I think it will give better results in the long term too, since it will get players to their appropriate ranks faster.

The problem is that any sort of rating system based on individual stats will distort how people play to some extent.

I honestly don't think it will change the way people play. The main emphasis will still be on whether you win or lose. Then adjustments will be made based on other things. But like I said, as long as you have an algorithm which doesn't reward spammable mechanics, there is no reason for people to change the way they play.

DaenOct 2, 20151 votes

Not to say there couldn't be some value in using individual stats to weight MMR changes. But the simplistic approaches people usually suggest (like basing it on KDA or assists or gold earned) are badly flawed.

What would your opinion be on Mastery being at least a slight factor in ranked progression, provided it works as Riot intended? The system takes a huge number of things into account, from damage taken vs received to ward placement and effectiveness. This would allow players that excelled overall to not be hit quite as hard for a loss and wouldn't really change the values for players that struggled (since they still lost).

FrenziclesOct 2, 20151 votes

As in, champion mastery? Does it consider all those things? (I'm curious, no idea what the point in it is lol)

DaenOct 2, 20151 votes

Yep, Champion Mastery. I believe it does include those things, though I can't be sure because Riot hasn't released the metrics. The point is that it basically shows you how you did with the champion in the role as compared to other people playing the same champion in the same role. Getting an S means that you played better than the vast majority of players with the champion as per the metrics the system uses, essentially.

FrenziclesOct 2, 20151 votes

Ah interesting, if that's the case and mastery can do this successfully then that would be cool to consider.

Matthias9119Oct 3, 20151 votes

I don't feel like I have enough information -- Riot would need to expose more about how it grades players. AFAIK it measures your stats against other players with that champ and role. At least to a rough degree -- it probably decides if you're jungle/lane/support based on items and CSing, but I don't know if it could properly grade, say, double jungle.

If it's solely based on individual stats, I would have the same concerns about stat-whoring, and strategies that result in worse personal stats being underrated (things like "proxy singed"), and "off-meta" strategies potentially being underrated.