Boards›[ARCHIVED] Miscellaneous

Statistical analysis of the significance of rank differences: seeking test subjects!

CupcakeTrap··5 votes·742 views

TL;DR: I'm gathering test match data for a statistical examination of how the various ranked tiers measure up. Click here for instructions on how to participate.

The Question
Imagine that a big match will be played. All you know about the teams is that they've been created by randomly drafting Summoners based on their rank.

  • Team A is all-Silver (SSSSS): 5 random Silver players plucked from the master list of all Silver-Tier Summoners.
  • Team B is 3 Silvers, 1 Gold, and 1 Bronze (GSSSB). In other words, their advantage is that they have a Gold in place of a Silver, and their disadvantage is that they have a Bronze in place of a Silver.

You're placing a bet. They'll run 10 matches like this (fresh draw of players each time), switching off Blue/Purple side. Team A will always be SSSSS, and Team B will always be GSSSB.

Cast your vote here. (And click here for poll results.) Then read on to see what we're doing to put this to the test.

The Experiment
How could this question be experimentally examined?

Well, if you have a large set of matches, you can statistically assess how often various team comps beat other team comps. If you're willing to make some modeling assumptions, e.g. an additive theory of skill or a multiplicative theory of skill, you could even run a regression of sorts and try to compute a "power rating" for each rank.

Meanwhile, you'll also want to take a look at variance: it's possible that rank just doesn't really correlate with game-winning power in a particularly strong way, at least not in mixed-rank games. (e.g., the "Elo hell effect": maybe Diamonds get confused in an all-Silver match and play poorly because they expect their teammates and opponents to behave in a certain way.)

Preliminary Results
Right now, we need more data. However, I've been able to do some very rough preliminary analysis using data from Factions matches. (Factions is a community game mode based on custom matches of the "Noxus vs. Ionia", "Piltover vs. Zaun", etc. variety.) This data is obviously imperfect, but it's what we have.

Click here to read the preliminary report. We still need lots more data. The more matches we have, the more instances of each team comp we have, and the better the tests we can run. The preliminary results above, for instance, resort to a hack-ish "winrate for each tier" approach to squeeze some sort of conclusion out of a rather small dataset.

Play A Couple Custom Matches For Science
Planning on playing some normals tonight? Why not make one of them a quirky mixed-rank custom match? This will help expand our dataset with some non-Factions matches.

Click here for instructions on how to participate. It's really nothing more elaborate than joining a chat and either starting up a quick custom match with others there or being on the lookout for invites from match organizers.

Why Does This Matter?
There are a few reasons I'm looking into this.

  • Curiosity. It's an interesting problem, IMO, and I'm curious to find out just how big of an impact different ranked players have on mixed-rank matches.
  • Duos. Matchmade games are special because matchmaker has access to true MMR, but still, I've often wondered, when I get matched up with or against (say) a Plat/Silver duo, just how big the gap between them probably is.
  • Factions. We score matches in Factions. This allows us to use match outcomes to influence how the storyline develops. However, it doesn't seem fair that an all-Gold Noxian team being at all-Silver Ionian team should be worth as many points as a win in an evenly matched game. To properly adjust point values, we need data on how the tiers stack up.

Anyway, I hope at least some of you will replace the occasional Normal with one of these balance testing matches. Click here for instructions on how to participate.

9 Comments

Angry Monster2 votes

So question about the pool of players. When you say random players, how random? Like people fighting over the same role random, random premade group of 5, Or people like in team builder who never met but are getting roles that they want?

Each variation changes the equation of who will win. Example a Plat jungle can lead his team in a premade through strategy and calls. But a random group may or may not listen since their is a basic trust issue inherent with working with people the first time.

CupcakeTrap1 votes

Probably more like a blind pick no-duos Normal. It gets a little blurry in Factions, in that some of the Summoners probably know each other. You raise a good point about high-ranked Summoners: in Factions, the highest ranked people are often the best known, and probably get a reasonable amount of deference. That might help them to carry harder than they could in a pure blind pick situation. ("Hey, it's Nhan, let them play their Riven.")

Thoroniul1 votes

The line between silver and gold is really pretty thin. The only difference between gold V and silver I is that the gold V players got lucky to win 8 games worth of LP and got promoted. There is a big wall of gold elo hell players keeping silver people from getting promoted. The good ones who want to improve go higher into better tiers while the once who hit gold and don't care if they lose anymore hinder the higher silver tiers they are matched with.

The chance of having a really clean spirited with no troll, afk, dc, rage, toxic, and people who don't understand objective taking is rather low at the high silver level. It gets really annoying with a constant flux of winning and losing Lp without getting higher.

The difference between plat and silver is much higher. At the plat level you focus more on team composition, and objectives, and countering.

CupcakeTrap1 votes

The line between silver and gold is really pretty thin. The only difference between gold V and silver I is that the gold V players got lucky to win 8 games worth of LP and got promoted.

True, but we're not comparing Silver I with Gold V exclusively: we're comparing the entire Silver population and the entire Gold population. For every Silver I who's basically Gold, there might be a Silver V who's basically Bronze. (In part what we're asking is how it all balances out. It may well not be 1:1.)

Angry Monster1 votes

its obviously not 1 to 1 only 18% of the player base is gold, While 33% is silver.

http://www.lolsummoners.com/stats

Those that can not break out of silver 1 are not lucky, the y are not good enough. There is an actual skill gap.

CupcakeTrap1 votes

By "1:1", I meant that it's not clear whether Silver as a whole is closer to Bronze or closer to Gold.

Angry Monster1 votes

Ah well i cant help you there.

But from a older post(this season) Riot mentioned that the difference between B5,B4 and B3,B2,B1 was like saying challenger to players from D5 players. And that maybe B5,B4 should be another division in its self.

Also the difference between G5 and G1 is also huge. I fought from placements S2 to G1 currently. While the people in G1 are more arrogant ,but they are mechanically superior.

People want to believe there is no difference and it is all luck. I disagree. While some luck will always be there you can over come luck with skills.

MrSc0tty1 votes

Problem: role and champ variance. If you have a team with a bronze and you tell him to just power farm jungle Udyr, or stick him on Leona support he's much less likely to feed than if he was up against a silver in midlane.

CupcakeTrap1 votes

This is a good thing to notice. However, I think it's not so much a "problem" as it is a "factor". Indeed, part of mixed-rank gameplay is managing your ranks. This is an aspect of the Factions metagame: do you send your Diamond and your Plat botlane and try to knock the tower down at 10 minutes? Or do you have a Diamond jungler who can sneak around and snowball every lane with perfect ganks?

As for the general question, this is part of why I'd rather look at specific team matchups than individual tier analysis. "How much stronger is Gold than Silver?" may not be a meaningful question. e.g., perhaps a Gold on a team of Silvers and Bronzes is more likely to realize "hey, our Bronze should go powerfarm Udyr jungle". Maybe such a tactic wouldn't be necessary on an all-Gold team. If Silvers or Bronzes are less likely to realize this kind of metagame strategy than a Gold, then adding a single Gold to a Silver/Bronze team may provide a significant advantage.

In other words: perhaps part of being a certain rank is having a certain level of understanding of other ranks.

More from [ARCHIVED] Miscellaneous

Open Letter to Parents of League of Legends Players
[ARCHIVED] Miscellaneous860 comments
2,611
votes
Rewards for positive play
[ARCHIVED] Miscellaneous19,712 comments
1,337
votes
I don't like Boards
[ARCHIVED] Miscellaneous324 comments
809
votes
STOP ALLOWING LEVEL 1 ACCOUNTS TO SEND FRIEND REQUESTS!!!
[ARCHIVED] Miscellaneous291 comments
679
votes
Wanting to do something special for an awesome friend. Riot, please help? <3
[ARCHIVED] Miscellaneous220 comments
614
votes
People that are telling others to kill themselves has to stop! Please help me get this noticed!
[ARCHIVED] Miscellaneous605 comments
600
votes

More by CupcakeTrap

Riot should reword its "I Agree" leaver prompts.
[ARCHIVED] Miscellaneous129 comments
373
votes
Retcons suck: I'm not that far from wishing I'd never gotten involved in League's lore.
Story, Art, & Sound282 comments
249
votes
Link this thread next time Narrative makes a massive change with no prior community input.
Story, Art, & Sound60 comments
152
votes
Let's talk about Veigar's counterplay.
Gameplay147 comments
129
votes
"We just can't find a way to tell stories with this whole League thing in the way." Well, WE did.
Community Creations101 comments
104
votes
PSA to Kalista players: My hatred for you is without limit.
Gameplay118 comments
98
votes