Boards›General Discussion

How many samples you need to take a win rate seriously

rtbf76109271·10/16/2018, 6:33:35 AM·9 votes·6,464 views
Understanding Binomial Confidence Intervals - SigmaZone

The "not enough samples" thing gets thrown around a lot. Clearly at low sample sizes, the sample may not represent the true probability due to random variances. As sample sizes grow, we approach the true probability of the event. Somewhere along the way, we get to "enough" samples to make a claim. There isn't really a single "point" where you have "enough" samples. What happens is the margin of error above and below what you measured the probability to be gets smaller and smaller. Based on that margin of error, you can make claims with a confidence level. The confidence interval shows the lower and upper limits of what our sample could represent, and is defined though choosing a confidence level. That is often 95% or 99%.

This would read something like this:

"after flipping a coin 10 times and getting 5 heads, we are 95% sure that the probability of a heads is between 19.0% and 81.0%" "after flipping a coin 50 times and getting 25 heads, we are 95% sure that the probability of a heads is between 36.1% and 63.9%"

I did calculate those numbers, and it obviously shows how much better our idea of the chance of heads gets with an increase in sample size.

Due to random variances, we will not usually see the true probability in the sample. Say you flip a coin 30 times, much more often than not, you will not get exactly 15 heads. However, the vast majority of the time, you will get CLOSE to 15 heads. The margin of error deviating away +/- from what we measured in our sample is called the confidence interval. We can calculate the lower and upper values of the confidence interval using the formula in the link I provided.

So lets look at Kaisa . She has 26 games, with a win rate of 69.23%. Lets use a confidence level of 95%. We have:

p = 0.6923 n = 26 z1- α/2 = 1.96 for 95% confidence

Plug that into the nice forumula, and we are now 95% sure that Kaisa 's win rate is between 51.49% and 86.97%

So, we are very confident that her win rate is NOT 50%. However it is possible she is reasonably balanced (51.5% win rate) and we happened to get a very win heavy sample so far at worlds at random. We will see after more samples what her win rate looks like. I imagine it will remain VERY high, and maybe within a 95% confidence we can say her win rate is >55%, which is by most peoples standards completely broken

12 Comments

BLACK REALM GOD10/16/2018, 6:42:28 AM5 votes

good use of statistics and probability. it's so sad when people use probability incorrectly.

DorkunedAuras10/16/2018, 6:52:09 AM4 votes

Thank you. I took Stats in high school and promptly forgot all of it. I'm glad you didn't do the same.

Just the Lip10/16/2018, 6:52:57 AM3 votes

OH my god you mathed me bro.

[slayer-pantheon-rainbows]

saltran10/16/2018, 9:37:51 AM2 votes

While that's an awesome maths work, if you are talking about Worlds winrates to determinate how strong a champ is I still think that winrate is a worhless stat and that what really matters is to see the games, with a game-knowledge analysis to apreciate how a champ impacts the draft (pick/bans) and how in game that champ is intended to work for the team that picks it + how the other team will try to play around it.

Vreivai10/16/2018, 8:51:16 AM1 votes

I'd like to see a similar response when people say "1000 games isn't enough to accurately determine a champion's win rate."

DJBlackVoice10/16/2018, 8:46:49 AM1 votes

I think a logistic model where 1=win and 0=lose would fit better this problem. Just use IC isnt enough for describe this problem. But for sure is this tard champ is broken, no reason for refuse that Hp =D

Jack Eron10/16/2018, 11:28:46 AM1 votes

I think nothing is enough to show a winrate. It depends on million things... also there is no "one champion's winrate" since in 99% you won't solo carry. Maybe you won your lane but the others were feeding. How will that one game look like in statistics?

What if i was awful but the team carried me. How will that one game look like in statistics?

If i had a counter?

If i had lag?

If i had bugsplat?

If we had afk?

If i had to go afk because of some serious reasons?

If there was a bug?

If i had a troll?

I could say tons of games where i was really good, i won my lane and right after a few kills i run to help the others. But in most cases ONE guy is enough to ruin everything and ONE guy is rarely enough to save the game. Show me statistics about how many % of bad games were saved by one guy. I highly doubt that the the amount of that would be higher than 10%.

And don't show anyone "high elo" stats. They are the lowest % of the whole community, so it doesn't prove anything.

Also what if my champ if very good for combos with the team but they don't want to cooperate? What if i had to build tank on a "not really tank" champ because others fed my enemy, or just we simply didn't have tank?

That's why i think that a champions winrate is irrelevant. There is no "your champ's winrate". You have a team and this means that you will lose even if you are the best and you will win even when you are the worst.

xZabaksx10/16/2018, 11:39:13 AM1 votes

Infinite

geeklove10/16/2018, 4:18:33 PM1 votes

I imagine it will remain VERY high, and maybe within a 95% confidence we can say her win rate is >55%, which is by most peoples standards completely broken

Nerf Kaisa

Fisherman Fizz10/16/2018, 6:50:51 PM1 votes

Saying youre 95% sure her win rate falls between those numbers is incorrect. A 95% confidence interval means that if you repeated the same sampling method 100 times, you would expect 95 of them to contain the true win rate.

You can't also say that her win rate won't change much based on that interval because you haven't accounted for which teams are picking her. Each team playing also have drastically different win rates that will influence how kaisa does each game.