In game client surveys?

vOmvbrawfU··1 votes·434 views

Obviously the boards only represent a small number of the League community. By nature of it, it also happens to typically be the upset "ermehgerd nerf dis" or "this champ is guttered buff them" people. (Who gets on to say, "wow, this is really going good right now!"? No1. Just like customer satisfaction surveys. You're more likely to get a negative bias...but that's neither here nor there.) We also have to take into account there are much more low elo players than high elo players on the boards.

Anyway, point being that it seems like everyone on the boards pretty much has a universal "stop adding fucking damage to everything rito" mentality. Rito keeps adding damage to things. Whether that be by champion buffs. the new mastery system (that everyone seems to universally hate as well), or champions like Vi being able to point and click immune to cc dive bomb your carry and 100-0 them while being tanky enough to get back out. So obviously rito believes that the people who think thunderloads and warlords and the new mastery system has way too much damage and is dumb are in very small numbers, despite it looking like it's the overwhelming majority on the boards. This also tends to lead people to thinking rito just doesn't listen to them. They'll say, "so i know the boards is extremely tired of Lee Sin being THE meta for 5 seasons straight, but we have evidence saying otherwise."....but then the players don't know what any of that evidence is lol.

So why don't they have in client surveys asking questions about the game. Just a lil pop up box with 1 to 2 questions when you sign on. It would take a total of about 3 seconds to answer a question about "do you think there's too much damage in League?". Then rito can have a very large sample size and report it. Like, well only 15% of players think Lee Sin is a problem. And they can even have accurate opinions from all ELOs. Like, 50% of bronze players think Master Yi is OP but 98% of diamonds think he is garbo.

5 Comments

DrCyanide1 votes

They regularly send emails with surveys out, which allows the varied opinions as well as more questions that can be answered to give deeper feedback.

vOmvbrawfU1 votes

Hm, I haven't gotten one. The only surveys I receive are follow ups to a support ticket. The "how'd we do" ones. So I'm guessing the surveys they send out in emails are also a small sample size, just like the boards.

DrCyanide1 votes

A sample size smaller than your entire population isn't a bad thing, so long as it represents your population.

Like you pointed out, the boards tend to think certain things are a problem, people on Reddit are likely focusing on slightly different issues, people who watch only streamers are looking at other problems, and people who just play the game without looking at all the external stuff have their own opinions. You then add the different elos people are in, and the view of the state of the game varies quite a lot.

The advantage of email is that it can survey people in all of those categories, without being biased towards any one in particular. An in client survey would be biased by "what answer will make this go away fastest", by multiple people in a party seeing the question at the same time, talking about it and biasing each other's feedback, and by people logging in with multiple accounts.

vOmvbrawfU1 votes

But who do they send these emails out to? Selection bias could become a pretty big thing here.

DrCyanide1 votes

I imagine they send the emails out to a subset of the people who have verified their email account, de-duplicating the list to account for people using the same email for multiple accounts, then using a random number generator to pick emails to send it to.

It's possible that they use a more complex method (maybe they try to screen for x% of active board uses, y% active Reddit users, z% neither), but honestly it's pretty unlikely that selection bias will be a significant issue so long as they're sending it to enough people. Considering how I've gotten 4 or more of these things over the last few years, I'd guess that they send them out to somewhere between 10,000 and 100,000 people in NA alone each time they send them.

More from [ARCHIVED] Miscellaneous

More by vOmvbrawfU