Example- Being reported by the same guys when you are matched with them multiple times in a row and reported each time for verbal abuse, getting the box notifying you that you have been reported even if you have not said a thing to them. Was just curious.
Look4 votesFalse reports will be disregarded by the system, and if that player abuses the report system the reports will lose value. If you did nothing wrong you can ignore those messages.
Vethykos3 votesThank you
RiotWookieeCookie5 votesTo build upon what Look is saying; don't think of a report as the end all be all when it comes to determining a punishment. They help our systems record patterns and look closely at specific games.
We're at a really cool point in League where our systems have a certain level of awareness. If toxic behavior is not detected, then no disciplinary action will be taken. Even if you received 9 reports in a game, you wouldn't have to worry as long as your behavior wasn't toxic in any way.
The more reports a person makes for silly things the less "value" the system will place in their report, eventually ignoring it all together.
Linna Excel5 votesWe're at a really cool point in League where our systems have a certain level of awareness.
Skynet was created by LOL to punish flamers? EVERYBODY RUN!!!
The more reports a person makes for silly things the less "value" the system will place in their report, eventually ignoring it all together.
This is something that scares me. I get why you do it, but on the other hand, if I'm having a streak where someone on either side is being toxic enough that I think it is worth looking at, I'm worrying if the system is starting ignore me because I'm on a bad streak when it comes to people in my games. Let's say you've got a 1 in 10 chance of having a toxic person in a game. Sure that translates to 1 in 10,000 for having 4 in a row, but there's 8 million LOL players or whatever. Hypothetically you should be getting that occur 800 times everyone of those 8 million people play 4 games.
So let's say whoever is right on his reports half of the time and the other half are borderline stuff riot lets slip. That's still 2 silly reports for 800 people. If anything, this makes me a little hesitant to report something that I think is questionable because I dunno how the system thinks of my reporting.
I'm not getting any feedback on them. Ever. No good job for a 90% reporting accuracy. No bad job if I only hit 10% of them.
So if I run into someone who's AFK but isn't red postgame, that's one of those things I worry about reporting.
RiotWookieeCookie5 votesYour concerns make sense to me. It's hard for me to answer because we don't want to provide specifics about how reporting works. Doing so opens up the opportunity for less honorable players to try and game the system.
What I would say is that what you described is something that's on our minds when thinking about reports and how they work. Similar to how our systems for bad behavior are based off of patterns rather than just single incidents, our view of reports is the same. The goal isn't to punish players who may report behavior they consider bad (even if that behavior is in the grey area), but instead to get those who use the report system as a retaliatory tool against those who are playing the game legitimately.
Rueian2 votesQuestion about the reverse end of Linna's question. Given a player who is has semi-consistent negative behavior, is falsely reported and then a punishment case is made. The person who falsely reports would have their report weight increased would they not?
Also how does the automated system handle false reports? I've seen recent reform cards with relatively tame games listed on them, which means the false report was used to build the case and not discarded.
Linna Excel2 votesWell, I'm glad you guys are thinking about this at least.
So as long as I'm not using reports as retaliation, I should be fine more or less?
Deep Terror Nami2 votesI think there should be some sort of indication that your reports are losing weight (or are now worthless). Could help show people that their reports are worth something, and if they aren't, show them that they need to stop reporting frivolously.
Madtepez2 votesThe more reports a person makes for silly things the less "value" the system will place in their report, eventually ignoring it all together.
Hey, it would be very nice to have some feedback on what value the system is giving us.
I understand (not in the sense that i really understand but in the sense that i got the memo) that you are not going to give us feedback on what happened to people we did report. But it would be really nice to at least get some sense of what the system "thinks" of our reporting.
I for instance report a lot of players, cause I think that even if they are not "toxic" in the way they don't insult or harass, i still think that calling out the entire team (noob, NOOOB, NOOOOOBSSS) 5 times in the game isn't fostering a positive attitude. By how quick people leave the end game lobby, i'm pretty sure that most people don't bother reporting those kind of people (maybe they don't care, maybe they think it's a waste of time). For my part, I think that if we want a better community, i should take the time to report them. But having no feedback i don't even know if the system agree with me.
I would be happy to tune it down (or up) and adjust the treshold of what should be reported or not if I had any kind of feedback concerning how valuable the system thinks my input is
MaleficChemistry1 votesTo build upon what Look is saying; don't think of a report as the end all be all when it comes to determining a punishment. They help our systems record patterns and look closely at specific games.
We're at a really cool point in League where our systems have a certain level of awareness. If toxic behavior is not detected, then no disciplinary action will be taken. Even if you received 9 reports in a game, you wouldn't have to worry as long as your behavior wasn't toxic in any way.
The more reports a person makes for silly things the less "value" the system will place in their report, eventually ignoring it all together.
The best thing to do is to just not talk altogether as the players of league of legends are the most sensitive people in the world. Riot caters more towards the afkers than it does to the people who get mad at afkers.
Case in point Dunkey was recently banned for cussing out a guy who was intentionally feeding and he was indeed reported by the guy intentionally feeding. Riot long ago gave up all pretense they had of making a game where a competitive spriit rewards you and would rather ban their long time fans than have some 12 year old tell their mom that a league of legends player called them a trash.
Don't worry about being the guy who doesn't say a word and farms lane or jungles afk because thats exactly what riot wants in their ranked system at this point. Claim you only speak swahili or something and all will be fine if you dont communicate or care about winning, then you can report other players when they cuss you out for not helping the team too.
Jaawn1 votesTo build upon what Look is saying; don't think of a report as the end all be all when it comes to determining a punishment. They help our systems record patterns and look closely at specific games.
We're at a really cool point in League where our systems have a certain level of awareness. If toxic behavior is not detected, then no disciplinary action will be taken. Even if you received 9 reports in a game, you wouldn't have to worry as long as your behavior wasn't toxic in any way.
The more reports a person makes for silly things the less "value" the system will place in their report, eventually ignoring it all together.
I really wish there was a way for us to know how much weight our reports carry relative to other players. It might actually be good to give some kind of feedback like that anyway. For example, maybe people who give out silly reports should be notified that their behavior has been noticed. Likewise, maybe players who give useful reports should be encouraged so that they continue to give high quality reports.
Jubbinaut3 votesHypothetically, it's possible. Any false positive case (wherein an innocent player is punished) must inherently have involved false reports. However, it's very rare - somewhere less than 1 case in 6000 (or ~0.01667% of cases).
As such, I don't think it's worth worrying about.
EDIT: Corrected a rounding error I inadvertently left in.
EDIT2: Apparently math is hard - I massively screwed up my calculations. But it's fixed now. Apologies for any confusion.
Lucifer2 votesWhich butt cheek did you pull those numbers out of?
Jubbinaut2 votesWhich butt cheek did you pull those numbers out of?
Well, I started with Lyte's quoted error rate from their spot checks of the system (posted 5 months ago):
currently hovering around 1 error in every 5000 cases to 6000 cases
Then I did math to find the upper bound of possible false positives (1 divided by 6000 = 1.66666667 x 10^-4 or 0.0001667, then multiply by 100 to get percentage = 0.001667%). I rounded strangely, because I'm strange. Actually, it's because I had originally intended to indicate that the 6 seems to repeat, but then decided not to, but didn't fix the rounding error that left.
If you have a correction to my math, feel free to post it. If you have concerns with Lyte's numbers, perhaps take that up with someone who might know something about it. I'm just sharing relatively-inaccessible information. Nothing more, nothing less.
Rueian3 votesCould be higher if there were many uncontested false punishments. Statistical Accuracy is entirely judged on what was contested and found as false.
Cant count something as a false positive if the person never stands up for themselves.
Jubbinaut2 votesCould be higher if there were many uncontested false punishments.
Again, these numbers are based on manual reviews initiated by Riot for the express purpose of verifying the accuracy of the system.
So the question of whether or not they were contested isn't really relevant, since player response (of any kind) isn't necessary for these particular reviews.
Rueian3 votesa handful of manual reviews are also grossly inaccurate when dealing with such a tiny sample size relative to the base data.
Jubbinaut2 votesa handful of manual reviews are also grossly inaccurate when dealing with such a tiny sample size relative to the base data.
I'm sorry, would you mind sharing where you found information regarding the sample sizes? I thought I'd been following his discussions on failure rate pretty closely, but I don't think I ever saw him mention how many cases they actually reviewed for testing - only that 1/6000 failure rate.
And a 1/6000 failure rate could mean they only tested 6000 cases. It could also mean they tested any number of cases greater than that.
Lucifer1 votesNo it doesn't. It can simply mean they picked up 5000 cases, and only one person was able to successfully argue (or tried to argue) they didn't deserve their punishment. GOOD LUCK arguing with a player support specialist, if you can even manage to get a reply that isn't automated, or one that will actually give you your logs.
Rueian1 votesNo he is right, I didn't provide my source to their sample size because riot doesn't give out that kind of information.
You can get logs, you just have to be persistent.
Jubbinaut2 votesNo it doesn't. It can simply mean they picked up 5000 cases, and only one person was able to successfully argue (or tried to argue) they didn't deserve their punishment. GOOD LUCK arguing with a player support specialist, if you can even manage to get a reply that isn't automated, or one that will actually give you your logs.
Presented again:
Well, it's not going to be a satisfactory response until it's in a paper somewhere, but we basically ran several weeks (months even) of tests where Player Behavior and Player Support teams flew into LA and hand-reviewed thousands and thousands of cases. Team members had to review the same case independently, and then we calculated everyone's consistency and reliability. During these tests, the system was recording live data and making decisions about games without actually handing out penalties. We'd then hand review the data, with multiple team members reviewing each case. From this, we saw approximately a 1 in 6000 case globally where we saw a game or chat log that had 0 wrong-doing and did not deserve a penalty.
Also, you may have missed it, but I did mention that it could simply be a matter of them reviewing the minimum necessary to see that result. However, given the length of time he refers to here, and running a (probably inaccurate) estimate on the number of cases generated per day (~1% of players are issued punishments, ~27 million daily players = ~270,000 cases daily), it's reasonable to assume that they reviewed significantly more than the minimum number of cases.
Rueian2 votesThey initially only hand reviewed the first 1,000 post launch of the automated system, after that even randomly drawing a handful of random cases leaves out the possibility of uncontested false positives.
Not only that but like our other conversation, many of the early reform cards, and still now have false reports in the showcased games, which not only poorly represents the player's behavior if they deserve punishment and gives them a poor message of what they are doing wrong, but also judges them unfairly along side those who's case are entirely 'true' reports.
The standard is vague, the statistic is vague, Riot likes to be vague about everything.
Given that at the time of release ~67 million accounts were registered, assume only 1% of players are punished, that's still 670,000 cases. lets be generous and say they hand reviewed 60,000 (10x their provided ratio) cases... that is still less then 10% of the cases, there is a lot of possibility for variance.
[Forgive the prior math error, been working on calc3 homework for about 7 hours and its almost 1AM, brain is a bit mush]
Jubbinaut1 votesThey initially only hand reviewed the first 1,000 post launch of the automated system, after that even randomly drawing a handful of random cases leaves out the possibility of uncontested false positives.
If they only reviewed the first 1,000 cases, how could they possibly have concluded an error rate of 1 in 6,000? That's extremely arbitrary - it's impossible to see a fraction of an error due to the binary nature of the thing, so they'd either see a 0% error rate, or a 0.001% (1 in 1,000) error rate.
Not only that but like our other conversation, many of the early reform cards, and still now have false reports in the showcased games, which not only poorly represents the player's behavior if they deserve punishment and gives them a poor message of what they are doing wrong, but also judges them unfairly along side those who's case are entirely 'true' reports.
That's absolutely true. It's one of my biggest points of concern with the current Reform Cards. I strongly believe that the Reform Cards should focus on showing the games that are most strongly indicative of the behavior that led to the player's punishment, rather than simply the most recent reported games.
The standard is vague, the statistic is vague, Riot likes to be vague about everything.
It's... pretty straight-forward. Errors will occur in approximately one case out of every 6000. Even if it's off by a lot, that's still an absurdly low error rate.
Given that at the time of release ~67 million accounts were registered, assume only 1% of players are punished, thats still 6.7million cases. lets be generous and say they hand reviewed 600,000 cases... that is still less then 10% of the cases, there is a lot of possibility for variance.
Studies are done using sample sizes of a few hundred, and are expected to be indicative of of the ~319 million residents of the United States.
I think they're pretty ahead of the curve, there.
Rueian2 votesThis is just a problem i have with Statistics in general, due to the fact that it can be somewhat manipulated and the margin of error can be pretty big because of manipulation. I mean say they reviewed 12,000 (random number to work with the 1:6000 ratio) and found 2 false positives, but if they reviewed the next 6000 and found 5+ more false positives. The ratio could be MUCH different. Plus I feel like the focus on just the 0 bad outlier really skews the data as with out other conversations there are people who get punished for partially false cases.
Jubbinaut2 votesThis is just a problem i have with Statistics in general, due to the fact that it can be somewhat manipulated and the margin of error can be pretty big because of manipulation. I mean say they reviewed 12,000 (random number to work with the 1:6000 ratio) and found 2 false positives, but if they reviewed the next 6000 and found 5+ more false positives. The ratio could be MUCH different.
That's certainly true, but it's highly unlikely. Not because of anything they're doing, but because of math.
Plus I feel like the focus on just the 0 bad outlier really skews the data as with out other conversations there are people who get punished for partially false cases.
If you want to discuss that, let's discuss it in the thread it started in.
For the purposes of this discussion, let's accept that (at least according to Riot), there's no such thing as a "partially false" case - because the overall behavior is reviewed - not just the only-negative games.
EDIT: I decided that I wanted to look at the difference between the two samples you put forth in your first paragraph. Specifically:
I mean say they reviewed 12,000 (random number to work with the 1:6000 ratio) and found 2 false positives, but if they reviewed the next 6000 and found 5+ more false positives.
I'm doing this math on an actual spreadsheet, rather than my own head. As I'll show, I screwed up my earlier math, so I'll go back and edit that post to correct the screw-up.
In the first case, we see 2 errors out of 12,000 cases - an error rate of 0.01667%. Which means an accuracy of 99.98333%
In the second chunk, we see 5 errors out of 6,000 cases - an error rate of 0.08333%. This gives an accuracy of 99.91667%
Combining the two, we see 7 errors out of 18,000 cases - an error rate of 0.03889%. This gives an accuracy of 99.96111%.
I'm not drawing any conclusions from this, because it came from a hypothetical. I was just curious about what kind of impact that might actually have, statistically.
Rueian1 votesI trust Riot to do their work correctly and do solely blame the 'possible variance' in any margin of error.
Given that the false positive rate was based on an absolutely clean case, i can accept the 1:6000 ratio.
Lucifer2 votesNowhere in his post did he say that. You're simply making stuff up now.
Jubbinaut2 votesNowhere in his post did he say that. You're simply making stuff up now.
Apologies. I didn't realize I needed to dig up every comment he made on the subject. How about this one:
Well, it's not going to be a satisfactory response until it's in a paper somewhere, but we basically ran several weeks (months even) of tests where Player Behavior and Player Support teams flew into LA and hand-reviewed thousands and thousands of cases.
Team members had to review the same case independently, and then we calculated everyone's consistency and reliability. During these tests, the system was recording live data and making decisions about games without actually handing out penalties. We'd then hand review the data, with multiple team members reviewing each case.
From this, we saw approximately a 1 in 6000 case globally where we saw a game or chat log that had 0 wrong-doing and did not deserve a penalty.
I highlighted what I believe is a pretty key statement in there - these cases weren't issuing punishments while they were doing these tests. Meaning they had to be undertaken explicitly at Riot's direction - because nobody was going to be contesting bans they didn't receive.
Rueian1 votesJumping in on that quote... i find
game or chat log that had 0 wrong-doing and did not deserve a penalty.
Disturbing. Because that means their justification of 'false positive' was super strict
Jubbinaut1 votesDisturbing. Because that means their justification of 'false positive' was super strict
That's always been the definition of false positive - specifically, it's a case that is issued a punishment when no punishment was deserved/warranted.
Rueian2 votesWhat of 50% cases, what of 75% cases.
Say a case consists of 10 games is a 1 game out of 10 reason enough to warrant punishment? That is the point I am making.
Jubbinaut2 votesWhat of 50% cases, what of 75% cases.
Say a case consists of 10 games is a 1 game out of 10 reason enough to warrant punishment?
That strongly depends on the specific behavior demonstrated.
Player calls early "gg," or "ggez" in 1/10 games? Probably doesn't warrant a punishment.
Player drops racial, sexist, homophobic slurs at his teammates in 1/10 games? Probably does warrant a punishment.
Again, let's try to keep this discussion where it belongs, rather than hijacking this thread. That'd be pretty rude.
Rueian3 votesPart of me feels like the definition of what is a valid or invalid punishment case is super important to define, and the lack of definition leads to a lot of confusion for players, being punished which is why i bring it up on the discussion of what a 'false positive' is. but like in the other quote branch, we kind of came to an agreeable conclusion... which is a first for us haha.
Jubbinaut2 votesPart of me feels like the definition of what is a valid or invalid punishment case is super important to define, and the lack of definition leads to a lot of confusion for players, being punished which is why i bring it up on the discussion of what a 'false positive' is.
On the one hand? I agree with you - having a solid understanding of what is and isn't acceptable is pretty important.
On the other hand, though, I think this is one of those cases where it may be overall better for it to remain under-defined. There's outlier cases (like yours) where stricter definitions would serve well, but there's also cases where strict definitions allow for manipulable loopholes.
In the long run, I'm not sure which is the better option, but I trust that Riot is at least trying to make the best decision they can given the data they have.
but like in the other quote branch, we kind of came to an agreeable conclusion... which is a first for us haha.
My favorite kind of conclusion!
Rueian2 votesThose who abuse the system make it harder for those who don't. I'm always on the business end of the system due to personal issues, that is why I even bother to try to understand it.
I feel like because they tread on fairly new territory for games, and behavioral management in general. They take risky steps in one direction, but are almost overly cautious everywhere else. I mean the every behavioral system since the tribunal was a huge risk.
I'm just more surprised we actually managed to make a conclusion, rather than just ignore each other and move on.
Anyway good night, i have a test on double integrals in 9 hours.
Lucifer2 votesOr if they are told they are wrong. E.g., I got a 10 game chat restriction for saying ":/" in games, which was apparently too negative. IMO that is an example as a false negative, but it would not be counted as such under Lyte and co. That was after I had spoken to like 6 different agents before finally getting someone who was willing to actually give me my chat logs. I call complete BS on those numbers. "We're never wrong because we're never wrong!"
Rueian2 votesBeing a person consistently on the receiving end of punishment. It is grossly unlikely that that is the only reason you got restricted, but if you are absolutely sure, make a support ticket and speak with support, ask for chat logs of your negative behavior.
Like i'm arguing... People have to contest what they believe is wrong. Require proof, understand what you are doing wrong and try to change it (if you find yourself in the wrong)
Lucifer2 votesDid you read my post? I /did/ speak with support, which is the reason they gave me. It took me going back and forth multiple times before I was even told that. Again, I wasn't even given logs before I asked time and time again. This was months ago, and I have not been chat restricted since then. I don't know if policy has changed since then, but what I am saying is that the "we're only wrong 1 in 5000 times" thing is complete crap.
Rueian1 votestry tweeting @RiotSupport next time They have been super helpful to people who ask and follow up on specific support ticket numbers if those who are working directly behind the ticket aren't being completely helpful.
Lucifer2 votesI should not have to go to twitter to get a specific issue dealt with. And it doesn't bode well for their argument that their automatic system makes no errors.
Jubbinaut1 votesOr if they are told they are wrong. E.g., I got a 10 game chat restriction for saying ":/" in games, which was apparently too negative. IMO that is an example as a false negative, but it would not be counted as such under Lyte and co. That was after I had spoken to like 6 different agents before finally getting someone who was willing to actually give me my chat logs. I call complete BS on those numbers. "We're never wrong because we're never wrong!"
As I said:
If you have a correction to my math, feel free to post it. If you have concerns with Lyte's numbers, perhaps take that up with someone who might know something about it. I'm just sharing relatively-inaccessible information. Nothing more, nothing less.
I'm not here to discuss whether or not Riot's statements about their system are trustworthy, because it's a pointless discussion.
If they're trustworthy, then it's a waste of time, because there's no external data to support them.
If they're not trustworthy, then it's a waste of time, because there's no valid external data to contradict them.
So, if that's the discussion you want to have, feel free to have it with someone else. I've no interest in wasting my time on the matter.
MilfHuntUnitXJ93 votesYes you can, btw jubbinaut is full of crap. He just likes hearing himself talk.
Anyways false chat restrictions happen quite often. Even less is the amount of people who seek help for it, so there are quite a few cases.
Lajneen2 votesI once got chat restricted because my friends reported me after every game for the hell of it. So yes, you can get chat restricted unjustly, anyone that tells you otherwise is full of shit and have no clue what they're talking about.
Çhârm1 votesMy account was hacked by someone he had changed my email on the account. After all this happened i got some RP i never spent it so then the person who had hacked my account he went on my account and sent his own account a skin with my RP. After this happened i sent a support ticket to riot the next day after i sent the ticket my account has been permanently suspended. Since the person changed my email i have no idea my account got suspended if i could get my account back that would be great because i did nothing in order for my account to be suspended all i did was send a support ticket. The persons account name in game is LiividHaZe this is the person who went into my account and sent himself a skin why is he not banned. But my account gets suspended permanently why riot if you could tell me why or just give me my account back that would be great please at least tell me why you have suspended my account