IFS ramblings

Awf Meta··1 votes·338 views

Riot says the IFS learns from our reports. Well, let's look at the reports. Players pick a subject and are given a box for additional details.

Situation A: Player doesn't fill out additional details. How in the world is the IFS supposed to know what the player found offensive? The IFS is given the entire chat logs and told something in there is supposedly offensive.

Is there a valid reason for not providing additional details? I say yes. Players are encouraged to mute offensive chat. If chat is muted, the player won't know what is being said. Player can't provide additional details because they never saw the details. They muted at the first sign of negativity, therefore, preventing anymore negativity from showing. At best, they would be able to provide a single example.

Situation B: Player fills out additional details. Player has to be exposed to offensive chat so that they can accurately report it. Furthermore, player has to remember the offensive chat because they can only report at end game.

Situation C: Player fills out additional details but is insincere. IFS has to some how determine whether the reported chat was actually offensive. I suspect the IFS does not. Instead, it logs that chat. If that chat becomes reported past a threshold, IFS can then consider the reports valid.

When we consider all the situations together, IFS operates on keywords and phrases. There is nothing there to suggest IFS can determine intentions. That is why "kys" is punished but other forms of the phrase are not punished.

We actually have some evidence that IFS struggles to determine intentions. The intentional feeder detection. We hear it all the time, intent is hard to prove. So only the most obvious of intentional feeders get punished. Well, the same thing applies to IFS. Only the most obvious words and phrases are punished. Over time, the list of obvious negative chat grows. It's not the IFS punishing intentions.

11 Comments

ModThe Djinn5 votes

{quoted}Situation A: Player doesn't fill out additional details. How in the world is the IFS supposed to know what the player found offensive? The IFS is given the entire chat logs and told something in there is supposedly offensive.

Yep. But remember this is big data: if the system gets 100,000 reports, all without context, it can look for similar patterns across them vs. patterns found in games without reports, and draw pretty strong conclusions on what the community at large has issues with.

We actually have some evidence that IFS struggles to determine intentions. The intentional feeder detection. We hear it all the time, intent is hard to prove. So only the most obvious of intentional feeders get punished. Well, the same thing applies to IFS. Only the most obvious words and phrases are punished. Over time, the list of obvious negative chat grows. It's not the IFS punishing intentions.

We've seen quite a few times that it is able to pick up on words used in context, such as complaining about a champion vs. a person. It can't know a human's intention, no, but intentions are often conveyed in language at least somewhat, and the system can figure that out.

Awf Meta1 votes

it can look for similar patterns across them vs. patterns found in games without reports

Do you have any proof IFS scans games without reports? And that it weights the chat logs against reported logs.

I've seen the Riot Tantram post about multiple reports not counting. It says IF report = review. That says to me, if no report = no review.

We've seen quite a few times that it is able to pick up on words used in context, such as complaining about a champion vs. a person.

Can you explain a little more?

RallerenP2 votes

Not Djinn but I want to weigh in.

Do you have any proof IFS scans games without reports? And that it weights the chat logs against reported logs.

I would imagine that it has to scan games without reports to learn at all. If it just finds patterns in games with reports, it would find patterns that happen in all games. It would have to compare the two types of games and ignore patterns that are too similar or it would slowly just begin to classify everything as toxic.

For alot of machine learning algorithms where you have to sort patterns of behaviour you need atleast two datasets or it wont be able to find patterns in either. I can't imagine that this system is any different.

I've seen the Riot Tantram post about multiple reports not counting. It says IF report = review. That says to me, if no report = no review.

The part of the system that 'learns' would be seperated from the rest. So no matter if there was a report or not, the game would be sent to the machine learning part. The IFS has multiple parts and Tantram only detail one of them with that information.

ModThe Djinn2 votes

{quoted}Do you have any proof IFS scans games without reports? And that it weights the chat logs against reported logs.

I've seen the Riot Tantram post about multiple reports not counting. It says IF report = review. That says to me, if no report = no review.

I don't have exact proof, no. But you DO need to feed data into the program so it can learn. It would make sense to feed all games into the system, but only trigger the behavioral review portion by specific trigger.

i.e. the way I'd probably have designed the system is that all games are filtered in, but "If reported='YES' then [run behavioral review]."

Can you explain a little more?

Sure. I've never seen a chat log where someone was punished for things like "Annie is such a fucking stupid champion" or "Alistar's combo is so fucking dumb" that didn't also include other toxic behavior.

Sure, if you're using words like "gay" or "retarded" or "whore" etc. to describe non-player entities, that's a problem for other reasons. But the system seems to know the difference between "Annie is such a trash champion" and "This Annie is trash."

Awf Meta1 votes

Well, proof would be nice. However, allow me to give an example to see if I understand.

2 games. 1 game has a report. Game A: everything gets +1. Game B: everything gets -1. The "negative" chat is in game B but not in game A. Therefore, the "negative" chat has a score of -1. All the other chat has a score of 0.

But the system seems to know the difference between "Annie is such a trash champion" and "This Annie is trash."

Those aren't very similar in my opinion. How would the system treat, "Annie is trash"? Seems the player would have to add context or be in danger of punishment.

Also, do the chat logs show all the chat? Thus, why people come to the boards to find out which part of the chat was the problem.

ModThe Djinn2 votes

{quoted}

Well, proof would be nice. However, allow me to give an example to see if I understand.

2 games. 1 game has a report. Game A: everything gets +1. Game B: everything gets -1. The "negative" chat is in game B but not in game A. Therefore, the "negative" chat has a score of -1. All the other chat has a score of 0.

Possibly? Again, not sure of exactly how it works -- I suspect it's a little more complex than that, but that is the basic idea.

Those aren't very similar in my opinion. How would the system treat, "Annie is trash"? Seems the player would have to add context or be in danger of punishment.

I don't exactly know, but my guess is that it might lean towards punishment unless it sees a conversation about the champion.

Also, do the chat logs show all the chat? Thus, why people come to the boards to find out which part of the chat was the problem.

They do, yes. I can't speak as to why every line the reported player said is included, but my guess is simply for completeness.

Umbral Regent1 votes

They do, yes. I can't speak as to why every line the reported player said is included, but my guess is simply for completeness.

To add to this, I personally feel that each player should generally be able to see the negativity or rule breaking in their chat standing in contrast to the rest of their chat. It's easy enough to ignore it in the moment when they're actually saying it, but, to some degree, they should be fully capable of seeing and parsing out what went sour for them to have been punished.

That's only a personal hypothesis, though, and it leans heavily on the knowledge that all the punishments prior to a permaban are meant to push a player to reform.

91929298DEL11 votes

[deleted]

zPOOPz1 votes
Awf Meta1 votes

The thought is, IFS is learning "what gets reported" not "what is negative". Therefore, IFS could potentially flag anything. Enough people report "have a nice day" and IFS would start punishing that phrase. Furthermore, the system would be perpetually behind the trends. As it requires a report threshold. Lastly, how would the system handle subtle differences? For example, "kys" vs "ky$" vs "kill your self".

ModThe Djinn2 votes

{quoted}Enough people report "have a nice day" and IFS would start punishing that phrase.

That's the intent. The goal is that the system learns from what the community wants to report. If we all suddenly decide that "GG" is punishable, it quickly would become so. This sort of happened to "kys", actually. Luckily, you can't really organize a community this large to all start doing that to something just on a whim, so it's not really a huge issue.

Furthermore, the system would be perpetually behind the trends. As it requires a report threshold. Lastly, how would the system handle subtle differences? For example, "kys" vs "ky$" vs "kill your self".

I imagine it can have specific things put in manually. I also imagine it's good on a lot of common number/character replacements. Those are both pretty easy additions to a system.

More from Player Behavior

More by Awf Meta