Reports: How do they work?

Since launching the instant feedback system in the spring, we’ve spent all summer and fall powering it up. We updated reports in 5.21, so let’s talk about the why's and how's of reports, and how they fit into the broader ecosystem.

So What Changed?

We made two major changes to the report system. First, we cleaned house, analyzing report data and community feedback to dispatch redundant, useless, or simply out-of-date report categories.

Second, the old system only allowed reports for one offense, but we’ve all been in games with players who show multiple unsportsmanlike behaviors in one game. The system now allows up to three categories to be selected when reporting an especially disruptive player.

Selecting multiple categories won’t increase the severity of a report because the system verifies the accuracy of every category of every report. Later on, we’ll discuss how we handle dishonest reports, but generally speaking, the more earnest and accurate the report, the better chance of appropriate action being taken.

Why Reports Matter

We occasionally hear some players wondering whether reports matter in a world with automated systems like instant feedback. Short answer: Reports matter a lot. For the longer answer, let’s zoom out to the big picture.

We view aggregate report data as the collective voice of the playerbase and the main way the community defines and establishes standards of behavior. When reporting a player, you add your voice to that chorus. For example, report data shows the vast majority of players can’t stand racism, sexism and homophobia, and that’s why we tune systems to specifically target hate speech.

Importantly, each region owns their own community standards and we customize the system for every region to account for cultural differences. For example, in most regions, calling another player “bronze” is heavily reported, but we’ve learned that in Korea, even calling players “silver” is offensive. Youch. “Your mom” jokes vary by territory, too. In English speaking territories, “Your mom” can be silly or mean, but in Korea, “Your mom” is always deeply offensive. So keep that in mind on your next swing through Seoul.

Reports are significant on a per-game basis because no one is punished on game data alone. So if there are no reports in a given game, the system assumes no one had a bad time, and there’s no reason to consider punishing a player. This means that if you play with a negative player and don’t report them, they won’t receive a punishment and likely won’t reform.

How Reports Help

When a player’s reported, the instant feedback system collects all reports from the game and compares them with in-game data (statistics, chat, item builds, metrics from past games, etc.) to make a call on a whether or not to punish. If a player ends up punished, they receive a reform card (an in-client message calling out the punishment and evidence of the behavior that caused it). Even if no one is punished, the system stores the report and game info to better inform future decisions when players are reported again.

Reports go beyond supporting instant feedback; player support specialists, designers, data scientists, researchers, and engineers use them to:

• Hand review difficult or nuanced cases (a situation where hand-written comments really help out)
• Analyze trends in report types and comments over time to inform the design of future social systems
• Build and power our automated systems like instant feedback

Sometimes a player attempts to punish another player by reporting them as a form of retaliation. In a system that relies on honesty, that’s potentially a big problem. We can detect false reports, filter them out, and decrease the effectiveness of future reports from consistently dishonest players. As we continue to build more systems that rely on honest reports, the penalties for false reporting will become more strict.

Some cases require human intuition and judgment, and for those, we’re rebuilding the Tribunal voting system. In the future, the Tribunal will lean heavily on reports (and honors) to build cases for both positive and negative behavior. Accurate report selection and comments will be critical to letting Tribunal voters know what really happened. We’re looking forward to rebooting the Tribunal after we’re through pushing new champ select to all regions and queues.

Comments? Questions? We’ll stick around for a bit to talk player behavior.