Boards›Esports

Explaining the Echo Fox vs. Clutch Gaming Ruling

Riot·6/21/2018, 11:43:49 PM·0 votes·11,936 views

Chris “Riot Commish” Greeley here. I’m the Commissioner of the NA LCS. This past weekend we used a rule, the Awarded Game Victory rule, for the first time. In this Quick Notes, I want to talk about how the decision was made and continue to provide greater transparency about how the NA LCS functions behind the scenes.

The Circumstances

In the Week 1 match between Clutch Gaming and Echo Fox on Sunday, June 17, the game server hosting the match crashed at 32:26 into the game. If you were watching the stream, the game simply froze on screen, and the players weren’t able to take any further action.

Our Live Operations team repeatedly attempted to Chronobreak the game during the stoppage of play, but due to the  server crash, the Chronobreak tool was not able to restore the game. After approximately 20-25 minutes of work to restore the game to its current state, we acknowledged that Chronobreak was not going to solve the issue.

Game Breaking Bugs & Awarding Game Victories

The advent of Chronobreak has allowed us to avoid a number of situations in which a game impacting bug may otherwise have required a complete remake. Despite that safety net, we still know that there are going to be rare situations where Chronobreak is not available (for example, in the event of a game server crash or where a game breaking bug is discovered that simply going back in time will not correct or avoid).

In these cases where a critical bug is not the fault of either team (i.e., not a situation where a player knows of a bug that they, in our opinion, intentionally trigger to cause some kind of issue), and Chronobreak is not available, the inclination should always be in favor of a remake, except in situations where it would be more fair to award a winner based on the current game state than playing an entirely new game. In those cases, punishing the team that has the game in hand by remaking has to be carefully considered against depriving the other team of a chance to continue playing through an awarded game victory.

During the biannual review of the rules, we spent a lot of time with our statistics team pouring over game data looking for some set of criteria that were objectively indicative of victory. What we learned is that they don’t exist. Short of one team attacking the open Nexus with the enemy team dead, it is exceedingly difficult to articulate objective standards around when victory should be awarded in the case of some kind of terminal failure. Even the guidepost examples provided in the rulebook are sensitive to changes in the game, objectives, gold and experience values, and the meta.

Ultimately, the awarded game victory rule vests discretion in the Head Referee monitoring the game and entrusts them to determine whether, in extreme situations, an awarded game victory or remake are the most appropriate outcomes.

The Decision Making Process

Our first priority in any situation where there is a bug that affects gameplay is to quickly ascertain what happened, whether we need to take some action (rather than just unpause and continue), and what options we have if some action is required (both practically and philosophically).

In this case, when the Head Referee’s view of the game froze, he asked the stage referees what they saw on the pro screens on stage. He quickly learned that all ten players had frozen as well, and each was showing a window that the game was attempting to reconnect. The broadcast team and the observers were asking about a pause, but the referee team concluded that no one had actually paused the game. This led our Live Operations team to believe that there was an issue with the game server.

The Live Operations team engaged the Chronobreak tool to try to recreate the game and get the players back into game as fast as possible. At the same time, the Head Referee (who was able to rewind the game through the observation tool) began to look for an appropriate point to Chronobreak the game to. However, after several minutes of working with the Chronobreak tool, the Live Operations team reported that they were unable to Chronobreak the game through standard methods.

The Live Operations team continued to work on the Chronobreak tool in an attempt to restore the game. In parallel, the Head Referee began planning for a situation in which he would have to choose between issuing an instruction to remake the game from scratch (including a new champion select phase) or awarding the game victory to Echo Fox. As part of our usual backstage procedures, the Head Referee pulled up the rulebook and reviewed the relevant portions to make sure he had all of the information at hand necessary to make a decision.

The awarded game victory rule turns on a determination that the team losing the game “cannot avoid defeat to a degree of reasonable certainty.” Put simply – what is the likelihood that the losing team can come back and win?

At the time of the crash, Echo Fox was up 7 turrets to 0. There was 1 inhibitor down on Clutch Gaming’s side, a 4 dragon lead and a 10k gold lead for Echo Fox. After reviewing the current state of the game (including towers, objectives, gold differential and objective timers), the Head Referee determined that in the event that the game could not be recovered through Chronobreak, that the victory would be awarded to Echo Fox because it was sufficiently highly unlikely, in his opinion, that Clutch Gaming could avoid defeat in the game. That doesn’t mean it was impossible – there are always situations in League where a team that is down by a large margin mounts a comeback, but that in this situation, it was highly unlikely.

After the Head Referee made his decision, we gathered the management from Echo Fox and Clutch Gaming backstage and informed them of the decision. Subsequently, we asked the players to take their gear, leave the stage, informed them of the decision backstage, and announced the decision on broadcast. At roughly the same time, we learned from our gameplay team that the likely cause of the crash was a bug which can manifest when a player selects Taliyah with smite in a game and disabled Taliyah for the remaining game of the day.

The Aftermath

We have had several conversations with Clutch Gaming organization at all levels explaining our decision and rationale. Separately, we have reached out to all the NA LCS teams asking for feedback on the rule, its application in this game, and application to situations in the future. This is the first time we have used the awarded game victory rule, and we want to ensure that in the future the League and teams are aligned on the use of the rule in the rare circumstances where it may be needed. 

While we are confident in the decision based on the information we had on hand at the time it was made, we have subsequently taken a look through patch 8.11 statistics from the NA LCS and other leagues to get some sense of whether or not our conclusion on-the-spot makes sense. Across 131 games in NA LCS, EU LCS, NA Academy, LCK, CBLoL, CLS, OPL, and TCL:

  • No team has lost when they had a gold lead as large as Echo Fox’ at thirty minutes;
  • No team has lost when they had a 7 tower advantage;
  • In 74 games where a team has had an advantage of at least 10,079 gold (Echo Fox's exact lead at the time of the crash), the team with the gold advantage has won every single time.

Obviously, none of these statistics are dispositive and comebacks can happen, but the point is that such comebacks are few and far between, and when weighing between two unpalatable options (complete remake and awarded game victory), the Head Referee was and remains reasonably certain that Echo Fox would have been able to close this game out had the crash not rendered the game unplayable.

We have confirmed that the Clutch Gaming and Echo Fox game was derailed by a bug that can be triggered when Taliyah (or Karthus) spams abilities in fog of war. The bug drives CPU usage towards 100%, at which point the game server terminates the game. Due to the nature of this particular bug, even if we were able to Chronobreak the game, it is highly likely that the game would have continued to trigger this bug, rendering it unplayable. This bug was diagnosed earlier this week and a micropatch was applied to prevent it from occurring in the future.

 

Quick Notes is a NA esports series similar to Meddler’s where we’ll deep dive into topics that the community wants to know more about, discuss work in progress, and share our decision-making process.

Chris “Riot Commish” Greeley is the Commissioner of the NA LCS and ran League Operations for MSI and Worlds in 2017. He is a recovering lawyer from New York City who can’t write a rulebook in less than 50 pages. He channeled years of raiding on his Resto Druid into life in the bottom lane dropping wards and shooting glitterlances. You can find him on Instagram, Twitter and Reddit as @IAmGrza.

27 Comments

Sulfuric6/22/2018, 2:50:19 AM8 votes

Thanks for laying out the decision with some data to back it up. Tough decisions rarely feel correct but they have to be made one way or the other.

Ahri Carries All6/22/2018, 1:36:26 PM5 votes

I saw a high elo YouTuber attempting to recreate the bug that crashed the game server and over numerous tries, he only got it to happen once. This is the first time I've seen a bug like this and I think with it being hard to trigger, it isn't on Riot for not seeing this coming. No one could. As for the ruling on the game and the existence of the rule which only goes into effect if the game cannot be Chronobreak'd... It's remarkable that this rule Riot put in place years ago hasn't seen use until this match and I think that when you look at how the game was panning out, CG really didn't have much of a chance. I'm not saying FOX was definitely going to win, but in a majority of games, with this kind of lead, the team in the lead is slated to win 99% of the time. Good ruling in my opinion but a shame that it came to this.

kill3rlance6/22/2018, 6:12:28 PM5 votes

FORGET RIOT, THIS COMPANY SUCKS. There's always a small chance, there has to be some sort of alternative or mending of the situation.

"In 74 games where a team has had an advantage of at least 10,079 gold (Echo Fox's exact lead at the time of the crash), the team with the gold advantage has won every single time."

Oh.

ShadowZPhoenix6/22/2018, 8:08:52 AM3 votes

I remember reading a post a couple of years ago about LCS rulings, and none of which had been applied because they allow games to play out 99.99% of the time. These Rulings include when it is a referees decision to end the match if one of these conditions are met, meaning that a team has no chance to recover. a 10k gold lead, 5 tower or double inhibitor advantage when 1-2 towers are taken on the other team are among those that fulfill the ruling where a Ref can award a win to a team.

These are RULES, stating that when a team has enough of a gold lead Refs say game is over, you win. The other team has a forced surrender. Has that happened once in the history of league? I don't think so, at least not in LCS competitive play, yet there is that rule. Cut Riot a little slack, yea it is a terrible thing to have happen, but things like this do happen

ChompyWulf6/22/2018, 5:48:22 AM3 votes

I'm just here to laugh because with all the massive profits, with game breaking bugs forcing restarts in the past, and a program supposedly designed to recover games that crashed, it happened. After 7 seasons and new Esports games/genres on the rise, League looked bad. Really bad. It's embarrassing and Riot should not only be explaining the decision, but also issuing an apology to both teams for an unsatisfactory ending, offer some small reimbursement to fans who bought tickets to watch live, and given the clear problem this presents in Best of 1 play, drop it and go back to Best of 3. At least in a series there is far less chance of a single bug deciding which team has the W at the end. Maybe revise your rules so in an awarded game scenario it instantly puts the teams into a Best of 3.

Ukhozi6/30/2018, 2:24:32 AM1 votes

I would like to point out something unrelated to most of the discussion (that part about working around bugs and the format of the LCS rules) but about what to do if there must be a decision to remake or use AGV.

Suppose hypothetically, that Riot did have some way to precisely determine the odds of a team making a comeback. What should be the thresh-hold for deciding to remake or AGV? It should be the odds that team had of winning in the first place (i.e. from start of draft phase). e.g. If Team A has a 20% chance to beat Team B, but their chances have fallen to 10% when the problem occurs, AGV should be used because the margin is in favor of Team A losing.

In a situation where you cannot use Chronobreak to be fair to the actual probability, use whatever option brings you closest.

RHIT6/22/2018, 2:22:20 AM1 votes

I understand the game server crashed, but can you explain (here or in a technical blog) why the game data isn't running on one server and being copied to another during the match? This way if the server the game is being played on spontaneously combusts (or goes does to something more reasonable like harddrive malfunctioning), you could switch to the 2nd server and chronobreak to a reasonable time to resume play. I know in this case there was a high likelyhood of the problem reoccuring so it wouldn't have helped, but there is the chance of this being needed in the future.

AbysmalDrunk6/22/2018, 2:47:26 AM1 votes

You still broke your own rules. Read your rule book again, no conditions were met. Rito fails again.

SquigglyLine6/26/2018, 4:21:30 AM1 votes

No team has lost when they had a gold lead as large as Echo Fox’ at thirty minutes;

Cloud9: Hold my beer

.#NotMyC9

The Bad Touch6/21/2018, 11:55:26 PM1 votes

No team has lost when they had a gold lead as large as Echo Fox’ at thirty minutes; No team has lost when they had a 7 tower advantage; In 74 games where a team has had an advantage of at least 10,079 gold (Echo Fox's exact lead at the time of the crash), the team with the gold advantage has won every single time.

So can we just have games fucking autoend at 30 minutes if one team is 10k ahead?

Idiots that wont FF get old.