General Philosophy on Balance Hotfixing

Hey folks!

As you’ve likely seen, we pushed out a second balance hotfix for Janna at the end of last week, and saw that it caused some confusion with players (rightfully so) and I wanted to jump in and provide some context! Earlier this year we got some fancy dancy new tech (“micropatching”) which allows us to create a change (bug fix, balance change etc), bundle them up, and ship them within the very short time frame of an hour or two depending on the changes in question. This was a huge improvement over how that process used to work, but we never really talked about that, so I’ll take a second to run through that as well.

In the past, changing the live environment for something like a balance hotfix was quite time intensive. We'd first make the change, which then required a sweep of testing from QA for several hours to a day, and then members of our release team would take up to a day to ship it out to all of our severs around the world. It was a super unwieldy process and hampered our ability to react to things as quickly as we would like. It also incurred more risk as shipping a change meant we needed to be 100% confident it wouldn’t break anything, since we couldn’t just “roll it back” without another run-through of the process mentioned above. This also meant that we couldn't do these on Friday, since breaking a champion/item on the weekend meant we'd have to leave them disabled until Monday, which was something we ALWAYS strive to avoid. Micropatching lets us bypass all that and get changes to you in a fraction of the time, with a fraction of the risk.

One thing we have to remember though is:

With great power comes great responsibility (man this meme is weaker when image embedding doesn't work)

Balance hotfixing is incredibly powerful, but it’s also dangerous. Hotfix too much and it looks like we don’t have our shit together, hotfix too little (when it’s necessary), and players may have to deal with a shitty situation longer than they’re happy with. When it comes to newly released content, be it a new champion, new item, or an update, our stance is that we’re going to be more than happy to pull the hotfix trigger if we feel the situation necessitates it. When their power ends up being stronger/weaker than we’re happy with, we believe it’s overall better to quickly hotfix out some numbers changes before everyone settles into the update, versus waiting two weeks and then making everyone do more re-learning in the next full patch. I’ll use the changes we made to Janna last week as an example since they're recent, though we’ve applied this thinking to smaller updates like what we did with Singed as well:

(Screenshot from Wednesday morning after patch 7.19’s release)

In the chart above, the green bar indicates that Janna’s WR had increased 2.7% from the patch before, up to a total of 58.8% as shown with the red bar. We’d had some inclination that she would release strong and would require follow up some suspicion on as alluded to in the patch notes, (and even talked about it a bit here, so we decided to react quickly and get our first change out in the wild before her power would continue to sway the results of the games she was in. We underestimated how quickly Janna players would adapt to the more aggro playstyle, and overestimated how quickly players would figure out how to play around Janna in the lane phase. This is borne out in her initial performance stats, which were on the high side, so we reacted.

(Screenshot from Thursday morning after our first hotfix)

The numbers above show that our hotfix from Wednesday brought Janna slightly back down to earth, with the red bar indicating that Janna’s win-rate at all MMR (56.4%), and High MMR (56.1%). The smaller numbers on the left—0.3%, and 0.0%—represent the change in her winrate from the patch before. Note that our the driver for these types of changes are almost NEVER based on WR alone, but in the early stages of a new release more nuanced things like frustration, counterplay and abuse cases are far harder to pinpoint with certainty, so WR is often the only concrete thing we have to go off of.

Getting back to Janna, we knew that Wednesday’s changes were getting back on the right track based on the data represented above. At this point, with her performance back in line with 7.18 levels (which was also certainly on the high side), we could have left followup until the 7.20 patch given that, in 7.18, that same WR wasn’t something we were considering hotfixing Janna for. That’s where the previously-mentioned efficiency of micropatching comes into play.

We had just worked on Janna, players were clearly figuring her out but weren’t “used” to her new changes, and she was still stronger than we wanted long term. Rather than wait for next patch we chose to respond a second time since we could with relative ease and it felt like the correct thing to do. Fast forward to today and Janna is sitting at around 53.8% at both High and all MMR which is getitng much closer to what we would expect from her. We’ve been testing some additional changes to her on the PBE which could end up shipping in few patches, but current thinking is that we may just wait until after Runes Reforged ships to give us the time to see where everyone lands. It's going to be a hot mess for a little bit which is honestly really exciting, but it makes any attempt to fine tune a champion in the next few patches rather temporal. That doesn't mean we won't, just that there's likely to be fewer changes

BUT ANYWAYS! I want to be clear that this isn’t a post about Janna, she was just a good current case study to illustrate our philosophy around hotfixes and why we are likely to pursue them with the same rigor in the future. We won't go out of our way to find candidates that need this attention, but if we miss the mark we'll respond quickly. We're also working on finding some better ways of communicating that they're happening/happened. Things like using the esports ticker that informs you of an ongoing match, or even a notification in champ select to name a few options. More on that to come.

I'll try to kick around in the comments to answer a few questions, but let me know what you think of this approach!