If PBE is a testing ground, isn't it about time it started being used like one

Fed by Snu Snu·Mar 19, 2015·47 votes·2,195 views

I'd love to see some actual testing finally happening there finally.

Like you know.

Experimenting how Zed would function without his %AD modifier and balancing accordingly if it proves to be healthier. Experimenting how Shen would function with %bonus/total hp on his shield and balancing accordingly if it proves to be healthy enough. Experimenting how to have Riven viable without her having a million %AD modifiers in her kit.

Stuff like that.

And you know, since you boast about being so good analysts, if something proves to not be good for the game, you can actually EXPLAIN why to us after the experiment is finished and why it didn't see live.

You know, just stuff someone would expect from a testing ground.

But no, we don't even get an explanation why a miniscule buff like the Darius one got reverted. We get no explanation why Zed gets no attention whatsoever or even an actual attempt to be looked at. We get nothing.

28 Comments

RetillinMar 19, 201513 votes

It is a testing ground. For bugs (unless it's Azir), and skins. That's it.

MrBuffingtonMar 19, 20151 votes

^ and the vacuous "feel" of certain changes. It's hard to really test balance on crazy laggy/buggy servers, even if there was a larger playerbase.

If we're talking about larger changes and experimental stuff, that's what internal playtest and the playtest team (who are mostly very high elo players) are for. PBE is like the last run-through before launch, just to iron out all the changes.

Fed by Snu SnuMar 20, 20154 votes

As an ex-high elo player, I don't see how that holds any water when watching the majority of the changes in retrospect. Most of them seem to show a lack of understanding to the game, not the other way around which you'd expect from high elo. (e.g. , hasty Yasuo buffs, hasty Gnar buffs, hasty Kalista buffs and her having an uncapped E which is only 30-ish dmg short from Twitch's poison at same stacks while also scaling with AD, etc.)

And then there's changes that are being done completely backwards. E.G. Azir, who first got buffs and then had his bugs ironed out properly. Well, obviously he would end up broken that way when you give him power on a budget that you can't even see its proper power curve yet.

And let's not even delve into the latest guttings.

That is, assuming there is no agenda being upheld, which I also find doubtful, but I'll let the benefit of doubt slide for this post.

SmokingPuffinMar 20, 20154 votes

I don't think anyone who plays on PBE would seriously advise using it as a testing ground for balance changes. Games are, quite simply, crap. Poor matchmaking. Lots of screwing around. 10x more afks than live server. Not nearly enough tryharding to actually assess the value of a kit.

Just as a concrete example of this, the PBE generally thought the Ahri rework was underpowered. The lols. This is not a tool you can use for balance feedback.

As an ex-high elo player, I don't see how that holds any water when watching the majority of the changes in retrospect. Most of them seem to show a lack of understanding to the game, not the other way around which you'd expect from high elo. (e.g. , hasty Yasuo buffs, hasty Gnar buffs, hasty Kalista buffs and her having an uncapped E which is only 30-ish dmg short from Twitch's poison at same stacks while also scaling with AD, etc.)

Their playtesting team is a bunch of high diamond players. The playtesting team gives feedback to the designers. They are mostly not the designers themselves.

I think that, by and large, Riot patch notes are quite good. Generally speaking, when I look at a balance change, mostly it's "that sounds pretty reasonable". The exception cases are...

And then there's changes that are being done completely backwards. E.G. Azir, who first got buffs and then had his bugs ironed out properly. Well, obviously he would end up broken that way when you give him power on a budget that you can't even see its proper power curve yet.

Riot tends to fix bugs without regard to how it will affect the power level of champions. There have been a couple exceptions, most notably #BetterNerfSona after they found her W power chord did nothing for years.
Riot tends to go hard on unhealthy play patterns and eliminates them without putting new healthier ones in, sometimes leaving champions in a pointless state.
Riot really hasn't figured out any process for balancing significant design changes -- releases of new champions and new reworks (e.g. Bard and Ahri, respectively) often come out way, way wrong.

On the second point, Riot's 5.5 patch rundown show issued a pretty clear "my bad". I'd recommend watching it. Sounds like they're doing some serious soul searching as a design team after the complete disaster that was 5.4 patch.

MrBuffingtonMar 21, 20151 votes

Their playtesting team is a bunch of high diamond players. The playtesting team gives feedback to the designers. They are mostly not the designers themselves.

I also wanted to add it's uncertain how much playtime the playtest team actually has on certain changes; from what I've read from playtesters themselves, they don't seem to have more than a handful of games per day (interspersed with feedback sessions and like, lunch). Given the two-week patch cycles and the fact that many changes get locked in after like, 1-1.5 weeks, plus that a particular champion can only be played once/twice per game (unless they're doing like all-for-one-mirror-mode playtesting or something), I actually find it kind of surprising that there aren't more of these extreme outlier cases, especially given how drastically designers can (and often do) change the game. Remember when CertainlyT made Alistar's Headbutt a pass-through skillshot? Sure it was never meant to go through, but that's the kind of stuff that they do over lunch; imagine what kind of crazy stuff designers get to when they're actually working on the clock. And yes, designers actively seek to change the game up in drastic ways to keep things interesting, so this sort of stuff (and crazier) is probably pretty commonplace behind the scene.

Granted, many larger changes (new champs, larger scale reworks, etc) probably have a longer than one patch cycle of testing, but again, most of the changes are actually pretty decent; the outliers are just that: outliers.

TehNACHOMar 19, 20159 votes

[{quoted}](name=Fed by Snu Snu,realm=EUW,application-id=3ErqAdtq,discussion-id=AUG4YyTN,comment-id=,timestamp=2015-03-19T13:34:02.520+0000)

But no, we don't even get an explanation why a miniscule buff like the Darius one got reverted.

No, that's just you being uninformed.

Fed by Snu SnuMar 19, 20157 votes

{quoted}

No, that's just you being uninformed.

Well, thank you for giving me a stepping point on further expanding my initial claim.

First of all, as you can see, the explanation there was only given AFTER it was requested by the community and got upvoted to a huge point. That's not professional work, that's just a "we have to do something about it" reaction.

Normally, assuming any sort of professionalism exists, there should be at least a sticky thread that lists the changes and has the overall thoughts on it. And that's if they were to do it sloppily and lazily.

KaisadiIIaMar 19, 20155 votes

There are a lot of things we can criticize from RIOT, but they often explain changes on the PBE sooner or later.

GloriousKaiserMar 20, 20151 votes

So with all that said, my only real gripe with PBE is that Riot only explains themselves later in AMAs or when the community gets enough backing to warrant a response. That's fine in itself, but if Riot wants to step up their game, keep players satisfied, etc, then PBE context explanation should be a thing. Not responses to questions.

The thing OP probably thinks Riot should be doing is predicting the questions people will ask. In the end, only Riot knows what Riot's doing in the PBE and that doesn't sit well with people.

The temporary removal of the Darius changes were taken the wrong way. People thought Riot really didn't want to buff Darius/Gave up, so then Meddler has to come meddle and damage control before Riot looks bad... again.

Fed by Snu SnuMar 20, 20152 votes

Thank you for the quite accurate clarification. And yes, that's one of the things I'm seeking. Riot should be more professional about explaining their changes to the public. We shouldn't have to get a thread up to 70+ upvotes in order to get a clarification on a simple change.

aperson1Mar 20, 20159 votes

PBE only exists to hype new content and squash early bugs. Nothing more.

Don't think a Rioter has ever denied it.

DisasterSolaMar 20, 20152 votes

{quoted}

PBE only exists to hype new content and squash early bugs. Nothing more.

Don't think a Rioter has ever denied it.

Yeah. IMO if riot actually wanted to use it as a testing grounds, they should have something like a monthly weekend of PBE events. Like "play these champions with wild and crazy changes!". as it is, its just "get a head start on new content".

I say as a PBE player 99% of the time. xD

flibitydooMar 20, 20153 votes

If you change all the variables in an experiment, then you aren't going to learn anything.

Sightless66Mar 20, 20153 votes

It's a beta testing server. By the time something the PBE, it should already have gone through numerous stages of testing beforehand. Just throwing things in isn't the best way to use a beta testing server. It is used for finding major balance concerns and for bugfixing, not for testing overall design directions. That is a perfectly fine use for it. It doesn't have to be testing for anyone's random balance complaints, and it shouldn't be.

But no, we don't even get an explanation why a miniscule buff like the Darius one got reverted.

Yes we did. It was removed because they wanted to make a change to it, and they didn't have enough time to make it before the patch deadline (they wanted to make it a mana refund instead of a 0 mana cost so he couldn't ult if he oomed himself). It will be back next PBE cycle. Everyone just assumed it was gone forever. I actually think this illustrates a problem with your proposal. Absolutely any change they threw on there and then removed would need to come with a full discussion of why the change was a bad idea, and people will be pissed even after the explanations come through. If you're just doing basic game health testing, then having to justify yourself to the community for anything you want to try out is a terrible situation to be in. It's much better to be able to just test things and see if they work or not without getting everyone hyped for something that might not come out.

I'm not saying Riot is doing well. I haven't liked the balance much this season, and I think they're showing some pretty significant favoritism. I think Riot needs to communicate more that they are taking looks at certain champions (and even then, they have given opinions on things Lee and Zed a couple times in the past, so it just becomes a question of how often they need to communicate the same things again). More communication would be great, but getting the community involved in experimental testing is way, way too risky to be worth it. Internal testing is much better for that stuff. Showing the community highly experimental stuff would backfire, and people would be angrier than they are now. I have no doubt of that.

Besides, it wouldn't even give them enough useful information to be worth the increased workload. Experimental testing for things like game health is something they can do quicker by themselves on private servers. The PBE would just add a lot of extra work without giving a major increase in useful information. The only advantage would be a potential increase in community communication, but that can be done independently of changing the PBE's purpose.

Fed by Snu SnuMar 20, 20151 votes

Well, there's some holes in your reply.

First of all, the majority of bugs go through PBE and are fixed on a later date. Some of them we have for years to come. So this doesn't seem to apply for all bugs if PBE is supposed to be a testing ground for them. Some bugs are also plain ignored because they aren't on a popular champion, so why bother? (like for example the Aatrox bug I posted 2 patches ago, where he doesn't lower the scruttle's armor/MR with his knock up as intended with all hard-cc or, you know, the entirety of Quinn's kit, which surprisingly had one of her endless bugs fixed after a thread about her reached like 250 upvotes. She would be obviously be left to rot in her bugs if not for the boards exposure.)

And to expand, yes, you do have to explain yourself for every change, because this way you give the community a better understanding of your vision and the route you are taking to materialize that. Otherwise the community will either:

a) Think you have no idea what you are doing.

b) Think you aren't doing anything at all.

I don't bite. That's the reason why the threads for the reworks were made in the first place. (let's ignore for a second that most of the feedback wasn't even taken into account.)

In the end of the day, I'll just focus on this:

It's much better to be able to just test things and see if they work or not

I don't see much of that happening to begin with. The pace at which changes are even tried is extremely slow.

Sightless66Mar 20, 20151 votes

First of all, the majority of bugs go through PBE and are fixed on a later date. Some of them we have for years to come.

Many do. Many don't. The point is that they get found, and some of them get fixed. In particular, the ones that make the game completely unplayable or which make certain textures or particles completely disappear usually get picked up and fixed (you have no idea how many times Trundle's pillar has been invisible). I play on the PBE a lot, and bugs do get fixed in between patches. Even if many bugs don't get fixed, many others do get fixed because of the PBE. Other bugs get noted, and get fixed later, because fixing bugs can take a lot of time. The PBE has a job, and it does it. That is a better use for it than highly experimental testing.

And to expand, yes, you do have to explain yourself for every change, because this way you give the community a better understanding of your vision and the route you are taking to materialize that

That can be done without wasting resources on making the PBE do something that can be done more effectively in private testing. More communication on what champions they are and are not looking at would be a big improvement for Riot, but that can be done just as effectively without PBE changes. Straight communication also doesn't run the risk of having people get attached to a really unhealthy game change and getting pissed when it gets removed (and it would happen. If it buffed a weak champion, there are a lot people wouldn't care if it made every other champion suffer). There is a lot less risk in just communicating without presenting whatever highly experimental, possibly terrible idea a developer has to the easily-annoyed community. I mean, just a few days ago a Rioter said they were working on a tenative new Ashe passive that might come out later this season. I saw people swearing to uninstall the game if the changes didn't go through, and they don't even know anything about it. There is risk in showing people a lot of stuff..

I don't see much of that happening to begin with.

If it was happening, would you see it? It's private testing. You shouldn't be seeing it. You should be seeing good balance changes, and hearing responses to community complaints about champions. Seeing the full testing process isn't necessary for either of those, and it probably wouldn't be helpful.

Correct me if I'm wrong, but I think a lot of the basis for this idea is frustration with Riot's bad balance and lack of communication about certain champions, right? Well, I can agree with that, but I just don't think this idea is going to fix anything. It just seems like a way of forcing communication, but not actually using the PBE for any useful purpose. I think it would actually hinder testing a lot. I just want them to communicate more, not try to get people involved in the really experimental parts of design.

Fed by Snu SnuMar 20, 20151 votes

{quoted}

Correct me if I'm wrong, but I think a lot of the basis for this idea is frustration with Riot's bad balance and lack of communication about certain champions, right? Well, I can agree with that, but I just don't think this idea is going to fix anything. It just seems like a way of forcing communication, but not actually using the PBE for any useful purpose. I think it would actually hinder testing a lot. I just want them to communicate more, not try to get people involved in the really experimental parts of design.

Sort of.

It's also frustration with the pace at which attempts to balance things are happening.

There seems to be little proper effort put into it .

Mind you, this is from someone who has been here since the newly released champion was Garen and Akali.

I can understand screw ups, because balance is delicate, especially as long as they keep on insisting on silly goals like "champion picking shouldn't be important" (straight up impossible to do, but I won't stop them from trying.). I don't like hypocrisy though and most of Riot's changed the past 2 years has been mostly that, with buzzwords being used to nerf one champion and then another being guilty of the exact same thing getting a "get out of jail" card, etc.

I want some consistency to summarize. Or at least some proof that they are working towards it, which is why I request feedback on their actions instead.

SmokingPuffinMar 20, 20151 votes

I don't like hypocrisy though and most of Riot's changed the past 2 years has been mostly that, with buzzwords being used to nerf one champion and then another being guilty of the exact same thing getting a "get out of jail" card, etc.

They talk about exactly this on the 5.5 patch rundown video. Highly recommend you watch it.

2000boxesMar 19, 20152 votes

but that nerf ruined zed's split pushing he can't push towers that fast anymore

JunkοMar 20, 20152 votes

As a Zed main i can say Zed would be perfectly fine without his % bonus AD on W. Hell i build him glass cannon AD and i'm seriously surprised that i'm allowed to get 150 free AD from his W with a full build.

VoidgolemMar 20, 20151 votes

The PBE, as I understand it, is for catching the dumb things like ahri being able to deny minions and turrets for no conceivable reason or specific combinations of setups and configurations and Yorick Interactions that make the server shit itself and die.

Notsomuch actual gameplay stuff. That's what internal testing is for I guess.

JacctheInsomniacMar 20, 20151 votes

[deleted]

JunkοMar 20, 20151 votes

Yasuo builds glass cannon AD, gets nerfed because his ult has an amazing AD scaling and his damage is no longer random and reduced.

Kuroi86Mar 20, 20151 votes

It's a place where players play. And Riot thereby looks at numbers that occur from resulting games to get a good idea of what happens when certain factors are changed.

As for

But no, we don't even get an explanation why a miniscule buff like the Darius one got reverted.

Riot has pointed out on multiple occasions, including recently that certain changes are reverted prior to the patch being locked in. It's happened quite a few times.

And

We get no explanation why Zed gets no attention whatsoever or even an actual attempt to be looked at.

The nerf he recieved was a minor dent in his split pushing to be fair, though it did impact his popularity, so all in all the nerf did what it was meant to do. And with recent buffs on Athene's coming up on pbe, if that goes through it'll be an indirect nerf to him.

So yeah.

mi ramfanMar 20, 20151 votes

When the PBE actually was used to test experimental balance stuff like a Lee Sin rework, the community exploded with rage. Half of the community was outraged that Riot was considering a Lee Sin rework in the first place, while the other half was outraged that the rework didn't go live despite being on PBE.

It's because of stuff like that that Riot does not use the PBE for balance testing, and mostly just uses it to preload their next patch.

Occams RaiserMar 20, 20151 votes

From some of the comments I see here, It makes me think that players don't realize RIOT has paid game tester that do what you think PBE is supposed to do. You aren't really even beta testing. They can tweak things any time they want. This isn't a physical release. Just act like you got into Comic-con on Preview night. It's not Hall H. What.... You never been to comic con?