[Idea] Player Mods

Hyrum Graff·Jun 23, 2014·2 votes·321 views

Edit: I have since come up with a better idea, IMO, which is here. OP at bottom.

Last week, a rioter (I don't remember who) made a comment to the following effect:

Player: If a thread is in the wrong sub-comm, why do you close/delete it, instead of just moving it to the correct sub-comm?

Rioter: It's a more sustainable long-term practice. As the CB's current size, we could do what you proposed, but when GD migrates over - the extra couple seconds to do this per post adds up into thousands of hours of work; we just won't have the manpower. Furthermore, we want to encourage people to post in the right sub-comm to begin with; if we make a practice of moving posts, it encourages the mindset of "I'll just post this wherever because I know it'll get moved to the right sub-comm."

The fundamental issue here is scaling - as the CB population rises, Riot just can't do so much involved moderation. The problem with their intended approach, though, is that many people who post, aren't regular posters, and are liable to make mistakes. Having your post deleted because of an honest mistake, isn't a good experience.

So, with that in mind, what if we had a network of Player Mods, who are normal community members who are able to move content to the correct boards, while Riot Mods were looking for inappropriate content, spam, etc. The larger the community gets, the larger the pool of potential Mods, so there's no scaling issue. Here's how I envision it to work:

Right now, there's a "Report" button for posts, and one of the options is "Incorrect Board".

Under the new system, when you click that option, it will also ask you "What board does this post belong in?"

Riot and Player Mods can look at a list of posts that have been flagged as being in the Incorrect Board (I'll just call these misposts/misposted from now on), and move them to the correct board. When they move the post, the CB keeps track of which players reported the post, and which selected the sub-comm that it was actually moved to.

When a player consistently reports misposts and reliably selects the correct board for them, they are automatically promoted to player mod status. Now this player can go look at the list of posts that have been reported as misposts and move them.

If this player moves a post, and after they move it, it is reported as a mispost again, this post will be brought to the attention of the other player mods (put at the top of the listing of misposts), where they will vote on the sub-comm it belongs in. If they vote that it was moved incorrectly, the player who moved it will lose their mod privileges (maybe they'll get two strikes).

Now we have an automated system for the community to self-promote and self-regulate mods. Riot can adjust their algorithms so that they always have the optimum number of active player mods to keep the forums well-, but not over-regulated.

If this promotes the mindset of "It doesn't matter where I post it because it'll get moved to the right forum anyway," I don't think this is a problem. It's only a problem in the current situation because Riot lacks the manpower to regulate misposting of that magnitude.

Moreover, I think that mindset is preferable to the mindset of "Riot's Mods are evil dictator police who over-regulate their forums and delete constructive threads just to enforce their forum rules," which is something that could arise from Riot's current vision of Mods.

####OP Below

What if there were a group of known, trusted, constructive, active forum users (like the lovely Sir Armamalum) who were given SOME mod privileges (ability to close/move threads for being in the wrong sub-comm; NOT the ability to ban/restrict players' access)?

They could potentially alleviate the amount of work that the paid mods. I also feel that "Community regulating itself" is a nicer way to have things than "Riot regulating the Community."

Thoughts?

10 Comments

12talesJun 23, 20141 votes

It seems to me that this would only be necessary if the current moderators were unable to adequately cover the entire community beta, which would surprise me - with the exception of the posts that show up in the client, the community beta seems to be relatively low activity.

It's also worth asking yourself - if Riot would have the power to instate the members of the community who would do the regulating, and would presumably outline exactly what is expected of said memebers - is there any meaningful distinction between Riot[whomever] and Community Member [whomever] doing the moderating? It seems to me that exactly the same things would be done either way.

Hyrum GraffJun 23, 20141 votes

It seems to me that this would only be necessary if the current moderators were unable to adequately cover the entire community beta, which would surprise me - with the exception of the posts that show up in the client, the community beta seems to be relatively low activity.

Atm the CB is still pretty small, but I think the plan is to have GD eventually migrate over (Rio's not going to force this - they want it to happen naturally - but they're just going to keep working on this platform until the swap happens on its own), and it would be helpful to have tried this on smaller scale, first - it's much easier to run experiments (well, those that don't involve the need for tons of data) on a community of this size.

Also, despite the small size of the community, I see tons of posts in the wrong sub-comms that have not been moderated yet. This is, in part, an issue with sub-comm naming being really bad (I made a separate post about that), but the bigger point is that if this stuff happens, and isn't being fixed instantaneously with a community of this size, that doesn't bode well for when GD migrates over.

It's also worth asking yourself - if Riot would have the power to instate the members of the community who would do the regulating, and would presumably outline exactly what is expected of said memebers - is there any meaningful distinction between Riot[whomever] and Community Member [whomever] doing the moderating? It seems to me that exactly the same things would be done either way.

The distinction is that Player Mods are only allowed to moderate content, whereas Riot Mods are allowed to moderate players, too (ie, issue temp or permanent forum bans).

12talesJun 23, 20141 votes

The distinction is that Player Mods are only allowed to moderate content, whereas Riot Mods are allowed to moderate players, too (ie, issue temp or permanent forum bans).

My point is that the set [hypothetical player mod actions] has 100% overlap with the set [current Riot mod actions]. Meaning that the change would only be significant to the degree that more of the same stuff would be done - which is only desireable if more of the same stuff needs to be done, e.g. if the moderating team is shorthanded.

If it isn't - my experience with moderating forums has been that a lot of unnecessary conflict and latency can arise in the course of carrying out moderator duties when multiple moderators have equivalent privileges over any given content. I've generally found moderation to be most efficient when each set of content has exactly enough moderators to ensure that everything is read in a timely manner, and no more. Sites where every moderator is given global authority, such that any given subforum (or sub community, in out case) has potentially over a dozen people trying to moderate it, or sites where there are extremely large numbers of moderators assigned to given subforms, tend to moderate content much more slowly and inconsistently.

There are also concerns of infrastructure, to a certain degree - even the most constructive, intelligent, well mannered posters are likely not sufficiently familiar with Riot's vision of the community beta to moderate it ideally, and I could foresee a relatively difficult period of adjustment if multiple such users were taken on simultaneously.

Hyrum GraffJun 23, 20141 votes

Ok, those are interesting and fair points - you seem to have more expertise than me in this field, so I'll defer to your wisdom here.

Last week, a rioter (I don't remember who) made a comment to the following effect:

Player: If a thread is in the wrong sub-comm, why do you close/delete it, instead of just moving it to the correct sub-comm?

Rioter: It's a more sustainable long-term practice. As the CB's current size, we could do what you proposed, but when GD migrates over, it'll just be too much work. Furthermore, we want to encourage people to post in the right sub-comm to begin with; if we make a practice of moving posts, it encourages the mindset of "I'll just post this wherever because I know it'll get moved to the right sub-comm."

So, with that in mind, what if Player Mods moved content to the correct sub-comms, while Riot Mods essentially continued their current duties?

Which sub-comm a topic belongs in, shouldn't be a hotly contested topic, if the sub-comms are designed properly, so it shouldn't matter if a player mod has "Riot's Vision of the CB" or not. Furthermore, since you can bring on as many player mods as possible, there's no scaling issue.

You'll note that I didn't address the rioter's last point. That's because I don't think it's a problem. Let me give a little more detail of my new vision of this system:

Right now, there's a "Report" button for posts, and one of the options is "Incorrect Board".

Under the new system, when you click that option, it will also ask you "What board does this post belong in?"

Riot and Player Mods can look at a list of posts that have been flagged as being in the Incorrect Board (I'll just call these misposts/misposted from now on), and move them to the correct board. When they move the post, the CB keeps track of which players reported the post, and which selected the sub-comm that it was actually moved to.

When a player consistently reports misposts and reliably selects the correct board for them, they are automatically promoted to player mod status. Now this player can go look at the list of posts that have been reported as misposts and move them.

If this player moves a post, and after they move it, it is reported as a mispost again, this post will be brought to the attention of the other player mods (put at the top of the listing of misposts), where they will vote on the sub-comm it belongs in. If they vote that it was moved incorrectly, the player who moved it will lose their mod privileges (maybe they'll get two strikes).

Now we have an automated system for the community to self-promote and self-regulate mods. Riot can adjust their algorithms so that they always have the optimum number of active player mods to keep the forums well-, but not over-regulated.

Moreover, I think that mindset is preferable to the mindset of "Riot's Mods are evil dictator police who over-regulate their forums and delete constructive threads just to enforce their forum rules," which is something that could arise from Riot's current vision of Mods.

12talesJun 23, 20141 votes

Last week, a rioter (I don't remember who) made a comment to the following effect:

Player: If a thread is in the wrong sub-comm, why do you close/delete it, instead of just moving it to the correct sub-comm?

Rioter: It's a more sustainable long-term practice. As the CB's current size, we could do what you proposed, but when GD migrates over, it'll just be too much work. Furthermore, we want to encourage people to post in the right sub-comm to begin with; if we make a practice of moving posts, it encourages the mindset of "I'll just post this wherever because I know it'll get moved to the right sub-comm."

I'm surprised that simply moving content could be considered unsustainable. In fairness, my experience is with vBulletin, but the difference between moving and deleting a thread there is probably about 5 seconds.

If you're starting with the assumption that misfiled threads will be moderated at all, it seems to me that relocation is preferable to removal. Deleting posts for reasons that will likely be seen as incredibly nitty-gritty and arbitrary weakens the premise that, if you take time and effort to submit content to the community, you will be rewarded by having your content displayed and archived for all to see. If you take away that premise entirely, there's simply no reason to bother posting. Even weakening it hurts user retention, though, which is bad for any community's health.

You'll note that I didn't address the rioter's last point. That's because I don't think it's a problem. Let me give a little more detail of my new vision of this system:

-snip automated system-

I strongly disagree with your proposal. Automatically granting mod authority, even to a limited extent, leaves you incredibly open to abuse. Using your system as an example, if I want to be granted the authority to move posts, I can get a bunch of friends together, register a bunch of smurfs, submit a bunch of threads in the wrong sub communities and get brownie points for reporting them on my main. And now we have a system that rewards violating the rules (this is part due to the broader problem of allowing multiple accounts in the first place, but whatever).

For sure, keeping track of players who make consistently helpful reports is a good idea if you're entertaining the possibility of enlisting their service in the future, but no automatic promotions please.

If this promotes the mindset of "It doesn't matter where I post it because it'll get moved to the right forum anyway," I don't think this is a problem. It's only a problem in the current situation because Riot lacks the manpower to regulate misposting of that magnitude.

Moreover, I think that mindset is preferable to the mindset of "Riot's Mods are evil dictator police who over-regulate their forums and delete constructive threads just to enforce their forum rules," which is something that could arise from Riot's current vision of Mods.

As an aside, I have literally never encountered a forum where people didn't claim the moderators were evil dictator police. It comes with the territory.

And the easy solution to the indicated mindset is to restrict the thread creating privileges of anyone who continues to misfile threads after having been corrected. The vast majority of people, though, once informed they have made a mistake, attempt to not make that same mistake. I don't see the indicated mindset as anywhere near the intuitive progression from helping out people who file their threads incorrectly.

Hyrum GraffJun 23, 20141 votes

Found it!

Original question:

How about instead of removing threads in the wrong place, you just move them to correct place?

Tamat's Response:

Great question! I can think of many reasons, but I only have time to go into a few.

First, that approach doesn't scale well with hundreds of thousands of posts per week. Let's say that you get 100k posts and 10% are posted in the wrong category. This is 10,000 posts that need to be moved, roughly taking 15-30 seconds to accomplish. Do the math.

It makes more sense to purge the content that was posted in error. We'll be sensitive to the fact that everyone makes mistakes, but we want to train everyone to post content in the right place, and that won't happen if we take a "housekeeping" approach.

Second, we want you to be able to find relevant content in the relevant place. There needs to be some structure, otherwise we'll have the same situation we have in our old forums. There are 20+ boards to post in, but General Discussion is the only active board. It's a dumping ground for everything; little is properly curated. This creates many problems in a social platform, all of which we'd like to avoid here.

To do that, we need to train people to use these boards correctly. Riot, and you guys. Once we have a solid base of early adopters (you guys) that are using the platform correctly, it becomes an environment where new users will learn by the examples right in front of them.

Does that make sense?

Pardon any mistakes in grammar, I'm in a rush!

The math: 15 seconds * 10k posts = 2.5k minutes per week = ~42 hours per week

####Now to respond to you:

Moving posts seems to be preferable to deleting them.

I agree here, which is one of the reasons I'm proposing Player Mods being a thing - it gives riot the manpower to do what I think is the better option.

Automatically granting mod authority is bad

If you say so. I think it's well within Riot's abilities to use automation to determine when people are making lots of smurfs like this, and prevent them from becoming mods at all in the first place (and even if they did become mods and wreak a little havoc, they would quickly lose their mod privileges the second they misplaced a thread, and the thread would quickly be relocated to the correct sub-comm). Whether or not you automate the process, though, it still takes less effort to appoint a mod than to move 100 forum posts.

People will always blame mods

Well, yeah, but there's no reason to give them legitimate complaints.

I don't see the indicated mindset as anywhere near the intuitive progression from helping out people who file their threads incorrectly.

This sentence confuses me. Do you mean, people whose threads get moved/deleted, won't make the progression to "Mods = dictator police"? I agree that's true for people whose threads get moved, but less so for people who get their threads deleted, which is Riot's current intended course of action.

12talesJun 23, 20141 votes

Found it!

-snip-

I'm a bit confused here. Posts cannot be filed in the wrong category - only threads (and 100k threads in a week seems implausible). Does the mod interface in the community beta require you to move every post in a misfiled thread individually or something?

Moving posts seems to be preferable to deleting them.

I agree here, which is one of the reasons I'm proposing Player Mods being a thing - it gives riot the manpower to do what I think is the better option.

If they lack the manpower, then sure.

If you say so. I think it's well within Riot's abilities to use automation to determine when people are making lots of smurfs like this, and prevent them from becoming mods at all in the first place (and even if they did become mods and wreak a little havoc, they would quickly lose their mod privileges the second they misplaced a thread, and the thread would quickly be relocated to the correct sub-comm). Whether or not you automate the process, though, it still takes less effort to appoint a mod than to move 100 forum posts.

In that case, you have to question whether it's more work to design a program that automatically detects all forms of abuse in your automatic system, or to just promote sensible folk manually. I think the latter would be a lot easier, considering that it really doesn't take that many people to manage something as trivial as thread filing.

People will always blame mods

Well, yeah, but there's no reason to give them legitimate complaints.

Specifically, I mean that a significant number of people seem to intuitively react to moderation with indignation. I don't think polemics on moderation policy are really avoidable or concerning.

I don't see the indicated mindset as anywhere near the intuitive progression from helping out people who file their threads incorrectly.

This sentence confuses me. Do you mean, people whose threads get moved/deleted, won't make the progression to "Mods = dictator police"? I agree that's true for people whose threads get moved, but less so for people who get their threads deleted, which is Riot's current intended course of action.

I mean that, if your typical user gets his thread moved, he'll probably react with something along the lines of 'Oh crap, I'll try not to do that again.' I don't think most people will react with something along the lines of 'Oh sweet, Riot files my posts for me.' And those that do can be accordingly disciplined.

Oh, and sorry if I'm being hard to understand. I'm pretty tired atm, and for some reason that makes me super wordy.

Hyrum GraffJun 23, 20141 votes

I'm a bit confused here. Posts cannot be filed in the wrong category - only threads (and 100k threads in a week seems implausible). Does the mod interface in the community beta require you to move every post in a misfiled thread individually or something?

I have no clue. I assume not, though. 100k threads/week -> 14.3k threads/day -> 600 threads/hour -> 10 threads per minute. GD's frontpage has 50 threads. I just did a test bump. There were around 24 threads that were bumped to a higher position than the one I bumped, in around a minute. While the average number of replies was 9 (not inc OP), 14 posts had 3 or fewer replies: 6 of those were 1-reply threads and 4 were no-reply threads.

10 posts with only 0 or 1 reply in a minute - considering how unscientific this was, that's close enough for me to say that Tamat's estimate isn't completely out of the ballpark.

If they lack the manpower, then sure.

It also has the secondary benefit of having players moderate players, instead of Riot moderating players - look at how often the Tribunal was questioned pre- and post-closing: when it was run by players, very few people complained about its accuracy (note: nothing to do with effectiveness). After the swap to the current system, we saw a rash of "Got banned for no reason" threads. Of course, all these were #Rekt by WookieCookie or others, which bolstered people's confidence in the system, but there were still accusations thrown at Riot that didn't exist when the Tribunal was in place.

In that case, you have to question whether it's more work to design a program that automatically detects all forms of abuse in your automatic system, or to just promote sensible folk manually. I think the latter would be a lot easier, considering that it really doesn't take that many people to manage something as trivial as thread filing.

Fair Enough, though I'd like to raise one counterpoint: such a program would have broader applications than only detecting this kind of abuse on the forums.

Specifically, I mean that a significant number of people seem to intuitively react to moderation with indignation. I don't think polemics on moderation policy are really avoidable or concerning.

Do they react to player moderation the same way, though?

I mean that, if your typical user gets his thread moved, he'll probably react with something along the lines of 'Oh crap, I'll try not to do that again.' I don't think most people will react with something along the lines of 'Oh sweet, Riot files my posts for me.' And those that do can be accordingly disciplined.

I think we're in agreement here: moving posts is a better solution than simply deleting the content.

Oh, and sorry if I'm being hard to understand. I'm pretty tired atm, and for some reason that makes me super wordy.

Haha, no problem; I know the feeling.

We've hit the maximum thread depth - you can just reply to the OP and we'll pick up from there, if there's anything more to say.

12talesJun 24, 20141 votes

re: Hyrum Graff

I have no clue. I assume not, though. 100k threads/week -> 14.3k threads/day -> 600 threads/hour -> 10 threads per minute. GD's frontpage has 50 threads. I just did a test bump. There were around 24 threads that were bumped to a higher position than the one I bumped, in around a minute. While the average number of replies was 9 (not inc OP), 14 posts had 3 or fewer replies: 6 of those were 1-reply threads and 4 were no-reply threads. 10 posts with only 0 or 1 reply in a minute - considering how unscientific this was, that's close enough for me to say that Tamat's estimate isn't completely out of the ballpark.

This is difficult to address without the real statistics on thread creation in front of me, and even with those numbers I'd need to know how the mod interface works on this platform to weigh the amount of work involved.

Still, let's assume that the Community Beta will be getting 100k threads a week at some point in the future, and use Tamat's numbers (10% of them are misfiled, it takes 30 seconds to move a thread). At the moment, there are 7 permanent subcommunities that have significant activity. If we assume that threads are misfiled indiscriminately, that gives us ~1400 misfiled threads per subcommunity, or just shy of 12 hours of work per week per subcommunity. We then end up with 1.7 hours of maintenance per subcommunity per day, which we can divide by the number of moderators who will be performing that maintenance. With three mods per sub community, for instance, that totals to just over half and hour of work that each person needs to do each day. Naturally, all of these numbers are halved if his lower estimate (15 seconds) is correct.

Of course, that half hour is the difference between moving misfiled threads, and just throwing up your hands and never touching the mod interface again. In evaluating how much extra work would need to be done, you'd have to subtract the amount of time it takes to delete all of the threads.

So, at worst, with a community beta that gets 100k threads every week, it takes 20 bros willing to put in ~20-30 minutes per day to completely resolve the issue of misfiled threads. That can be weighed against the frustration and cynicism that deleting content people have put time and effort into causes, and the negative impact inspiring such feelings has on user retention.

-snip nazi mods-

The Tribunal was anonymous and at least nominally democratic. If you've ever seen how a kid in school gets treated when he tells someone off to the teacher, you know how a lot of people react to being 'moderated' by one of their own.

Do they react to player moderation the same way, though?

I'm inclined to say so. The forum I moderate uses volunteer mods exclusively, and we've still been accused of favoritism, communism, fascism, nazism, and simple dickishness on a regular basis for as long as I can remember.

Hyrum GraffJun 24, 20141 votes

My only disagreement is that, if Player Mods only move threads, not dole out punishments, I'm not sure there would be so much of a resentment issue there, especially if the "friendly community members helping to keep everything in the right place" persona were cultivated.