Two Simple Ideas For Upvote/Downvote Mechanics

CupcakeTrap·Aug 7, 2014·3 votes·4,893 views

Upvotes and downvotes do not need to have the same weight.

Perhaps out of pure mathematical simplicity, Reddit makes upvotes worth +1 point and downvotes worth -1 point. It is not obvious to me why it necessarily has to be like this. I propose that downvotes should have less weight than upvotes. Here are three reasons why.

Abusive downvotes are a bigger problem than undeserved upvotes. There seems to be a consensus that we have a problem with unpopular opinions getting buried due to spurious "I disagree with your opinion!" downvotes, or simply mean-spirited "I'm not interested in lore/eSports/artwork/Twisted Treeline, so you shouldn't be either!" downvotes. In contrast, there aren't as many obvious abuse cases for upvoting.
Upvotes promote discussion, downvotes stifle it. Upvotes and downvotes are inherently different, in that upvotes increase visibility and invite further discussion, while downvotes kill discussion. A stupid upvoted post might trigger discussion that will help turn community opinion against such repetitious preaching-to-the-choir posts, while a thought-provoking but unpopular downvoted thread will never be processed.
Really awful posts are currently overkilled, so it's safe to reduce downvote weight. One important function of downvoting is to get rid of obvious trash posts ("get 4000 RP free just click here"). If we removed downvotes altogether, these might start to rise up and muddle the forum. But these posts get downvoted so heavily that, even at (say) quarter-strength, the tide of downvotes will suffice to quickly bury them.

Votes can work differently at different phases of a thread's life.

The system could be set up so that voting works differently at the very start of a thread's life, such as for the first hour or the first ten votes. I propose that downvotes are temporarily ignored during the first part of a thread's life. Here are two reasons why.

The first test of a thread should be "does anyone care?" It should be easier to be promoted to "hmm, this warrants a glance by a larger set of people" than it is to rocket to the top and become a Big Thread. To make a slightly absurd comparison, it takes less than a majority of Justices for the Supreme Court to agree to hear a case. Many parliamentary systems work similarly: it's much easier to get a bill put forward for general consideration than to adopt it.
The Reddit vote-weighting mechanism doesn't cut it here. Reddit does change how votes are counted over time, by making the first 10 votes as impactful as the next 100 votes, which are as impactful as the next 1000 votes, and so on. However, this doesn't really address the major problem of good posts being insta-killed by a few non-representative, mean-spirited jerks stalking through the New section: it just makes their votes stronger.

12 Comments

PendragonAug 8, 20144 votes

Thanks for the well thought-out post. I think one thing to consider here is that what matters most is the desired outcome - not necessarily the philosophical viewpoint on what the functionality represents.

Ultimately what we're looking to do is make it so the content that every player sees is relevant. We build instrumentation around what it means for content to be relevant like views, read-throughs, votes, comments, etc and each of those things is increasingly an indication that content is relevant.

Then we have to look at what influences those things. Maybe it's what other people like (upvotes) or didn't like (downvotes), maybe it's content from people I respect (pros, content creators, Riot, etc), maybe it's because it's from a board that I'm more interested in (Esports, Concepts, etc), maybe it's because the content is really long, or really short, or maybe I prefer really deep discussions, and so on. Lots of things can influence whether or not you like something - much more than just up and down votes.

Then over time you break it out a few ways -- first you have to make really strong defaults. The average player won't log in, won't opt in/out of certain board content, and won't participate by voting or commenting - so you have to have a reasonable generic view.

Then once that's strong - you can start to create slightly more sophisticated, but still generalized views with things like board subscriptions - ways for large groups to exist that will share some sorts of preferences.

Then over a long enough time you can individualize - so that your feed is uniquely you. We may never get to this point but it's certainly where it can go.

Of course you also have the people who don't want to trust the system - so you still leave all of the controls in place for people who want to chart their own content course.

CupcakeTrapAug 8, 20141 votes

Thanks for the well thought-out post. I think one thing to consider here is that what matters most is the desired outcome - not necessarily the philosophical viewpoint on what the functionality represents.

I agree that outcomes are important. For example, one major rationale for downvotes is that they allow the community to toss out zero-content troll threads, so that the system can distinguish between "thread that not a lot of people care about, though some do" from "spam that one or two people upvoted for kicks but most people want to get rid of". There's also some utility to be gained from balancing out "I think this is interesting" data with "I don't find this interesting" data, and specifically from distinguishing between "not really interested" (no vote) and "I'm upset the system even put this forward for me to look at" (downvote).

That said, since I conceptualize upvotes/downvotes as ways for the system to collate and process a large number of user reactions to content, I think that looking at what this communication represents is worthwhile. Figuring out when and why people upvote or downvote seems important to making use of that data effectively.

Of course you also have the people who don't want to trust the system - so you still leave all of the controls in place for people who want to chart their own content course.

It would certainly be interesting to have a mode that displays posts as they would appear on GD: i.e., a traditional forum where a post takes a thread to the top. I actually think I might enjoy occasionally browsing using a bump mechanic for thread visibility, but with multi-threaded comments.

PendragonAug 8, 20144 votes

sure but you can't look at the philosophical "value" of the function, and instead you have to look at the practical value of the function. We will tune/educate/whatever the weight of the buttons to generate the most relevant possible suite of content based on an observed outcome regardless of if that impact matches our intuition.

It would certainly be interesting to have a mode that displays posts as they would appear on GD: i.e., a traditional forum where a post takes a thread to the top.

Spoiler alert.

CupcakeTrapAug 8, 20141 votes

Spoiler alert.

^_^

It would also be nice to figure out some solution to the "good posts instakilled in New" problem, if you decide that this is happening and is a problem.

Part of why I'm speaking about philosophical values here is because I just don't have access to non-anecdotal data. I do see some philosophical merit behind the idea of treating a post's journey toward visibility (or obscurity) as a two-phase process, with an upvote-focused "is this interesting to anyone at all?" first phase and a more comprehensive "how interesting is it?" second phase.

I'm also a fan of the idea of including a random sampling of new posts in the default view. I'm not sure it's good if people can self-select into the role of "gatekeeper/arbiter of new content". Their preferences might significantly diverge from the general community's.

For example, let's say I just HATE artwork for some reason. I think it's silly and distracts from hardcore competitive gaming, maybe. So I go and start downvoting all the artwork in New, and upvoting all the eSports material, every day. I'm going to have a disproportionate impact on what threads the general community sees.

I'm really excited to see what Riot might be able to come up with here. I think a richer, multi-input sorting system has the potential to produce much better outcomes than what a stripped-down Reddit-clone system could accomplish.

PendragonAug 8, 20142 votes

I'm also a fan of the idea of including a random sampling of new posts in the default view. I'm not sure it's good if people can self-select into the role of "gatekeeper/arbiter of new content". Their preferences might significantly diverge from the general community's.

Yes - you would always want to include some content that you're less certain about, to do larger experimentation.

http://en.wikipedia.org/wiki/Hill_climbing

For example, let's say I just HATE artwork for some reason. I think it's silly and distracts from hardcore competitive gaming, maybe. So I go and start downvoting all the artwork in New, and upvoting all the eSports material, every day. I'm going to have a disproportionate impact on what threads the general community sees.

I'd rather build the system with the notion that players are all fair, just, and honest - and figure out systems to manage against abuse than vice versa. Also - you have to consider how (likely) little impact that individual outlier would actually have. Purity of the output is a lot more important than purity of the input. if 5% of all votes are mis-cast it actually doesn't matter that much as long as 5% of votes are ALWAYS mis-cast. Change in levels of abuse could potentially be damaging though.

Otherwise yes - we'd like to solve the single downvote issue. We think it's bad.

Hyrum GraffAug 14, 20141 votes

Change in levels of abuse could potentially be damaging though.

Which is why we need to get GD over here, as I think GD currently sees a lot more abuse than these boards.

Hyrum GraffAug 14, 20141 votes

Of course you also have the people who don't want to trust the system - so you still leave all of the controls in place for people who want to chart their own content course.

Thank you <3

Hyrum GraffAug 8, 20143 votes

Another reason in favor of making upvotes be worth than downvotes is this

Post gets downvoted early, is never seen again.
Post gets upvoted early, is seen by more people, who realize it's bad content, and downvote it again.

exec3Aug 9, 20143 votes

I wish they'd show the number of upvotes and downvotes instead of just the sum of the total. You could have 100 downvotes and 50 upvotes but you'd only see 50 downvotes thinking no one liked your post. It often just feels mean spirited when you mean well when you write a post and it gets downvoted to hell and back for whatever reason without knowing that some people actually did like your post.

I'd also be curious what would happen if the forums were only an upvote system with a different button for getting rid of "free 4000RP" posts or alternatively not displaying a negative number of downvotes (stopping at 0 even though internally the number might be -20), this will still show a general indifference/disagreement with the post without coming off as mean spirited (proof of downvote system stifling conversation, I almost didn't write these other ideas because of how unpopular they'll probably be resulting in downvote hell).

DaenAug 7, 20141 votes

Abusive downvotes are a bigger problem than undeserved upvotes.

Are they? The example you give is "If I'm not interested in this, no one is interested in this", but that can very easily be flipped into "if I'm interested in this, everyone is interested in this".

Upvotes promote discussion, downvotes stifle it.

I do actually agree with you on this. However, there is a very important flip side as well. It is possible for comments to become so inflated that other comments cannot compete for visibility and thus become stifled.

I propose that downvotes are temporarily ignored during the first part of a thread's life.

While I understand the premise, this would completely undermine the community's ability to self-regulate for however long downvotes are ignored. It would be impossible to get rid of troll posts or even really feel like you are able to show a dissenting opinion during that time period, which to me is a very significant downside.

Ideally, downvotes should be calculated in conjunction with upvotes and comments. It should be very possible for a thread to be negatively received by the community but still visible, as long as it promotes discussion.

CupcakeTrapAug 7, 20141 votes

A compromise combination solution: for the first phase of a post's life, downvotes are weighted less than upvotes. Troll threads still get buried, but potentially controversial or quirky posts get a shot.

It's true that visibility is a rival resource, and every thread moving up means other threads moving down. However, this is a much more diffuse effect than downvoting.

Your comment has also led me to another possible idea: voting could be somewhat subforum-specific. If we can come up with a way to identify users as, e.g., "an eSports fan" or "an art fan", we could make their votes have extra weight when the system computes visibility for people viewing from a subforum. I'm not sure how I feel about this idea, but there might be something to it.

GranspearZXAug 8, 20141 votes

"If we can come up with a way to identify users as, e.g., "an eSports fan" or "an art fan", we could make their votes have extra weight when the system computes visibility for people viewing from a subforum. I'm not sure how I feel about this idea, but there might be something to it."

Does it really need to be explained why turning the forums/boards into a meritocracy is a bad thing?