Today, 6 percent of all adult American Internet users are on Reddit. Like Yelp, IMDb, or Amazon.com, the site runs on the principle of voting aggregation. Each user can give a post or comment an upvote of approval or a downvote of disapproval. Highly ranked news and content float to the top of the site, and highly rated comments float to the top of their threads. It's a system that ensures that the best, most insightful, or most representative material rises to the top. Or so the theory goes.
Three business researchers, led by Lev Muchnik at the Hebrew University of Jerusalem, conducted their own online voting aggregation experiment, the results of which came out today in the scientific journal Science. They found that groupthink plays a surprisingly large a role in how an online communities like Reddit make quality judgments. For some users in the study, seeing a single extra upvote very early in a comment's life could change their mind about how they felt about a comment, regardless of what the comment actually said.
"It's important to understand the underlying mechanisms of how people vote. It helps us understand how websites can avoid this type of bias in the future," says Sinan Aral, a coauthor of the paper and social-media researcher at MIT and NYU.
"We used a site very similar to Reddit," Aral says. Here's how it worked: The researchers convinced an anonymous website–under the promise it would remain nameless–to allow them to test slight manipulations in online voting. Over six months, the research team took a small percentage of more than 100,000 written comments and slightly altered their vote rating as they were submitted. Four percent of the comments were given a positive edge (a single upvote) while another 2 percent were struck with an immediate disadvantage (a lonely downvote).
The researchers discovered that by increasing a comment's score with a single vote, they would boost its final score by an average of 25 percent. "There is a herding effect," Aral says. "It was quite dramatic. I was surprised to find that a single positive vote could create such a huge snowball effect."
Oddly enough, this snowballing didn't work in reverse—nobody was ganging up on commenters unfairly based on a downvote or two. Instead, the researcher's artificial downvote made users slightly more likely to respond positively, essentially negating the researchers' interference. "Basically there was a correction effect," Aral says. "One explanation is that people will go along with positive opinions but are more skeptical of the negative opinions of others."
The Hivemind
Reddit is occasionally chastised, even from the inside, for sameness in opinions and groupthink. Astrophysicist Neil deGrasse Tyson and comedian Louis C.K. are beloved almost beyond criticism, and many traditionally liberal political and social views dominate conversations. While the researchers are the first to quantify this up/downvoting online herding effect, it may not come as a surprise to many redditors—on the site, the tendency to groupthink is commonly referred to on the site as the hivemind.
"Twenty-five percent sounds about right," says Erik Martin, the general manager of Reddit. "It doesn't surprise me that comments that get a positive vote have that momentum. There are certainly some aspects of an echo chamber."
By separating the users they sampled into those likely to vote positively and those likely to vote negatively, the researchers broke down the herding effect into a combination of two behaviors. "There was evidence of both opinion change and voter turnout," Aral says. The researcher's extra upvote seemed to convince many nonvoters to join in, and it also encouraged positive votes from those who might have otherwise voted negatively or abstained.
And, Martin argues, the effect is not all bad: "I think the term hivemind is used pejoratively, and people forget all the things it enables. People are much more willing to participate in a thing, like to moderate a subreddit. It enables new subreddits to crop out and have support. If everyone was supercritical all the time, it wouldn't work."
Social Exploitation
One concern for Reddit or any website using voting aggregation is businesses attempting to exploit the heading effect of online users to create false social momentum. "I think it's a very real concern," says Duncan Watts, a social-network researcher at Microsoft. "Whenever you open up and do this kind of community ranking exercise, it's an invitation to manipulation."
There are already places to buy fake Amazon.com reviews, and false votes for Tripadvisor, Yelp, Citysearch, and somewhere, I'm sure, Reddit. The worry isn't just that these efforts are skewing the aggregated vote, but that they are actually influencing or changing the decisions of how real reviewers might also vote.
But neither Martin nor Watts seem overly concerned. "We've caught several high-profile publishers doing that over the years, and we've banned them," Martin says. "But we have counter measures—people who actively look for that stuff, a community that has no tolerance for that manipulation," and undisclosed algorithms the site uses to root out false or exploitative accounts.
"And there's a limit to how much manipulation you can do before the manipulation starts to undercut the effect," Watts says. "If the signal is so at odds with your experience—like if you go to Amazon and you see a book with 100 five-star reviews—you know it's just clearly fake."
For Watts, the takeaway is that for users of sites like Reddit, "you should be a little bit suspicious and shouldn't jump to conclusion about the quality of a story based on its apparent popularity," he says. "Not because it might have been manipulated, but because big effects are often due to accident and cumulative advantage. It could be that it got a lot of attention simply because other people were paying attention to it."
And for a site wholly dependent on user-submitted content, the herding effect might not be such a bad thing. "It's a community that's cynical and very skeptical," Martin says, "but their default is to say yes or to do something. It enables the site to exist."
