2.6 Grudin problems
The benefits of learning provide a partial answer to Grudin challenges faced by sites like Perlmonks and Everything2. But they do not answer the challenges completely. For instance, disparities between work and benefits may be a function of a user's level of expertise. What keeps successful sites from degenerating into a community of hacks that caters only to the lowest common denominator?
Two of Grudin's eight challenges, disparity in work and benefit and critical mass and the Prisoner's dilemma problem, attracted our attention at the beginning of this project. However, in this section we explain why a similar, but not identical, game theoretic problem to the Prisoner's dilemma, the free-rider problem associated with the private provision of public goods, may be a better choice. The free-rider problem may not only provide a more accurate representation of the problem faced by community based Web sites with respect to both the first and the second Grudin challenges, it also suggests the potential for interventions.
Our discussion is motivated by ideas from the economic theory of mechanism design and explains how the elaborate system of experience points (XP) and levels found in the Everything2 site may be interpreted as a mechanism to counteract the free-rider problem.
The role of community Web sites are portrayed in a utopian light in books like Howard Rheingold's The Virtual Community (1993). Rheingold wrote of the role community Web sites would play in the "self-design of a new kind of culture." While Rheingold's inspiration, The WELL, survives today, community Web sites account for a minute fraction of the material on today's Internet. What happened? The failure of many community Web sites to achieve critical mass is a likely explanation. The free-rider problem offers a specific explanation for why an initial burst of enthusiasm for community-based Web sites may not have translated into a lasting impact. In a typical private provision of public goods problem, those who have the highest value for the public good contribute (typically the community site founders) and the rest free-ride. Yet Everything2 and Perl Monks have both found ways to attract and maintain a critical mass of active participants, a key challenge for sites dependent on the quantity and quality of user contributions. With an interested in how this was achieved, we developed our research design with an eye towards answering the questions: "how did the developers of these sites achieve critical mass?" and "how did they maintain it?" More specifically, we asked, "what mechanisms, if any were added to these sites that might counteract the free-rider problem and how do they function in practice?"
2.6.1 Achieving critical mass: The Slashdot Connection
In our user survey, we asked where users first heard of Everything2 or Perlmonks. The top three responses were "Slashdot" (50 percent), "word of mouth" (25 percent) and "links from other sites" (14 percent). The results suggest that the ability of these sites to attract new members is due primarily to the popularity of Slashdot and to the strength of the social networks of existing community members. That "links from other sites" drew in only 14 percent of the respondents and all other means of communication drew in only 11 percent suggests a difficulty new sites may have in attracting members with the Internet's current state of maturity. A community site that has neither the advantages of a strong social network among the founders or a feeder site is likely to have to come up with some other method than the ones we observed for attracting the initial user base.
2.6.2 The Free-rider problem and the difficulty of sustaining interest
In addition to the challenge of drawing eyeballs, community Web sites face the challenge of keeping them. Our interpretation of the Grudin problem suggests that one problem they must overcome is that of free-riding inherent in the private provision of public goods.
Information that is contributed to a community site corresponds with what economists describe as a public good - meaning that it exhibits two properties, non-excludability and non-rivalry. Non-excludability means that once someone posts information to the site, it can be enjoyed by all. Non-rivalry means that one person's viewing of information on the site does not detract from another person's viewing of the same information. The central problem associated with the private provision of a public good is the free-rider problem, which shows up on community-based sites as lurking. Users have limited time resources (their private good) and presumably gain some utility from reading postings on the site (a public good). Any time they spend posting information to the site diminishes the time they have to spend on other activities; at the same time, users typically lack a way to influence the contribution others make to posting material on the site.
It may be that the act of posting information encourages others to post, but to assume this always follows without support is to make a specific assumption about human nature involving reciprocity that certainly doesn't hold under all circumstances. If we alternatively assume that most users are motivated by self-interest, the potential benefits of being a lurker, hoarding ones time, while counting on others to supply the site content become clear. The nature of the game resembles that of the Prisoner's dilemma, but it is not quite the same. In the Prisoner's dilemma, a defector inflicts punishment on his or her opponent. In the free-rider problem, a lurker's action alone is of no consequence -- since information is non-rivalrous, the lurker's enjoyment of the site is not a direct detriment to others. Of course, the server capacity of the site is limited, but developers of a community Web site would be blessed to have this problem. The real problem is that if lurking is a dominant strategy for one user, it may be the dominant strategy for many users. If everyone lurks, then nothing new is added to the site. With no new material, there is no incentive to visit the site and the community dissolves. While we did not find CSCW literature dwelling on the failure of community-based Web sites, we suggest that the pattern of an initial burst of interest that dissipates as users post less and less material on the site, one that is at least anecdotally consistent with the history of many failed community-based sites is entirely consistent with the explanation offered by the free-rider problem.
So what can a community Web site developer do? The free-rider problem exists because a user has no direct influence over the behavior of other users. This is a consequence of the properties of information as a public good. However, community Web site designer can build mechanisms that seek to control behavior indirectly. We believe the developers of Everything2 and Perlmonks have achieved this with their elaborate system of experience points (XP) accompanied by rewards in the form of increased permissions. Alternatively, developers may attempt to exert indirect control through the development of a shared culture, trying to mold presumably self-interested users into community minded members. We also found evidence of developers and editors attempts to mold community norms of behavior of Everything2. Indeed, the architectural interventions of the rating system and a status hierarchy and direct social intervention through the /msg feature and chatterbox appear to be complementary.
2.6.3 The Rating System From a User Perspective
Our contention is that the rating system influences the behavior of users. What did the users say? In our survey, 90 percent of the Perlmonks and the Everything2 users agreed that the rating system influenced their behavior. Forty-nine percent said it influenced their behavior a little, 36 percent said it influence their behavior quite a bit and 4 percent said it had an extreme influence on their behavior. We also asked users whether they thought the rating system improved the site. Ninety-three percent thought it did. Of these, 26 percent thought it improved it a little, 54 percent thought it improved it quite a bit and 14 percent thought it improved it a lot.
2.6.4 The Rating System from the Editors and Gods Perspective
In our e-mail survey of Everything2 editors and gods, the editors were asked how they thought the rating system influenced their behavior and the behavior of other users. With respect to their own behavior, the modal response was that they cared more about experience points when they were new users and ceased to care once they had attained status in the community. Only 15 percent of the editors said the rating system did not affect their behavior. A significant proportion specifically acknowledged, "somewhat sheepishly" in the words of one, that they were still affected by community reactions to their individual write-ups.
In terms of the influence of the rating system on the behavior of other users, editors credited the rating system with performing a socialization function and giving users encouragement and recognition. However, the rating system also produced noticeable negative effects. Editors said that users whined over editor responses, observed that some users tried to game the rating system, that some took the rating system too seriously and that the rating system encouraged trash that played to the prurient interests of the community. "Want to get voted up? Write about getting high and masturbating over another user," one editor responded.
The tension that between producing quality work and "whoring" for experience points is captured by write-ups such as "Node for the Ages":
"You can node for numbers or you can Node for the Ages. If you node for numbers, you can appear on the list of Everything's Best Users, but your more mediocre nodes may eventually be voted down or killed by The Administration. At the end of time, when the roll is called up yonder, you may be remembered as a noder who cared only for your own glory and not for the enlightenment of others.
However, if you Node for the Ages, you have a better chance of getting Cool Nodes, and your reputation as an Everything Demigod will grow. Someday, future generations will look upon your nodes with wonder, marveling at your wisdom and praising your name to the highest heavens."
Similarly, the node "Earn Your Bullshit" offers the following advice:
"…My best advice to any noder before they click "stumbit" is this: will your writeup be interesting to you in a year? That's the key - if you're adding facts then the answer is yes. The facts will, in most cases, remain unchanged in a year. That's a solid writeup. Are you, instead, writing about something that should have been covered in the chatterbox? A node like I only need 23 more xp to reach level 5! is a perfect example. I've probably deleted thirty such nodes. I'll probably have to delete 30 more."
Author's note: Since the time of publication, the "Earn Your Bullshit" node has been deleted. That, in itself, is telling about the dynamic nature of organizational architecture for a site like this.
2.6.5 Interpretation of the Rating System as a Mechanism
Our Web-survey and e-mail interview data combined with write-ups from the site provides a consistent, multi-faceted, picture of the role of the rating system within a larger set of mechanisms for indirect social control. The rating system creates interest from a user perspective. The week following the introduction of the rating system, traffic on the Everything2 site jumped from approximately 10,000 to 20-25,000 users. That 93 percent of users said the rating system improved the site and 90 percent agree that it influences their behavior suggests that it has had a lasting influence. The ability of users to rate each others work, with the prominence of write-ups rising and falling as a result, injects a democratic element into the Everything site. But as the editors point out it is not a pure democracy. Editors alone have the power to delete write-ups and editors are not elected on the basis of experience points, rather they are appointed by the developers. Were Everything2 a democracy, the data suggests the rating system alone might lead to mob rule.
Editors observe that some users attempt to game the system. This may provide a temporary advantage, but the incentive to game the system is checked by the powers of the editors, who have the ability not only to kill write-ups but to extract penalties in experience points as well. The experience point system and the power of editors create a barrier to entry for new users, which is likely to contribute to the site's success. Editors appear to take their powers quite seriously. Asked what makes a good editor, the two highest rated qualities were being helpful to new noders and exercising good judgment over what to kill and what to keep. These qualities were mentioned more frequently than a number of other qualities commonly attributed to editors in the world at large: good writing skills, an eye for what's interesting to the audience, hard work and diplomacy. The tensions of the editors job were reflected by frequent mention of patience as a one of the key qualities of a good editor.
A tension involving voting from a user's perspective also deserves comment. While the modal response of users to the question, "what do you like least about Everything2 or Perl Monks?" was voting (33 percent), users do it anyway. This suggests a perceived benefit to voting that prevails despite what appear to be conflicting emotions. Why? Is it motivated by a civic desire to contribute to the public good of the site? Maybe, maybe not. Ninety-three percent of users suggested that the rating system improves the site. But users also enjoy a private benefit, in the form of experience points for contributing and for voting on other users write-ups.
The theoretical discussion of the free rider problem suggested that one potential solution is to develop mechanisms for indirect control, particularly those that specifically address the nature of the free-rider problem by tying contributions to the public good to private gain. The rating system, quantified by experience points that are connected to tangible outcomes in the form of permissions granted to users as they progress through a system of levels does this explicitly. That users care about experience points is suggested both by their self-reports and observed behavior. Yet the social influence of the rating system appears to persist even after users have progressed to higher levels where experience points cease to be a factor, according to the editor's responses to our survey. The editor's responses suggest a natural human tendency to react to praise reflected by the user community's vote on individual write-ups. In addition, when users were asked how they interact with other users, only 10 percent reported that they don't. The primary means of interaction were the reported as the chatterbox and the /msg feature. The ability to interact directly with other users may serve to mitigate the free-rider problem making future contributions to the public good less anonymous.
2.6.6 Grudin Problem 1 Disparity Between Work and Benefit
While we believe free-rider problem provides a more accurate representation of the situation observed in virtual communities than the Prisoners dilemma, Grudin's first challenge "disparity in work and benefit" deserves a second look. The average time spent per survey respondent per week can be conservatively estimated at more than 8.5 hours. Their primary self-reported benefits were in the form of learning and social interaction (See appendix, User Survey Q. 19).
Both of these activities typically entail the exchange of complementary information. Further incentives are provided through the experience points users gain from posting and the privileges that experience points can buy. The benefit of experience points may also outweigh an aversion to voting on the part of some users. One-third of the user respondents said voting was what they liked least about Everything2 and Perl Monks. But the voting system still appears to work. One explanation is that the perceived benefit of experience points outweighs the cost of voting, including both the time and any psychic cost.
The editors have even higher time costs - an average of more than 17 hours a week on Everything2. When asked what they gained personally from Everything2, editors consistently mentioned three things: better writing skills, friends and knowledge of areas previously unknown. A number of specific responses also seem illuminative:
"…E2 got me writing again. Which kind of snapped me out of a funk, or a paralysis, i'd been in in my life and made me really start evaluating what i'd been doing. So, also, i started meeting people and having friends again (i'd been quite isolated). Sometimes the attention i get on E2 is embarassing and hard to deal with (seems very out of proportion), but i have to admit that the friends i've made have really helped my self-esteem."
"…a home away from home, new friends, lovers, a different sense of community"
"…a greater awareness of how people read. Hypertext has made us all antsy, on the Web and off of it"
Another mentioned that being homeschooled he never had to write any paper and credited Everything2 with success in college English classes. And finally, "I am an all around better writer and have absorbed useless trivia in bulk!"
Return to School of Information Analysis of Everything