Moderation And Rules in Online Communities

Effective regulation is one of the most important factors that make online communities successful. While communities are certainly governed by platform policies such as user agreements and content policies, the majority of the moderation work is accomplished by volunteer moderators who enforce rules created by their own communities. In this project, we deeply investigate the moderation ecosystem of online communities—What are the existing rules? How are these rules created and enforced? How do the moderation teams work behind the scene? And how does the moderation ecosystem expand across multiple communities and platforms?

Our prior work includes a mixed-methods study of 100,000 subreddits and their rules. Our findings characterize the types of rules across Reddit, the frequency of rules at scale, and patterns of rules based on subreddit characteristics. We find that rules appear to be context-dependent for individual subreddits but also share common characteristics across the site. Taken together, our findings provide a rich description of this ecosystem of rules, motivating further inquiry into underlying mechanisms for rule formation and enforcement in online communities.

Publications

TOCHI
A Trade-off-centered Framework of Content Moderation Jialun "Aaron" Jiang, Peipei Nie, Jed R. Brubaker, and Casey Fiesler
ACM Transactions on Computer-Human Interaction 30, 1: 1–34
- Abstract
- Reference
- BibTeX
- DOI: 10.1145/3534929
Content moderation research typically prioritizes representing and addressing challenges for one group of stakeholders or communities in one type of context. While taking a focused approach is reasonable or even favorable for empirical case studies, it does not address how content moderation works in multiple contexts. Through a systematic literature review of 86 content moderation articles that document empirical studies, we seek to uncover patterns and tensions within past content moderation research. We find that content moderation can be characterized as a series of tradeoffs around moderation actions, styles, philosophies, and values. We discuss how facilitating cooperation and preventing abuse, two key elements in Grimmelmann’s definition of moderation, are inherently dialectical in practice. We close by showing how researchers, designers, and moderators can use our framework of tradeoffs in their own work, and arguing that tradeoffs should be of central importance in investigating and designing content moderation.
CSCWPoster
Characterizing Community Guidelines on Social Media Platforms Jialun "Aaron" Jiang, Skyler Middler, Jed R. Brubaker, and Casey Fiesler
ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2020), Companion
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3406865.3418312
Social media platforms use community guidelines to enact governance and moderate content, but the limitation in their moderation capacity forces them to choose the types of misbehavior they focus more on. In this work, we analyze these choices through a content analysis of the community guidelines of 11 major social media platforms. We find 66 different types of rules across their community guidelines, with great variability in the coverage of these rules across different platforms. Our research reveals the types of misbehavior that platforms chose to focus on, and motivates further inquiries into policymaking and content moderation in specific problem areas such as inciting violence and voter suppression.
Doctoral Consortium
Identifying and Addressing Design and Policy Challenges in Online Content Moderation Jialun "Aaron" Jiang
CHI 2020 Doctoral Consortium
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3334480.3375030
My doctoral research identifies and addresses the design and policy challenges of content moderation on the levels of smaller communities and larger platforms, using a combination of interviews and surveys. My prior work studies voice-based communities on Discord, and shows that voice as a new technology brings unique challenges to community moderators and highlights the fundamental evidentiary need in content moderation. My dissertation expands on this work by (1) quantifying severity of violations as a way to address social media platforms’ challenge of prioritization, and (2) empirically evaluating existing moderation approaches and design recommendations by soliciting moderator and user perceptions of them. Combining these perspectives, my dissertation will provide guidelines for improving content moderation in online communities.
CSCW
Best Paper Honorable Mention
Conformity of Eating Disorders through Content Moderation Jessica L Feuston, Alex S Taylor, and Anne Marie Piper
Proceedings of the ACM on Human-Computer Interaction 4, CSCW: 1–28
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3392845
For individuals with mental illness, social media platforms are considered spaces for sharing and connection. However, not all expressions of mental illness are treated equally on these platforms. Different aggregates of human and technical control are used to report and ban content, accounts, and communities. Through two years of digital ethnography, including online observation and interviews, with people with eating disorders, we examine the experience of content moderation. We use a constructivist grounded theory approach to analysis that shows how practices of moderation across different platforms have particular consequences for members of marginalized groups, who are pressured to conform and compelled to resist. Above all, we argue that platform moderation is enmeshed with wider processes of conformity to specific versions of mental illness. Practices of moderation reassert certain bodies and experiences as ’normal’ and valued, while rejecting others. At the same time, navigating and resisting these normative pressures further inscribes the marginal status of certain individuals. We discuss changes to the ways that platforms handle content related to eating disorders by drawing on the concept of multiplicity to inform design.
CSCW
Technological Frames and User Innovation: Evidence from Online Community Moderation Teams Charles Kiene, Jialun "Aaron" Jiang, and Benjamin Mako Hill
Proceedings of the ACM on Human-Computer Interaction 3, CSCW: Article 44
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3359146
Management of technological change in organizations is one of the most enduring topics in the literature on computer-supported cooperative work. The successful navigation of technological change is both more challenging and more critical in online communities that are entirely mediated by technology than it is in traditional organizations. This paper presents an analysis of 14 in-depth interviews with moderators of subcommunities of one technological platform (Reddit) that added communities on a new technological platform (Discord). Moderation teams experienced several problems related to moderating content at scale as well as a disconnect between the affordances of Discord and their assumptions based on their experiences on Reddit. We found that moderation teams used Discord’s API to create scripts and bots that augmented Discord to make the platform work more like tools on Reddit. These tools were particularly important in communities struggling with scale. Our findings suggest that increasingly widespread end user programming allow users of social computing systems to innovate and deploy solutions to unanticipated design problems by transforming new technological platforms to align with their past expectations.
CSCWWorkshop
Volunteer Work: Mapping the Future of Moderation Research C. Kiene, S. Jhaver, E. Chandrasekharan, B. Dosono, B. Dym, S. Gilbert, J.A. Jiang, K. Lo, J. Seering, K. Shores, and D.Y. Wohn
Workshop Organized at CSCW 2019
- Abstract
- PDF
- Reference
- BibTeX
Research on the governance of online communities often requires exchanges and interactions between researchers and moderators. While a growing body of work has studied commercial content moderation in the context of platform governance and policy enforcement, only a small number of studies have begun to explore the work of unpaid, volunteer community moderators who manage the millions of different subcommunities that exist on platforms. This workshop will create a pathway for future scholars to tackle the challenges and opportunities of research on volunteer community moderators and establish best practices for engaging with volunteer moderators without disrupting their work. Through lightning talks, collaborative brainstorming exercises, and small-group activities applying principles to research practice, workshop participants will bring together their diverse experiences and perspectives to map the future of moderation research. Both industry and academic researchers as well as experienced moderators will lead this one-day workshop that may accommodate up to 20 participants.
CSCW
Moderation Challenges in Voice-based Online Communities on Discord Jialun "Aaron" Jiang, Charles Kiene, Skyler Middler, Jed R. Brubaker, and Casey Fiesler
Proceedings of the ACM on Human-Computer Interaction 3, CSCW: Article 55
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3359157
Online community moderators are on the front lines of combating problems like hate speech and harassment, but new modes of interaction can introduce unexpected challenges. In this paper, we consider moderation practices and challenges in the context of real-time, voice-based communication through 25 in-depth interviews with moderators on Discord. Our findings suggest that the affordances of voice-based online communities change what it means to moderate content and interactions. Not only are there new ways to break rules that moderators of text-based communities find unfamiliar, such as disruptive noise and voice raiding, but acquiring evidence of rule-breaking behaviors is also more difficult due to the ephemerality of real-time voice. While moderators have developed new moderation strategies, these strategies are limited and often based on hearsay and first impressions, resulting in problems ranging from unsuccessful moderation to false accusations. Based on these findings, we discuss how voice communication complicates current understandings and assumptions about moderation, and outline ways that platform designers and administrators can design technology to facilitate moderation.
ICWSM
Reddit Rules! Characterizing an Ecosystem of Governance Casey Fiesler, Jialun "Aaron" Jiang, Joshua McCann, Kyle Frye, and Jed Brubaker
International AAAI Conference on Web and Social Media (ICWSM 2018)
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1609/icwsm.v12i1.15033
The social sharing and news aggregation site Reddit provides a unique example of an ecosystem of community-created rules. Not only do individual subreddits create and enforce their own regulations, but site-wide guidelines and norms may also influence behavior. This paper reports on a mixed-methods study of 100,000 subreddits and their rules. Our findings characterize the types of rules across Reddit, the frequency of rules at scale, and patterns of rules based on subreddit characteristics. We find that rules appear to be context-dependent for individual subreddits but also share common characteristics across the site. Taken together, our findings provide a rich description of this ecosystem of rules, motivating further inquiry into underlying mechanisms for rule formation and enforcement in online communities.