Humanizing Algorithms

From 2018 to 2021, we studied algorithmic interaction design (AIxD): how to design systems that use algorithms and social data in a human-centered way.

Platforms increasingly use algorithms to curate content from data about people’s relationships, behavior, and identities. These algorithms usually optimize for metrics that are easy to calculate, such as likes, plays, and clicks, and they can only work with the social information a person has provided or the system can infer. As a result, they often miss the social context and human meaning that decide whether an interaction feels acceptable.

We looked at where algorithmic and human understandings of social data diverge. Those gaps show what people expect from algorithms, how they make sense of a misfire, and what they do next. Across six studies, we asked how people navigate sensitive algorithmic encounters, how those encounters affect them, and which social concepts algorithms fail to understand. The work included a research partnership with Facebook.

Sensitive Algorithmic Encounters

Upsetting encounters with ex-partners. Through 19 interviews, we studied how people experience algorithmically curated content about ex-romantic partners on Facebook. We found that encounters were made worse by the complex networks and data that create inferred connections around otherwise explicit, and often explicitly ended, relationships. We called this the “social periphery” and show how designing for it can give people agency over inferred as well as explicit connections.
Identity after a break-up. We studied how people handle digital objects and relationships after a break-up. People acted on their possessions and connections to build a post-break-up identity, and existing tools served poorly their competing wishes to present an authentic past and a future self.
Memories and reminiscence. We interviewed 20 people who had recently seen sensitive content through Facebook’s Memories features. They wanted to see “bittersweet” content, but when it was expected, in a context that felt appropriate, and when they could make sense of why the system showed it. We recommend drawing on low-technology artifacts, identifying which contextual features harm users, and studying how users make sense of recommender systems.
Everyday evaluations of the Like button. In interviews with 25 artists who use Instagram, we found three overlapping orientations to the Like: affective, relational, and infrastructural. The button’s ambiguity pushes people toward an economic way of evaluating that crowds out other values, and shapes how artists use the platform and make art.

Algorithmic Infrastructure and Bias

How computers see gender. We analyzed ten commercial facial analysis and image labeling services and evaluated five of them on a dataset of diverse genders, using images labeled by the people who posted them. The services performed worse on transgender people and could not classify non-binary genders. Bias came not only from engineering and training data but from how gender is codified in the data standards that surround the classifiers.
How databases define race and gender. We examined how race and gender are defined and annotated in image databases used for facial analysis. Most databases rarely document the source material behind those definitions or how annotation was done, which makes them opaque and less trustworthy. We offer guidance for database authors.

Funding

This work was supported by the National Science Foundation under CRII: CHS: Humanizing Algorithms: Empirical and Design Investigations of Sensitive Algorithmic Encounters (Award #1756028, 2018 to 2021).

Researchers

Anthony T. Pinter, Jialun “Aaron” Jiang, Katie Z. Gach, Melanie M. Sidwell, James E. Dykes, Morgan Klaus Scheuerman, Jacob M. Paul, Kandrea Wade, Caitlin Lustig, Artie Konrad, Blake Hallinan, Jed Brubaker

Publications

CSCW
Behold the Once and Future Me: Online Identity After the End of a Romantic Relationship Anthony T. Pinter and Jed R. Brubaker
Proceedings of the ACM on Human-Computer Interaction 6, CSCW2: 1–35
- Abstract
- Reference
- BibTeX
- DOI: 10.1145/3555097
After a romantic relationship ends, individuals are left to deal with the digital remnants of the relationship. These possessions and connections pose difficulties for users – they are identity markers of an identity that one may no longer want legible to their online audiences. Further, they can cause upsetting moments that might impede moving on from the break-up. Through interviews with 11 women who had had a recent break-up, this empirical study examined how people managed their online identity after their break-up. We found that people took different actions towards their possessions and connections in service of creating a post-break-up identity. Using Brubaker & Hayes’s (2011) representational framework, we find users attempting to deal with connections as if they were possessions, creating tensions that our current systems are ill-suited to address. Turning to Hogan’s (2010) framework of the online identity ’exhibition’, we see users creating exhibitions for an audience of one – themselves – while also making decisions about who is allowed to see their public-facing exhibitions. We conclude by arguing that existing tools are ill-designed to support competing desires to present authentic past and future online identities and offer design suggestions for consideration.
CHI
Best Paper
Designing for the Bittersweet: Improving Sensitive Experiences with Recommender Systems Caitlin Lustig, Artie Konrad, and Jed R. Brubaker
CHI Conference on Human Factors in Computing Systems (CHI 2022) Article 16
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3491102.3502049
It is difficult to design systems that honor the complex and often contradictory emotions that can be surfaced by sensitive encounters with recommender systems. To explore the design and ethical considerations in this space, we interviewed 20 people who had recently seen sensitive content through Facebook’s Memories feature. Interviewees typically described how (1) expectedness, (2) context of viewing, and (3) what we describe as “affective sense-making” were important factors for how they perceived “bittersweet” content, a sensitizing concept from our interviews that we expand upon. To address these user needs, we pose provocations to support critical work in this area and we suggest that researchers and designers: (1) draw inspiration from no/low-technology artifacts, (2) use empirical research to identify contextual features that have negative impacts on users, and (3) conduct user studies on affective sense-making. CAUTION: This paper discusses difficult subject matter related to death and relationships.
Journal
Living With Everyday Evaluations on Social Media Platforms Blake Hallinan and Jed R. Brubaker
International Journal of Communication 15 19
- Abstract
- PDF
- Reference
- BibTeX
The Like button is simultaneously a means of social interaction and a tool to evaluate social media content. Likes are built into the infrastructure of platforms, fueling personalized searches, recommendations, and targeted advertisements while contributing to the economic valuation of platforms and users alike. Working at the intersection of infrastructure and practice, we analyze the experience of everyday evaluations on social media to understand how platforms shape what people care about. Based on in-depth interviews with 25 artists who use Instagram, we identify three overlapping orientations to the Like button: affective, relational, and infrastructural. We show how the flexibility of the button creates ambiguity around the meaning of a Like which drives practices of metaevaluation, and argue that the platform architecture incentivizes an economic approach to evaluation that crowds out other value schemas, shaping how artists use the platform, make art, and even understand themselves. We conclude with a discussion of the broader implications of everyday evaluations, algorithms, and platforms.
CSCW
Best Paper Diversity and Inclusion Recognition
How We’ve Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis Morgan Klaus Scheuerman, Kandrea Wade, Caitlin Lustig, and Jed R. Brubaker
Proceedings of the ACM on Human-Computer Interaction 4, CSCW1: Article 58
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3392866
Race and gender have long sociopolitical histories of classification in technical infrastructures-from the passport to social media. Facial analysis technologies are particularly pertinent to understanding how identity is operationalized in new technical systems. What facial analysis technologies can do is determined by the data available to train and evaluate them with. In this study, we specifically focus on this data by examining how race and gender are defined and annotated in image databases used for facial analysis. We found that the majority of image databases rarely contain underlying source material for how those identities are defined. Further, when they are annotated with race and gender information, database authors rarely describe the process of annotation. Instead, classifications of race and gender are portrayed as insignificant, indisputable, and apolitical. We discuss the limitations of these approaches given the sociohistorical nature of race and gender. We posit that the lack of critical engagement with this nature renders databases opaque and less trustworthy. We conclude by encouraging database authors to address both the histories of classification inherently embedded into race and gender, as well as their positionality in embedding such classifications.
CSCW
Best Paper
How Computers See Gender: An Evaluation of Gender Classification in Commercial Facial Analysis and Image Labeling Services Morgan Klaus Scheuerman, Jacob M Paul, and Jed R. Brubaker
Proceedings of the ACM on Human-Computer Interaction 3, CSCW: Article 144
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3359246
Investigations of facial analysis (FA) technologies-such as facial detection and facial recognition-have been central to discussions about Artificial Intelligence’s (AI) impact on human beings. Research on automatic gender recognition, the classification of gender by FA technologies, has raised potential concerns around issues of racial and gender bias. In this study, we augment past work with empirical data by conducting a systematic analysis of how gender classification and gender labeling in computer vision services operate when faced with gender diversity. We sought to understand how gender is concretely conceptualized and encoded into commercial facial analysis and image labeling technologies available today. We then conducted a two-phase study: (1) a system analysis of ten commercial FA and image labeling services and (2) an evaluation of five services using a custom dataset of diverse genders using self-labeled Instagram images. Our analysis highlights how gender is codified into both classifiers and data standards. We found that FA services performed consistently worse on transgender individuals and were universally unable to classify non-binary genders. In contrast, image labeling often presented multiple gendered concepts. We also found that user perceptions about gender performance and identity contradict the way gender performance is encoded into the computer vision infrastructure. We discuss our findings from three perspectives of gender identity (self-identity, gender performativity, and demographic identity) and how these perspectives interact across three layers: the classification infrastructure, the third-party applications that make use of that infrastructure, and the individuals who interact with that software. We employ Bowker and Star’s concepts of "torque" and "residuality" to further discuss the social implications of gender classification. We conclude by outlining opportunities for creating more inclusive classification infrastructures and datasets, as well as with implications for policy.
CSCW
"Am I Never Going to Be Free of All This Crap?": Upsetting Encounters With Algorithmically Curated Content About Ex-Partners Anthony T. Pinter, Jialun "Aaron" Jiang, Katie Z. Gach, Melanie M. Sidwell, James E. Dykes, and Jed R. Brubaker
Proceedings of the ACM on Human-Computer Interaction 3, CSCW: Article 70
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3359172
Every day on social media, people see streams of content curated by algorithms that leverage their relationships, preferences, and identities. However, algorithms can oversimplify the complexity of people’s social contexts. Consequently, algorithms can present content to people in ways that are insensitive to their circumstances. Through 19 in-depth interviews, our empirical study examines instances of contextually insensitive content through the lens of people’s upsetting encounters with content about their ex-romantic partners on Facebook. We characterize the encounters our participants had with content about their exes, including where on Facebook it occurred, the types of social connections involved in the content, and participants’ perceptions of why the content appeared. Based on our findings, we describe the "social periphery"—the complex social networks and data that enable inferred connections around otherwise explicit relationships—and discuss the design challenges that the periphery presents designers.
CSCWWorkshop
The Changing Contours of “Participation” in Data-driven, Algorithmic Ecosystems: Challenges, Tactics, and an Agenda Christine T. Wolf, Haiyi Zhu, Julia Bullard, Min Kyung Lee, and Jed R. Brubaker
ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2018), Companion
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3272973.3273005
Contributing to a growing attention to algorithms and algorithmic interaction in the CHI and CSCW communities, this workshop aims to deal centrally with the topic of human “participation” and its changing role to data-driven, algorithmic ecosystems. Such a focus includes projects that involve users in the design of algorithms and “human-in-the-loop” systems, broader investigations into the ways in which “participation” is situated in data-driven activities, as well as conceptual concerns about participation’s changing contours in contemporary social computing landscapes. This one-day workshop will be led by academic and industry researchers and sets out to achieve three goals: identify cases and ongoing projects on the topic of participation in algorithmic ecosystems; create a tactical toolkit of key challenges and strategies in this space; and set a forward-facing agenda to provoke further attention to the changing role of participation in contemporary sociotechnical systems.