Guardrails in Communication Networks

Program

Abstracts

"Talking About Politics When Everything is Political", by Jaime Settle

Do people make their divisions deeper when they interact with each other about politics? This question has long motivated scholars of political psychology and communication, but it has become all the more pressing in an era defined by polarization, hyperpartisanship, and the politicization of many facets of society. In this research presentation, I theorize more fully about the nature of organic political interactions, both online and offline, and the implications of the underlying psychology of communication for the public’s willingness to engage about substantive political topics. Integrating insights from research using a diverse of methods, ranging from psychophysiological measurement to computational social science, we will unpack more realistic expectations about when political interaction might exacerbate our divides and when it might ameliorate them.

"Content Removal vs. Algorithmic Promotion: Exploring Their Combined Effect on User Exposure", by Laura Edelson

Content moderation and algorithmic feed recommendation are the two platform systems with the most direct control over what users see. Yet research and policy discussions almost always treat them in isolation — moderation scholarship measures removal rates and response times, while recommendation system research focuses on ranking objectives and engagement optimization. This separation obscures a basic question: what is the net effect of these two systems operating simultaneously, sometimes in tension with one another? In this talk, I draw on two empirical investigations into each side of this question. First, a measurement study of content removal on Facebook introduces the metric of prevented dissemination and finds that removals prevent only 24–30% of posts' predicted engagement, revealing the limits of moderation when engagement accrues faster than review. Second, a comparative survey of feed algorithm designs across six major platforms identifies key design dimensions — including usage intensity optimization, content timeliness, and inventory source selection — that create structurally different conditions for moderation to operate in. Building on these findings, I propose new directions for measuring the relationship between these two systems proportionally: How much of a user's exposure to harmful content is shaped by what the algorithm chose to promote versus what moderation failed to catch in time? What algorithm design traits make moderation more or less effective? I will outline approaches for connecting these measurement frameworks and discuss what such an integrated view could mean for platform safety research and policy.

"Age and Gender Distortion in Online Media and Large Language Models", by Douglas Guilbeault

Are widespread stereotypes accurate or socially distorted? This continuing debate is limited by the lack of large-scale multimodal data on stereotypical associations and the inability to compare these to ground truth indicators. In this talk, I will present our recent work in which we address this challenge in the analysis of age-related gender bias, for which age provides an objective anchor for evaluating stereotype accuracy. Despite there being no systematic age differences between women and men in the workforce according to the US Census, we find that women are represented as younger than men across occupations and social roles in nearly 1.4 million images and videos from Google, Wikipedia, IMDb, Flickr and YouTube, as well as in nine language models trained on billions of words from the internet. This age gap is starkest for content depicting occupations with higher status and earnings. We further show how mainstream algorithms amplify this bias. A nationally representative pre-registered experiment (n = 459) finds that Googling images of occupations amplifies age-related gender bias in participants’ beliefs and hiring preferences. We additionally show that when generating and evaluating resumes, ChatGPT assumes that women are younger and less experienced, rating older male applicants as higher quality. I conclude by discussing ongoing work that builds on this computational paradigm to show how we can leverage large-scale social data and artificial intelligence to discover novel dimensions of stereotypes that are predictive of human psychology.

"AI Writing Tools Can Improve Democratic Outcomes in Online Discussions", by Ethan Busby

How can advances in generative AI be used to improve the outcomes of online disagreement? We draw from robust existing research on procedural respect in conversations and value translation to build common ground to design AI-powered interventions for semi-synchronous interpersonal discussions. We also design and test multiple variations of the interface of the AI intervention to examine how interface impacts ease of use and author agency. Using a randomized experiment, we find that AI tools can improve political disagreements, but that there are important tradeoffs. The outcomes that produce the largest improvements in democratic reciprocity for the conversation partners are those that require the most effort and have the least benefit for tool users themselves. We end with a discussion of the implications of these results for future implementations of AI tools in political settings, including what this means for researchers, practitioners, and deliberative democracy broadly.

"How public involvement improves the science of AI", by J. Nathan Matias

As AI systems from decision-making algorithms to generative AI are deployed more widely, computer scientists and social scientists alike are being called on to provide trustworthy quantitative evaluations of AI safety and reliability. These calls have included demands from affected parties to be given a seat at the table of AI evaluation. What, if anything, can public involvement add to the science of AI? This talk, building from a recent article in PNAS, summarizes over a decade of experience in participatory science on AI and society. Starting with the the sociotechnical challenge of evaluating AI systems, it will review common models of engagement in AI research and address common concerns about the scientific viability of participatory methods. Through a series of case studies, the talk suggests five parts of any quantitative AI evaluation where participatory science can improve the science of AI: equipoise, explanation, measurement, inference, and interpretation. It concludes with reflections on the role that participatory science can play in trustworthy AI by supporting trustworthy science.

"Banned Communities Grow through a Distinctive Pattern of User Engagement and Networked Coordination", by Yijing Chen

Research on online content moderation overwhelmingly examines what happens after platforms intervene (e.g., subreddit bans or quarantines), with far less attention being devoted to the processes that unfold before such interventions occur. To address this gap, we analyze the full histories of 4,237 subreddits banned in 2020 using the Pushshift Reddit dataset, and compare them with a matched sample of unbanned subreddits with similar lifespans and popularity. Our results show that subreddits banned in 2020 follow systematically different trajectories from comparable baselines well before their bans. At the macro level, banned subreddits accumulate substantially more users, submissions, and comments. At the micro level, they attract more committed users who join earlier in their platform lifetime and remain active for longer durations. At the meso level, they are more structurally embedded in the subreddit ecosystem: they occupy more central positions in co-posting, referencing, and activity-flow networks, with greater user overlap, broader cross-subreddit references, and higher volumes of activity flows. Furthermore, we show that subreddit growth is more a product of network-mediated participation than atomistic individual decisions, a pattern that exists in both banned and matched subreddits but is more salient in banned ones. User entries into banned subreddits are more temporally clustered among users with prior shared community affiliations, and activity flows into banned subreddits are more strongly associated with references to them in other subreddits. Overall, our findings highlight warning signals of moderation-relevant risks, and reframe high-risk communities not as isolated cases of rule violations, but as products of networked engagement that warrant an ecosystem-level perspective.

"Unpacking How Context (Conversation History) Shifts the Framing of Large Language Model Outputs", by Vishwanath E.V.S.

One of the earliest and most prominent use cases proposed for Large Language Models (LLMs) is their potential to serve as search engines. Proponents have argued that querying LLMs would be akin to conversing with "domain experts" who would return comprehensive answers, instead of a ranked list of sources containing relevant information. However, new search and information seeking systems are not free of old problems, and in the case of LLM-powered search engines, the risk of creating or reinforcing echo chambers re-emerges through the new affordances and information retrieval mechanisms intrinsic to the algorithmic operation of these models . The existence of LLM-driven echo chambers is still an emerging area of research. Recent work in this area finds that the conversational design of LLMs can encourage users to rely on confirmatory queries. This project aims to study whether LLMs generate responses aligned with users' political priors through an in-silico study evaluating how LLM responses to the same questions vary as a function of the ideological slant of the language used in prior conversational exchanges.

"These Aren't the Guardrails You’re Looking For: Source-Based Signals for Healthier Online Platforms", by Mor Naaman

A trustworthy information ecosystem is essential for democracy. Today’s information platforms lack a critical guardrail: the ability to evaluate and communicate source trustworthiness. Without it, content moderation becomes an endless game of whack-a-mole. In this speculative talk, I will outline a framework that goes beyond item-based moderation by enabling systematic evaluation of source trustworthiness. The potential outcomes include: informing users, shaping creator behavior, improving ranking signals, and reducing the need for arbitrary platform actions. I will outline the concepts behind this "magical" framework. I will then admit that it is largely theoretical, and hard to study without platform control. And yet: I will show hints of the framework's potential demonstrated via a deployed Bluesky labeler that surfaces transparent, objective indicators of account behavior on that platform. .

"Between You and the Facts", by Matthew DeVerna

What happens when machines mediate how we find information? Large language models now sit between millions of users and the information they seek, and their impressive performance has raised hopes for automated end-to-end fact-checking—but well-known shortcomings raise a harder question about what gets lost when search becomes synthesis. In this talk, I present a systematic audit of 15 recent LLMs from OpenAI, Google, Meta, and DeepSeek across more than 6,000 PolitiFact claims, comparing standard models with reasoning- and web-search-enabled variants. Off-the-shelf performance is poor; reasoning offers minimal lift; and web search produces only moderate gains—even though the relevant fact-checks are publicly indexed. Providing models with a custom retrieval-augmented generation (RAG) pipeline built on curated PolitiFact summaries, however, improves macro F1 by 233% on average. I will use these results to argue that, as LLMs become default mediators through which people encounter information, the field urgently needs sustained, large-scale auditing of mainstream AI systems to understand how they are reshaping the information environment at scale.

"How information abundance, news negativity, and media distrust are changing the way Americans engage with journalism", by Ariel Hasell

As Americans increasingly turn to social media for news, they face political information environments that are overwhelmingly crowded, emotional, and lack clear epistemic hierarchies of informational sources. This has wide reaching consequences for politics when it comes to issues like misinformation and polarization, but it is also changing public perceptions of what news media are and challenging traditional understanding of the role of news media in democracies. Using panel survey data from 2024, this talk explores how digital information environments can encourage news distrust and disengagement from news and politics, but also how alternative voices in these environments, like social media influencers, may play a role in encouraging more engagement with news and politics. Together, these studies highlight how social media are shaping how Americans define and consume news media.

"Longitudinal Monitoring of LLM Content Moderation of Social Issues", by Emma Lurie

Large language models' (LLMs') outputs are shaped by opaque and frequently-changing company content moderation policies and practices. LLM moderation often takes the form of refusal; models' refusal to produce text about certain topics both reflects company policy and subtly shapes public discourse. We introduce AI Watchman, a longitudinal auditing system to publicly measure and track LLM refusals over time, to provide transparency into an important and black-box aspect of LLMs. Using a dataset of over 400 social issues, we audit Open AI's moderation endpoint, GPT-4.1, and GPT-5, and DeepSeek (both in English and Chinese). We find evidence that changes in company policies, even those not publicly announced, can be detected by AI Watchman, and identify company- and model-specific differences in content moderation. We also qualitatively analyze and categorize different forms of refusal. This work contributes evidence for the value of longitudinal auditing of LLMs, and AI Watchman, one system for doing so.

"Motivated moderation: How partisan alignment influences civility judgments in community-based meme forums", by Rehan Mirza

Content moderation requires moderators to balance harm prevention with free expression, a tension that is especially acute for political incivility, where norm violations must be weighed against salient political speech. Prior work suggests that moderation decisions may be distorted by motivated partisan reasoning, yet evidence from community-driven settings, where users create and enforce rules, remains limited. This study examines whether political alignment biases moderation judgments in community settings. Using a 2x3 within-subjects survey experiment simulating a moderator queue, U.S.-based participants evaluated social media memes varying in political alignment (aligned vs. opposed) and civility (uncivil, borderline uncivil, civil), applying explicit community rules to select enforcement actions with specified consequences. Outcomes were measured across two dimensions: violation recognition (whether content should be censored) and enforcement severity (the level of action taken). Using memes with political messaging overlaid on cartoon images, I test whether alignment effects persist in humorous, seemingly low-stakes contexts where political salience may be reduced. Focusing on civility rather than fact-based violations like misinformation, the study isolates motivated reasoning mechanisms, including “party promotion” and “preference gaps”. Results show that increased incivility and partisan opposition significantly increase violation recognition and enforcement severity, with no interaction between the two. This indicates that partisan bias operates independently of interpretative ambiguity. Political content is systematically judged more stringently than non-political content, suggesting it may arouse negative affect. This study carries direct implications for platform oversight at a time when platforms are increasingly delegating moderation duties to volunteer communities.

Logistics