ISAAC Corpus: An Open-Source Tool for Analyzing Social Group Discourse Across 527 Million Reddit Posts

An Open Infrastructure That Fills a Gap in Social Discourse Research

Studying how social groups are discussed online has long been hampered by data fragmentation — each research team scrapes, cleans, and annotates its own data, making findings difficult to reproduce or compare. The ISAAC (Illinois Social Attitudes Aggregate Corpus) developed by a team at the University of Illinois was built specifically to address this pain point.

According to a newly published paper on arXiv, ISAAC is an open, modular, and accessible English-language corpus containing over 527 million Reddit posts spanning 17 years from 2007 to 2023. The corpus was carefully curated around six key social group dimensions: race, sexual orientation, age, ability (disability), weight, and skin tone — covering nearly every major topic in contemporary social attitudes research.

Rigorous Data Cleaning and Multi-Dimensional Annotation

The value of large-scale data depends on quality. ISAAC employs a multi-step, human-reviewed filtering pipeline that keeps irrelevant content below 10% — both in the overall corpus and in each social group subset. This is a remarkably low noise rate for social media data, and it's central to the corpus's usability.

Equally noteworthy is its annotation system. Every post is algorithmically labeled with the user's estimated geographic region, along with a suite of validated semantic tags, including:

• Moralization: measures the degree of moral judgment in the discourse
• Sentiment: positive or negative attitude
• Emotion: finer-grained emotional classification
• Linguistic generalization: identifies overgeneralizing statements about groups

These labels draw from both off-the-shelf tools and custom models developed by the team, laying the groundwork for deep, cross-dimensional analysis.

Linguistic generalization is a somewhat specialized concept in this annotation framework and deserves a brief explanation. It refers to discourse that treats a social group as a monolithic type — expressions like "all X people are…" or sweeping stereotypes. Such statements are particularly significant in bias and discrimination research: compared to isolated slurs, generalizing claims are often more subtle yet more influential in shaping group attitudes. In NLP, detecting linguistic generalization typically requires going beyond the lexical level to rely on syntactic patterns and semantic reasoning, making it one of the harder annotation tasks to automate. The ISAAC team developed custom models to enable large-scale automatic detection.

Moralization is equally worth highlighting. It measures the tendency in discourse to link a topic or group to moral judgments — right or wrong, good or evil. Moralized discourse often has a more polarizing effect than purely emotional expression, and is an important analytical dimension for cross-cultural and cross-issue comparisons of social attitudes.

Validating the Corpus Through Real-World Trends

Whether a corpus truly reflects reality requires external evidence. The research team used convergent evidence to confirm ISAAC's validity — aligning signals in the corpus with actual social trends:

• Correlation with web search behavior
• Temporal spikes during major social events (both national and regional)
• Long-term shifts in public attitudes

When discourse fluctuations within the corpus align with these independent macro-level indicators, it demonstrates that ISAAC is capturing genuine social attitude dynamics rather than random noise. This validation approach elevates the corpus from a mere "data pile" to a credible research instrument.

Convergent validity is a classic verification method in social science measurement: if a measurement tool strongly aligns with results from other independent sources, it suggests the tool is genuinely measuring the same real construct rather than measurement error or instrument bias. Aligning Reddit discourse data with Google search trends, event timelines, or polling data is exactly this logic in action. This type of validation is especially critical for social media-based corpora — Reddit users do not represent the general population, exhibiting systematic biases in age, education, and geography. Whether the corpus can reflect broader social attitudes must therefore be demonstrated through external evidence, not merely by scale. This step positions ISAAC not just as a "data resource" but as a validated social measurement instrument.

Three Research Capabilities: Temporal, Spatial, and Cross-Category

ISAAC's unified public infrastructure enables research that was previously difficult to conduct at scale:

Cross-category comparison — placing different social group dimensions such as race, age, and weight within the same analytical framework to observe similarities and differences in discourse patterns.

Longitudinal tracking — leveraging 17 years of continuous data to precisely map how discourse about social groups evolves over time, capturing gradual attitude drift or sudden inflection points.

Spatial variation mapping — combining geographic annotations to correlate regional discourse differences with local public opinion and policy outcomes, exploring connections between online discussion and real-world politics.

These three capabilities correspond to the temporal, spatial, and categorical dimensions at the core of social science research — a remarkably comprehensive coverage.

Multiple Access Modes for Different Users

ISAAC's design for accessibility is thoughtful, catering to researchers with varying technical backgrounds:

• No coding required: accessible via a point-and-click website and annotator web app
• Programmatic access: SQL playground, Python toolkit, and datasets on HuggingFace

This means both social science and communication scholars unfamiliar with code, and computational social science researchers who prefer scripted workflows, can each find an appropriate entry point. Meanwhile, its fully open and modular pipeline supports easy extension to new platforms, languages, and social categories, providing reusable infrastructure for future researchers.

Significance and Outlook

ISAAC's core value lies in eliminating research fragmentation and enabling seamless reproducibility. At the intersection of computational social science and natural language processing, reproducibility has long been a persistent weakness. By making all data and processes openly available, ISAAC allows different teams to build on the same foundation — to dialogue, verify, and extend each other's work.

For readers interested in how AI is applied to the social sciences, ISAAC is a telling example: rather than chasing breakthroughs in model performance, it combines NLP annotation technology, large-scale data engineering, and rigorous social science validation methods to build sustainable, reusable public infrastructure. This kind of work may not be flashy, but it often has a more lasting impact on the broader research community.

In computational social science, the replication crisis is not confined to traditional experimental disciplines like psychology — it equally plagues NLP and social computing research that relies on self-collected data. Teams using Reddit data scraped across different time periods, with different filtering standards and annotation models, often produce findings that cannot be verified against one another, creating "data silos." By fixing the data source and pipeline and making the full process public, ISAAC effectively provides a shared benchmark for the field — similar to the role ImageNet once played in computer vision. This infrastructure-level contribution means that findings from different research teams can be stacked, compared, and accumulated within a common coordinate system, and is a key step in moving the field from individual case studies toward systematic knowledge accumulation.