Identifying Latent Textual Bias: Making a case for traditional NLP tasks in the era of AI

Bias detection remains an area of interest for digital humanists, computational linguists, and information studies scholars, who point to biases inherent in our algorithms, software, tools, and platforms, but we are only just beginning to examine how computational methods could be used to interrogate our primary textual sources. This project presents a method for bias detection that can be used at a study’s outset with little initial knowledge of the corpus, requires little pre-processing, and is both beginner-friendly and language-agnostic. Pairing topic modeling with sentiment analysis and targeted close reading of documents most closely related to topics of interest (based on document-topic weights) uncovered the stories of lesser-known actors in the history of Ottoman Algeria, as well as biases inherent in the writing of their histories. The anti-Arab and/or anti-Turkish sentiments one might expect to observe in French colonial texts were absent, but a latent anti-Semitic sentiment appeared in the topic models, indicated by sentiment analysis scores of topics related to Jewish people and verified through a close reading of related passages. 

Ashley R. Sanders is a multidisciplinary data scientist. She served as Vice Chair of the Digital Humanities Program at UCLA for more than six years and, prior to that, as a faculty member and Director of the Digital Research Studio at the Claremont Colleges. She holds a Ph.D. in History with a specialization in Digital Humanities and a B.S. in Mathematics and History. Her first book, Visualizing History’s Fragments: A Computational Approach to Humanistic Research is now available from Palgrave. Additional publications include “Silent No More: Women as Significant Political Intermediaries in Ottoman Algeria” (Current Research in Digital History, 2020), “Building a DIY Community of Practice,” in People, Practice, Power: Digital Humanities outside the Center (December 2021), and a maturity framework for DH centers. She has also been awarded a large National Endowment for the Humanities grant in collaboration with Co-PI, Jessica Otis (George Mason University) for their project entitled, “The Mathematical Humanists.” The grant funds a series of in-person, online, and asynchronous professional development workshops on statistics, graphs and networks, linear algebra, and discrete mathematics methods that inform computational humanistic research methods.