Sponsor: National Science Foundation CHS 1764089

PI: Danielle Albers Szafir  Co-PIs: Jed Brubaker, Michael Paul, Casey Fiesler

Abstract: This project focuses on the integration of people and computation in the context of qualitative inductive methods (QIMs), in which experts deeply engage with text corpora such as open-ended surveys, transcribed interviews, or collections of social media content.This engagement can produce insights, but constraints on expertise and time make these methods hard to scale to large datasets. Technologies like machine learning and natural language processing (NLP), which can mine certain kinds of patterns from text data at scales not feasible for even large teams of humans, may offer a way forward; however, machines make mistakes understanding the nuances of language, lack the context and expertise of human analysts, and may fail to detect interesting small-scale patterns necessary to solve particular problems. The goal of this project is to scale up the use of QIMs by inverting traditional models where humans are used to verify computational results ("human-in-the-loop"), starting instead with human insights that can be amplified through computational models, support, and suggestions for analysis ("computer-in-the-loop"). Working with collaborators in domains including mental health, public health, disaster response, policy making, and philanthropy, the team will conduct qualitative studies of their QIM practices and needs, then develop and evaluate systems with the goal of improving both the quality and scale of the insights experts can generate. The project activities will also inform courses on information visualization, human computer interaction, and applied machine learning, along with workshops aimed at recruiting high school women to careers in computing.


Elliott, M., C. Xiong, C. Nothelfer, & D. Albers Szafir. “A Design Space of Vision Science Methods for Visualization Research.” IEEE Transactions on Visualization, 2021 (to appear).

Sarikaya, A., M. Gleicher, & D. Albers Szafir. “Design Factors for Summary Visualization in Visual Analytics.” Computer Graphics Forum, 37(3): 145–156, 2018.

Muthukrishnan, H. & D. Albers Szafir.“Using Machine Learning and Visualization for Qualitative Inductive Analyses of Big Data.” Machine Learning from User Interaction (MLUI) at IEEE VIS 2019, 2019.

Shi, M., D. Albers Szafir, & E. Alexander. “A Survey of Data and Encodings in Word Clouds.” Presented at Digital Humanities, 2020.

Burlinson, D. & D. Albers Szafir. "Shape size judgments are influenced by fill and contour closure.” Presented at the Annual Meeting of the Vision Sciences Society (VSS), 2020.