CASTLE Seminar: Deductive qualitative analysis of text-based data at scale using text embeddings

CASTLE Seminar
Deductive qualitative analysis of text-based data at scale using text embeddings

Markus Kruetzer, Jonas Mjaaland, and Halvor Tyseng
Center for Computing in Science Education
University of Oslo

Abstract:
Qualitative analysis is a cornerstone of social science research, used in fields as diverse as education research, sociology, history, and business. Traditional qualitative analysis, however, is time consuming, labor intensive, and difficult to scale and replicate. To tackle these issues, we have devised a fast, replicable, and scalable technique for deductive, qualitative research on text-data, by utilizing novel advances in machine learning and artificial intelligence. To prove the concept, we thematically analyze 23 years of the Physics Education Research Conference proceedings literature. First, using a natural language processing technique known as embedding, we create vector-based representations of texts in a high-dimensional meaning space. Then we define topics, derived from a review of the physics education research literature by Docktor and Mestre (2014). To project these topics into the meaning space, we define them as centroids of representative domain-expert determined articles’ embedding vector. Thereafter, we calculate the distances between every article and centroid, and use the inverted, scaled distances between these centroids and articles to create a topic model. Based on this analysis, we critically discuss the features, uses, and limitations of this method and argue that it holds promise for flexible deductive qualitative analysis of a wide variety of text-based data that avoids many of the drawbacks inherent to prior NLP methods.

Bio:
Markus, Jonas, and Halvor are completing their Master’s degrees at the University of Oslo and are working under the supervision of Dr. Tor Odden in the Center for Computing in Science Education at the University of Oslo.

Intended Audience:
All are Welcome!

To request an interpreter, please visit myaccess.rit.edu

About CASTLE: The Center for Advancing Scholarship to Transform Learning, or CASTLE, is a network of faculty, projects and programs engaged in scholarship surrounding STEM education.