RIT professor rethinks AI’s role in qualitative research

Qualitative researchers face a daunting task.

A single study can generate hundreds of pages of interview transcripts. Then, researchers must sift through mountains of information to identify meaningful patterns and draw connections. And all without overlooking important details or allowing personal biases to shape conclusions.

Ha-Kyung Hidy Kong, an assistant professor at Rochester Institute of Technology, is exploring how artificial intelligence (AI) and data visualization can be used to help researchers analyze qualitative data. The goal is to integrate multimodal data and human-in-the-loop machine learning techniques to promote and support a transparent and trustworthy qualitative data analysis process.

During qualitative studies, researchers collect and analyze non-numerical data to understand people’s interactions, opinions, and experiences. These types of studies are prevalent in academia and industry, informing fields ranging from healthcare to public policy.

When conducting interviews, many researchers gather audio and video data, in addition to transcripts. Kong began by asking qualitative researchers if they were using that multimodal data. They told her, “Not really,” because it’s too time-consuming to locate that information based on the transcript.

“There is untapped potential here with the multimodal data,” said Kong, who teaches in RIT’s School of Information. “Tone of voice, gestures, and facial expressions can all provide context and nuance that words alone do not capture.”

Kong’s National Science Foundation-funded research led to ChromaScribe, a web tool that supports researchers in analyzing interview data. It combines optional AI assistance with a color-coded visualization panel that is linked to transcripts and audio files. Researchers can compare passages, check AI suggestions, and revise labels as they develop their own interpretations.

As technology advances, the tool could find meaningful moments in video or audio files and connect them with the transcript, such as extracting all the points where people are laughing. The ChromaScribe tool is free and open source.

Kong explained that linking multimodal data to an interview transcript could also be used to address confirmation bias.

“A participant might answer a question sarcastically or not enthusiastically,” said Kong. “But if I don’t have that audio or video data, I could interpret the text in a way suits my agenda or preconceptions.”

How AI can support, not replace

While developing her qualitative analysis tool, Kong discovered an open letter from more than 400 qualitative researchers. The letter’s title was “We Reject the Use of Generative Artificial Intelligence for Reflexive Qualitative Research.”

Some researchers worry that AI-generated codes and themes could introduce anchoring bias, oversimplify complex experiences, and undermine a researcher’s ownership of the findings. Others have raised concerns about what happens to sensitive research data when it is processed by cloud-based AI systems.

Those concerns inspired Kong’s second project, Reflexis, a free and open-source tool developed with researchers at the University of Toronto. The work was published at the 2026 ACM CHI Conference on Human Factors in Computing Systems, the premier conference in human-computer interaction (HCI).

‌

Carlos Ortiz/RIT

Ha-Kyung Hidy Kong, assistant professor in RIT’s School of Information, received a National Science Foundation grant to create open-source AI tools that can assist qualitative researchers with coding and reflection.

“I think it’s a change of thinking,” said Kong. “Rather than having AI replace what is currently being done, how can AI augment the current workflow and support users?”

Rather than asking AI to interpret data for researchers, Reflexis is designed to support their reflection. The tool offers prompts to help researchers examine their thinking, tracks how their interpretations change over time, and helps find points for discussion in thematic analysis.

Ester Chen, a Ph.D. student in computing and information sciences, joined RIT in 2024. She conducts research with Kong, focusing on HCI, and is an AWARE-AI trainee in the 2026-2027 cohort.

“As a graduate researcher who often uses qualitative methods, I was drawn to the practical challenge behind this project,” said Chen. “Qualitative analysis takes time and effort, especially when switching between audio recordings and written transcripts. It also involves careful reflection and collaboration. I wanted to explore how AI could support that workflow.”.

The findings suggest that researchers are not necessarily opposed to AI itself. During a comparative study between ChromaScribe and an established qualitative data analysis tool, 27 out of 28 participants chose to use the available AI feature for qualitative analysis. Only one person chose to do everything manually.

However, researchers differed in how much automation they were comfortable with, underscoring the importance of giving users control in the process. Some researchers wanted to read and label interview passages themselves before using AI and one participant reported that AI-generated labels influenced their thinking in an unexpected way.

“Researchers should be able to decide whether and when to use AI assistance,” said Chen. “When they do use it, they need to be able to check the suggestions and decide which to accept, change, or reject.”

Privacy is another consideration. Kong’s team used local AI models in its research on automated coding, allowing data to remain on the user’s machine rather than being transferred to a third-party service.

In the long term, Kong aims to make qualitative analysis more efficient, transparent, and trustworthy. With carefully designed AI assistance, she hopes that researchers can utilize larger amounts of data without losing the human perspective that makes qualitative research valuable.

RIT’s computing college is a top school for human-computer interaction (HCI) education and research, offering a bachelor’s degree, HCI master’s program, degrees in AI, and the opportunity to conduct research in the Center for Accessibility and Inclusion Research.