CASTLE Seminar Series -

Assistant Professor of in the Department of Integrative Biology, University of South FloridaTitle: Assessing student writing in biology using lexical analysis and machine learning:Abstract: Writing is an essential scientific practice, yet increasing university science class-size limits the use of written assessments. Written assessments allow students to demonstrate their learning in their own words and permit faculty to evaluate students_ understanding. However, the time and resources required to grade written assessments prohibit their use in large-enrollment science courses. This seminar will discuss the use of lexical analysis and machine learning algorithms to automatically analyze student writing and provide timely feedback to faculty about students writing in biology. Written responses to questions about matter and energy transformation were collected from large-enrollment undergraduate introductory biology classrooms. Responses were analyzed via human coding, and the LightSIDE text mining, and the IBM SPSS Modeler software. Predictive models achieved high agreement with human coding (0.7 Cohen_s Kappa or greater). Models captured that when writing about matter energy transformation at the ecosystem level, students focused on primarily on the concepts of heat loss, recycling of matter, and conservation of matter and energy. Students also held non-scientific ideas such as the interconversion of matter and energy. The models developed in this study can be used to provide automatic feedback about students understanding of these concepts to biology faculty who desire to use formative written assessments in larger enrollment biology classes, but do not have the time or personnel for manual grading.