Continuous Improvement - What’s the Secret?
RIT’s goal is that 100% of academic programs will demonstrate data-driven continuous improvement on the annual Academic Program Improvement Progress Report. After a decade of collecting Progress Reports, and with results remaining steady at around 85% (of programs demonstrating continuous improvement) for the past four years, it is a good time to provide examples of how academic programs have successfully articulated improvement efforts via their annual report, earning a rating of “Advanced” on the Continuous Improvement Rubric. Progress Report completers often ask what the secret is to receiving an “Advanced” rating. There is no secret! Advanced ratings are given to programs that articulate the following actions:
- Annual assessment of student learning outcomes
- Analysis of results
- Description of how results are used to make changes
- Re-assessment of program improvement over multiple academic years. What was the impact of the change?
Changes May Not Equal Improvement…and that’s Ok
The BS Computer Science program analyzes final exam scores in a core course to measure the extent to which their students are able to apply computer science theories and principles. The benchmark is for 80% of students to score a B or higher on this critical exam. This target was not reached and, in response, the program took several actions. First, they revised the exam to better align to the key competencies they sought to measure. Additionally, the program incorporated methods to reinforce the key competencies and new assessment opportunities to supplement the final exam. When the outcome was re-assessed the following year, results indicated that the changes did not result in improved student performance. After extensive faculty consultation, the program recommended modifying prerequisites to enhance students’ level of preparedness and changing the course designation to reflect the rigor of the course. Subsequent analysis reinforced that students who took the prerequisite performed at a higher level.
Building Sustainable Practices
The MS Criminal Justice program analyzed their students’ ability to conduct graduate level research. The target is for 70% students to complete the Capstone Project Defense with a grade of “B or above,” demonstrating competencies such as critical thinking and effective communication skills. The program was proud to see that the target was achieved but noted that a more collaborative, committee-based approach would improve the feedback provided to students and enhance project quality. Additionally, the program anticipated an increase in the number of students (and subsequent projects), making sustainability of grading processes a key factor in planning for the future. A comprehensive review of Capstone Projects was implemented, including all program faculty in the process. The following year, all of the students passed with A grades. The program plans to continue this approach and will be developing a web-based course shell to assist with the review of Capstone working papers.
Keeping an Eye on Improvement
The BS in Hospitality and Tourism Management, RIT Croatia assessed their students’ ability to work successfully in global business environment. The program focuses on effective oral, written, and visual communication skills and uses a Senior Research Project as an assessment opportunity. Students in the program exceeded the target score on the project; however, faculty identified the need to provide formative feedback to students throughout the project. The program introduced ‘milestones’ with structured deadlines and faculty mentoring and feedback that required students to complete portions of the project throughout the semester. The program re-assessed student learning the following year and found that project scores had improved, concluding that the milestones helped students to better manage their time and to approach the senior research project in a more organized manner.
It’s no longer a secret; all programs can achieve an “Advanced” continuous improvement rating with the requisite planning, iteration, and collaborative reflection. Contact the Office of Educational Effectiveness Assessment for assistance with your program’s continuous improvement efforts.