Online Annotation for Class Assessment
Leslie Kate Wright
When Kate Wright of RIT’s College of Science heard about the web-based program Nota Bene, she thought it could be a good tool to help her maintain a better understanding of her students’ learning and also increase peer interaction in a large lecture class. Through Nota Bene, Kate’s students can read and annotate pdf documents that she posts to the application website. Based on the comments and discussions among students, she can gauge their grasp of the material and modify her classes as needed to ensure that students are learning the content.
Teaching and Learning Goals
Since it can be a challenge to know where students are in large classes—are they barely keeping up or are you going over something that they already know?—Kate uses reading assignments to get a better idea of what students do and don’t understand. This enables her to adjust class plans to make sure she is using class time effectively.
In Nota Bene, Kate posts articles, papers, and other readings. Students can comment or ask questions about the content, as well as respond to each other to start a conversation.
According to Kate:
“The nice thing about the Nota Bene format is that unlike a discussion board or some other online forum, in Nota Bene the reading material is always on the screen… and when someone comments on a student post, they get an email—again, like Facebook—saying that someone has responded to your question or your comment and in the body of the email is a link that takes you right back to the reading material.”
By reviewing students’ annotations to the Nota Bene readings, Kate can identify vocabulary words that are misunderstood or being misused by students and find out what concepts are giving them trouble.
Teaching Strategies and Tools
Finding appropriate readings was the first challenge. While Kate can use primary sources, such as journal papers, with her upper level students, it has been a little tougher finding the right material for her introductory-level classes. She didn’t want to use selections that merely repeated the material in the main textbook, so she looks for complementary articles that illustrate the concepts she teaches in class.
Kate makes effective use of Nota Bene by structuring assignments, rather than simply posting readings. She also uses the application’s ability to break the class into groups, dividing the class into groups of about 15 students, so discussions are more manageable and it’s easier for all students to contribute.
It has also been important to make sure that the Nota Bene assignments are graded. While students recognize the value of the activities, unless it has an impact on their grade, they tend not to reliably follow through.
“One time I put practice exam questions on Nota Bene… it wasn’t a graded assignment, [but] I was hoping that students would comment on them and discuss back and forth like they’d been doing with the assignments… [Only] 5% of the students commented on anything. It told me that if it’s graded and they get a little encouragement to do it, they’ll do it—and as the quarter went on, usage went up, comments got better. If there were no points attached… students aren’t going to do it.”
Kate uses a simple three-point scale for each assignment based on the effort students put into their responses.
Once she could gauge student’s understanding of concepts through their comments and questions on the readings, she was able to use that information to modify her classes, if needed. For example, if students are having trouble with a concept, she can allot extra time to it; if they seem to have a good grasp of a learning point, she can check with some quick iClicker questions, and then move on.
Experiences and Results
As with many technologies, students took to using Nota Bene quickly since the commenting and discussion format is similar to Facebook.
Kate especially likes how Nota Bene brings in many of the elements of Universal Design and enables her deaf and hard-of-hearing (D/HH) and English Language Learner (ELL) students to contribute at the same level as others in the class, where they might be reluctant to do so in the classroom. This is also true of shy and introverted students.
Nota Bene also provides data that she uses to assess student engagement, such as how many and which comments lead to longer discussions, especially in the upper-level course, where typically 60 to 75% of comments receive peer responses.
Kate has one caution for faculty who want to try Nota Bene or a similar application: while these activities are resource-neutral for students, they require a substantial commitment from instructors, who need to select readings, read and respond to student comments, and modify class plans.