Faculty Governance
II.D Student Evaluation Procedures
1.0 Rationale for revising the Liberal Arts Student Evaluation Procedures
It is Institute policy that each instructor each quarter shall give the students in at least one section of each of his/her courses the opportunity to evaluate the teacher and the course in some standard written way. Up to the present time, the College of Liberal Arts has responded to this policy by distributing the 'Survey of Student Opinion' form developed some years ago by the RIT Office of Instructional Development. However, it is clear that numerous instructors in the College of Liberal Arts are dissatisfied with that form. This document, with its attached questionnaire, represents the efforts of the Liberal Arts Student Evaluation Committee to develop an improved procedure for student evaluation in our college.
2.0 Purpose of Student Evaluation Forms
Student evaluation forms are designed to measure student perceptions of the effectiveness of the teacher and the course. They can provide meaningful data, particularly in the areas of what transpires in the day-to-day classroom situation and/or on the nature of formal and informal contacts with students outside the Class. Research indicates that student judgments can provide a good source of first-hand data, but how much faith can be placed in these Judgments depends entirely on the quality of the instrument and the procedures employed to administer, collect, and interpret them. Instruments can be devised both for formative ends, i.e., designed to provide information to help an instructor improve in a specific Class situation; as well as for summative purposes, i.e., to provide administration and promotion and tenure committees with one useful source of data on the perceived quality of a teacher's performance in the classroom. Each type of evaluation demands its own type of questions, and the two should not be confused.
3.0 Difficulties Inherent in the Existing Procedures for Student Evaluation in the College of Liberal Arts
The present procedure in the College of Liberal Arts employs a standard questionnaire. Given the broad variety of courses and teaching techniques employed in the college, many instructors have found the existing instrument to be too inflexible. They find they are collecting information on a form which is unsuited to their particular courses; therefore, at least part of the data generated thereby is either irrelevant or misleading. In addition, items on most standardized rating forms are in great part limited to broad questions. Such questions, though appropriate for summative comparisons between teachers, often prove to be too general to help an individual teacher in his particular course context. When individual instructors, seeking more information, develop their own questionnaire - as many instructors are presently doing in Liberal Arts - another set of difficulties arise One must question if the individual instructor has designed the best instrument possible for his purposes. A well-designed questionnaire can gather impressions not only about clarity of presentations, difficulty of exams and the like, but can also probe academic outcomes and attitude change. Moreover, the diversity of many self-developed questionnaires creates difficulties for developing comparable data upon which to base salary, promotions, and tenure recommendations.
4.0 A New Questionnaire for the College of Liberal Arts
This questionnaire (see Section 5.0) and accompanying procedures (See Section 7.0 and 8.0) are based on an effort to resolve the difficulties in the current system now in use in the College of Liberal Arts. The questionnaire consists of four parts:
A) Section I: Student Data
Herein are included four items which relate to those student and course characteristics which can influence student ratings of the teacher and the course. Questions #l and #2 relate to the student's initial motivation to take the course. Questions #3 and #4 inquire into the perceived “load” or difficulty of the course.
B) Section II: General Questions
The five items included herein are 'Global' questions. Research indicates that with respect to an instructor's overall level of performance these questions yield essentially the same information as large numbers of more specific questions. Therefore, these items (though they may be phrased in different forms in different questionnaires) are generally accepted as being valid overall measures of student perceptions of teachers and course effectiveness. These five general questions thus comprise the 'core' items of the questionnaire, i.e., those questions which all instructors will ask their students, and which yield broadly informative results useful in making summative comparisons among faculty for the purposes of salary, tenure, and promotion recommendations. It has been found that for summative purposes, such global ratings are clearly more useful than the rating of specific practices.
C) Section III: Specific Questions
This section is designed by the instructor with items chosen specifically to inquire about a particular course. Its purpose is strictly formative; i.e., to collect specific suggestions and criticisms from the students to the end of improving teaching and the course. Section III provides space for sixteen such optional items. It is strongly suggested that the instructor choose his specific items from the question bank provided for that purpose.
The Question bank (see Section 6.0) is a revision of the catalogue of items developed by the CRLT (Center for Research on Learning and Teaching) at the University of Michigan. It consists of 144 items relating to three basic categories: Instructor Behavior, Course Elements, and Student Change.
After choosing his items, it will be up to the individual instructor to number and reproduce his personal item list. He will then distribute his list with the questionnaire. The students will record their responses in the op-scan column of the questionnaire.
Experience at Purdue and Michigan indicates that instructor-designed questionnaires clearly increased rating specificity. Instructors got feedback about their specific concerns, not about matters that were irrelevant to them. Moreover, the system tended to generate teacher involvement and thus increased the possibility of improvement.
D) Section IV: Comments
This section gives the student the opportunity to write comments about the perceived characteristics of the teacher, the teaching, and the course. This section is primarily designed to serve formative ends. Hence it is given some structure: The student is asked to focus his comments on what he perceived as particularly meritorious and what he perceived as needing improvement. However, if the instructor wishes to do so, he may submit these comments to any administrative person or body for consideration in the summative context as well. The question bank also includes some additional open-ended questions which the instructor may choose to augment those provided in the form.
5.0 The Questionnaire (see attached form)
6.0 The Question Bank
(See, if you wish, available 'Catalogue of Items for Instructor Designed Questionnaires')
7.0 Implementation
7.1 Distribution of forms to the faculty:
The Liberal Arts Central Office will distribute the appropriate number of questionnaire forms and envelopes to the faculty. The individual faculty member will supply an appropriate number of copies of his list of specific questions for Section III of the questionnaire. He will then hand out both sheets to his class(es).
After the questionnaires are completed, a student designated by the Instructor will (1) collect the completed op-scan forms, (2) place them in the envelope provided, and (3) deliver the envelope to a neutral person.
The faculty will designate an appropriate secretary as the neutral individual. This person will then check the envelopes for unused forms, extraneous marks on the forms, and course and/or instructor coding errors. Special instructions from Individual instructors for specific correlations may be included with the envelopes at this time. The neutral person will then deliver the questionnaires to the RIT computer center for processing.
7.2 Processing:
Processing will take place at the RIT computer center. Each set of evaluations will be processed into two printouts: a) a complete printout of all data produced by the questionnaire, b) a second printout of the results of the four questions in the student data section and the five summative questions only.
7.3 Printouts:
A) The complete printout will break down the answers to each question into the number of responses to each unit on the question scale. The number of students who submitted a response to the question will also be indicated. The mean and standard deviation will be calculated for each item where appropriate (i.e., all questions but 41 and #2). A summary mean will also be tabulated for questions #5 through #9. The mean for any item is the average of all the 1 through 5 alternatives to the item. A summary mean is the average of the means for a given question for all sections of the course taught in the past by this instructor. For all sections, with the sole exceptions of items #3 and #4, the higher means are 'better' or more positive. The standard deviation is best described as the spread among the responses to each question. The smaller the standard deviation, the greater the consensus among students in the class. The omits are excluded in computing the means and standard deviations.
B) The second printout will provide identical data, but only include the results of the four questions in the student data section and the five summative questions in Section II.
7.4 Mid-course Evaluation
The questionnaire form will be made available for mid-course evaluation on request. Since such evaluations are solely aimed at the improvement of the course and the teaching, it will be left up to the instructor to collect and score the forms on his own.
8.0 Use of the Results
The rationale of providing each instructor with two printouts is based on the fact that the results of the questionnaire can be employed for both formative and summative ends.
8.1 The primary purpose of the complete printout is to serve formative ends, providing instructor feedback relating to various instructional elements in a course. This information can then be used to support efforts to improve one's teaching in specific ways.
8.2 The second printout (the five global questions and the four student data questions only) is provided specifically for the summative purpose of developing a database which can be employed, along with other appropriate information, for annual salary recommendations, annual tenure reviews, tenure and promotion recommendations. As indicated in Section 4.0, research indicates that for summative purposes global ratings are more useful than ratings of specific practices; thus the reason for restricting the data in the printout. The results of the student data questions are included in this printout because such factors as initial desire to take the class and perceived load can have direct impact on student perceptions of the course and teaching.
8.3 All printouts and questionnaires will be the exclusive property of the faculty member. After processing, both printouts and the questionnaires will be returned to the designated faculty member. That person shall distribute both printouts and questionnaires to the faculty, though Only after grades have been submitted. Each instructor shall then have the option to submit the short printout to his staff chairman/director or the Dean for reference in the context of future summative procedures.
9.0 Recommendations
9.1 For the use of results for formative ends.
If an instructor believes his questionnaire response indicates a need for Improvement, a commitment to that goal might be fulfilled in several ways:
A) Self-assessment and accompanying experiments with change.
B) Requesting advice from colleagues. Research indicates peer evaluation is particularly valuable in the areas of course materials and organization, examination of grading procedures, and measuring knowledge of the subject matter.
C) Individual consultation sessions with Institute instructional consultants.
In every case, the instructor seeks out changes in student responses in one or more specific areas as measured by future evaluations, thus indicating improvement in teaching performance in those areas.
9.2 For the use of the results for summative ends
In the context of evaluative recommendations in response to the results of student questionnaires. the following factors should receive consideration:
A) To insure fairness in the context of such important recommendations, it seems important for the college to adopt a uniform instrument for student evaluation. If the relevant data are generated by the same instrument for all the faculty, comparisons should be more valid and reliable.
B) Certain given conditions such as the newness, level, and subject area of the course. the size of the class, whether the course was elective or required, teaching experience and expected course grade can influence students ratings. With these factors in mind, faculty might add submitted printouts with appropriate comments.
C) The consistency of the ratings over years and courses should be taken into account. Faculty should consider compiling data over time to facilitate long-tern overviews of their teaching performance. (Herein lies the rationale for including the summary mean data in the printouts.)
D) Fractional deviations in ratings from section to section, from quarter to quarter for a given instructor, or similarly between instructors, carry little or no statistical importance.
10.0 Summary
It should always be kept in mind that student evaluation instruments are only one means among others to measure the quality of teaching performance. Therefore it is very likely that a multiple indicator approach, i.e., student evaluation, self-evaluation, and peer evaluation, would be the most valid approach to evaluating teaching and course effectiveness. Given the Institute's commitment to good teaching, perhaps a future committee might consider the feasibility of such an approach.
Approved 1979
