SmartEvals

http://www.smartevals.com/ritevals
RIT Faculty Login Required
In the Spring quarter of 2013, the Academic Affairs Committee (AAC) of the Academic Senate recommended that SmartEvals, a third party online platform, be used to implement RIT’s new anonymous student rating of teaching effectiveness system. This recommendation was endorsed by both the Academic Senate and the Provost, who announced the decision to move forward with the implementation of SmartEvals ready for first use in the Fall semester of 2013.
Overview
SmartEvals is an online-only system, which means that, other than through the use of a computer, smart phone, tablet, or similar computing device, there is no alternative way for students to complete their student evaluation of teaching effectiveness surveys.
The initial SmartEvals cycle consists of five steps. Once we setup SmartEvals for first use, future semesters will be much simpler to work with. For example, Step 3, Survey Setup, will not need to be repeated at the university-wide and college levels, unless a decision is made to change any of the university-wide or college level questions. Faculty, however, will need to add their own questions for each course each semester. The five initial SmartEvals steps are:
1. Database Setup
This step is transparent to faculty and is managed by the SmartEvals campus coordinator in collaboration with SmartEvals personnel, ITS and the Registrar’s office. During this stage, we create a database that consists of three basic components: Student data, course data and course registration data.
2. Setup of Emails to Faculty and Students
This step is completed by the SmartEvals campus coordinator in collaboration with RIT’s colleges and SmartEvals personnel. It consists of constructing the emails that faculty and students will get throughout the rating of teaching effectiveness process. These emails are customized to their intended targets and include an initial email sent out at the beginning of the evaluation period and subsequent reminders to encourage students to complete their evaluations. Students only get reminders for courses whose surveys they have not completed. Therefore, when a student completes all his/her surveys, he/she receives no more SmartEvlas reminders. Likewise, faculty also get reminders only for those courses where students have not completed their evaluation survey. The timing and frequency of these emails are built into the process.
3. Survey Setup
This step completed by RIT’s Student Rating of Teaching Effectiveness Office, college representatives, and individual faculty. Our students know that each course survey will take 10 or fewer minutes to complete. To meet that goal, our surveys consist of at a maximum of 20 questions per course. Each survey includes:
- Seven university-wide Likert scale items (approved by the Academic Senate, see AACStudentRatingsPresentationPPt_03-21-2013.pdf)
- One Yes / No question (approved by the Academic Senate, see AACStudentRatingsPresentationPPt_03-21-2013.pdf)
- Two open-ended, free-text questions (approved by the Academic Senate, see AACStudentRatingsPresentationPPt_03-21-2013.pdf)
- Up to ten college and faculty-specific questions. College-specific questions are decided by each college. Faculty-specific questions are added by each faculty member and, unless the faculty member allows it, no one else will see the results or any summative data derived from these questions. Faculty who wish to provide access to the results of these questions to their supervisors, must do so through the SmartEvals system. The questions themselves are visible by the supervisor.
4. Survey Completion
Surveys open for student to complete. By recommendation of the Academic Senate’s Academic Affairs Committee, this will typically be weeks 13-15 of each semester, weeks 9-10 of the summer term, and during days 13-15 of the January intersession. A customized invitation to complete their surveys is sent to each student by the SmartEvals system. A customized email is also sent to faculty advising them that course surveys are open. During the evaluation period, students are reminded via email which specific course surveys they still need to complete. Students that complete all their course evaluations receive no further reminders from SmartEvals. Faculty may also receive email reminders for those courses with less than 100% survey completion.
5. Results Analysis
SmartEvals provides an extensive set of tools to help faculty view and analyze the results of their surveys. SmartEvals keeps all your historical data to help you make decisions based on trends and other indicators.
Documents
FAQs
What is SmartEvals?
SmartEvals is an online-only system which, as of Fall 2013, is the official student rating of teaching effectiveness of record for annual performance reviews and tenure and promotion processes.
Will teaching effectiveness data produced with systems prior to SmartEvals be available for promotion and tenure decisions?
Yes. All historical teaching evaluation data obtained with current and older systems will still be available as input to the faculty evaluation processes, as required by data timelines and process needs.
Can old student ratings of teaching effectiveness systems be used?
No. SmartEvals will be RIT's official student rating of teaching effectiveness system in place starting Fall 2013.
Where can I learn more about SmartEvals?
There is an extensive library of self-training SmartEvals videos available to faculty. Please take a look at the "Self-training faculty videos" tab in this page.
What do I need to login to the SmartEvals system?
A valid RIT login ID and your RIT password. Go to http://www.smartevals.com/ritevals to take your to RIT's customized SmartEvals webpage.
I cannot login to SmartEvals. What can I do?
There may be several reasons (keep in mind that for fall 2013, SmartEvals will become available for faculty access starting September 9):
- It may not be September 9, 2013 yet. Please wait.
- You may not have a valid RIT login ID. Please note that SmartEvals uses your RIT login ID to authenticate you. Please go to http://www.rit.edu/its/help and proceed from there.
- You may have a valid RIT login ID which you may not have used in a long time. Please go to http://www.rit.edu/its/help and proceed from there.
Note that the SmartEvals database runs independent of RIT's systems. Changes made to our systems may take time to propagate to SmartEvals. While the SRATE office and ITS personnel are working diligently to keep SmartEvals up-to-date, changes done on RIT systems each week (like enabling your RIT login ID) will be reflected on SmartEvals by the end of weeks 2, 3, 4, 5 and 6 of each term. A final update will be made at the end of week 12 in preparation for opening SmartEvals to our students by the start of week 13 each semester.
I can login to SmartEvals but I do not see all the labs and courses I am teaching. Why?
There may be several reasons:
- Certain labs that run concurrently with a course are not assigned a separate final grade. Instead, the course and its lab are issued a combined final grade. SRATE surveys will not be produced for the lab component of these courses. You may be scheduled to teach this kind of labs this term. Check with your department chair or program coordinator.
- You may not be scheduled to teach any courses this term. Check with your department chair or program coordinator.
- You may be teaching courses but your department may not have updated the SIS system yet. Contact your department chair or program coordinator.
Note that the SmartEvals database runs independent of RIT's systems. Changes made to our systems may take time to propagate to SmartEvals. While the SRATE office and ITS personnel are working diligently to keep SmartEvals up-to-date, changes done on RIT systems (like updating RIT's SIS course schedule) will be reflected on SmartEvals by the end of weeks 2, 3, 4, 5 and 6 of each term. A final update will be made at the end of week 12 in preparation for opening SmartEvals to our students by the start of week 13 each semester.
Is my RIT password safe?
Yes. RIT is providing full authentication services to our SmartEvals services; your RIT password is not being shared with or stored in SmartEvals authentication services. If you update your RIT password on RIT authentication services, the new password will provide you access to our customized RIT SmartEvals webpage.
How do I login to SmartEvals?
Use the following link: http://www.smartevals.com/ritevals
At the prompt, enter your RIT login ID and password.
When and for how long will surveys be open for students?
By recommendation of the AAC, surveys will be open weeks 13-15 of the fall and spring semesters, respectively; weeks 9-10 of the summer term and days 13-15 of the January intersession.
When exactly will surveys open for Fall 2013?
Surveys will open Monday November 18, 2013 through Friday December 6, 2013.
Does SmartEvals provide online help?
Yes. SmartEvals has a robust online help feature designed to provide faculty quick and to-the-point answers. When you login to RIT's customized SmartEvals page, click on "Help" and type your question, keyword or terms that the online search system can use to identify items of relevance.
Will SmartEvals be on campus to provide training sessions in fall 2013?
Yes. In coordination with the colleges and SmartEvals, the SRATE office is setting up faculty and department chair training sessions for September 2013.
Can promotion and tenure committees see results from college, department and university-wide questions?
Yes. When colleges are ready, they will provide the SmartEvals campus coordinator with the necessary data to setup tenure and / or promotion committee access rights. Tenure and promotion committee members will only be able to view evaluations from those individuals whose documents the committee is reviewing
Can promotion and tenure committees see results from my faculty-specific survey questions?
Only if you grant them access through RIT's customized SmartEvals page.
When will I be able to add my questions to SmartEvals?
By recommendation of the AAC faculty, colleges department and faculty will be able to add their own questions during weeks 3 through 7 of each semester.
Please note that faculty-specific questions can only be added and/or edited by each individual faculty member during weeks 3 through 7 of each semester. When the add/edit period is up, neither SmartEvals personnel nor the SRATE office will be able to add, delete or edit faculty-specific questions.
How many questions can I add to each of my student rating of teaching effectiveness surveys?
You can add up to five questions to each of your surveys.
What kind of faculty-specific questions will I be able to add to my surveys?
The SmartEvals environment allows faculty to add a wide variety of question types such as Likert scale items, multiple choice, yes / no, and open ended free text questions. Likert scale items, multiple-choice and yes/no questions are highly recommended because they are easier to analyze and provide summative data over time.
Before adding your own questions, please take the following into account:
- Each survey will include ten core university-wide questions; seven of these questions focus on your effectiveness as a teacher.
- Each of your surveys may also include a number of college-specific questions provided by your college's leadership. These questions may focus on other aspects of instructional effectiveness or the teaching/learning context.
- You may have been using questions of your own with previous surveys of teaching effectiveness. You should consider whether or not some of those questions may be duplicative of either the university-wide or college-specific questions.
- Asking the same or similar question more than once within the same survey may cause students to lose interest in that survey.
Does SmartEvals provide sample faculty-specific questions I can use?
Yes. SmartEvals provides a data bank of 40 Ascend™ questions. While RIT does not necessarily endorse the use of these questions, using them will provide youwith nation-wide comparative data, which allows you to compare yourself with results from other schools. Follow this video link to learn more about SmartEvals Ascend™ Normative questions. The questions can be found here.
Self-Training Videos
| Course evaluation instructor training page |
This page is designed to help instructors familiarize themselves with the student rating of teaching effectiveness system. To access this page you will need to first login to SmartEvals using your RIT login and password. |
| Main faculty activity page |
www.smartevals.com/evalcenter This 3-minute video demonstrates the SmartEvals faculty page. Here you will find out how to add questions, send out emails, find out survey begin and end dates, and get your results. Or watch the video on YouTube: |
| Increase your student response rate |
www.smartevals.com/responserates Or watch the video on YouTube: This 4-minute video offers testimonials from schools using the online rating of teaching effectiveness system as to how they achieved higher response rates. Please note that while some of the techniques highlighted in this video include incentivizing students with extra points or other academic rewards, this will not be followed at RIT based on the recommendation from the Academic Affairs Committee of RIT’s Academic Senate. |
| General concerns |
https://www.smartevals.com/reporting/ShowVideos.aspx?y=Vv9wyoZC-_A&e=jfnvse@rit.edu&access=1 Or watch the video on YouTube: This 5-minute video addresses general concerns you may have on SmartEvals online rating of teaching effectiveness system. It also provides faculty with an overview of the tools that SmartEvals provides to analyze data. |
For Students
At RIT we are committed to providing our students with a high quality education; evaluation of teaching effectiveness is a critical part of that commitment. Each semester, during weeks 13 through 15, you will be invited to complete a short 10-minute teaching effectiveness survey for each course in which you are registered. Success of the teaching evaluation process depends on your participation and on the quality of the answers that you provide. We encourage you to take the time to provide your faculty with thoughtful answers.
Starting in Fall 2013, SmartEvals will allow students to evaluate each of their professors using a uniform online platform with a common look and feel experience across all courses. Each survey should take no more than 10 minutes to complete.
What is SmartEvals?
SmartEvals is an online-only system which, as of Fall 2013, is the official student rating of teaching effectiveness of record for annual performance reviews and tenure and promotion processes.
Can I use my smart phone or tablet to rate my teachers?
Absolutely. RIT’s SmartEvals webpage can be accessed from a variety of web-ready smart phones and tablets.
How long will each course survey take to complete?
Each survey consists of at most 20 questions, most of which ask you to select an answer from up to five available options, and at least two open-ended, free text questions. SmartEvals has advised us that surveys of this length and characteristics should require no more than 10 minutes to complete each.
How will I access my course evaluations each term?
Sometime before at the beginning of week 13 of each semester, you will get a customized email from SmartEvals alerting you that student rating for courses you are registered for in that term are open for your input. The email will include a link to a customized RIT SmartEvals page which, when you click on it, will prompt you for your RIT email and your RIT password.
Is RIT sharing my password with SmartEvals?
No. RIT is providing SmartEvals with authentication services in a way that, when you login to RIT’s SmartEvals page, it is RIT – not the vendor— that authenticates you as an RIT student.
Are student ratings of teaching effectiveness anonymous?
Absolutely. SmartEvals is a system contracted by RIT to manage RIT’s evaluation of teaching effectiveness. At no time will survey results be shared with RIT faculty or administrators in ways that specific survey responders can be uniquely identified.
However, course surveys are likely to include open-ended free-text questions whose responses instructors will be able to see in their entirety. Therefore, students are advised not to enter information that could be used to identify them.
When will I be able to fill out my teaching effectiveness surveys?
By recommendation of RIT’s Academic Senate, course surveys will open weeks 13 through 15 of each semester.
For the Fall 2013 semester surveys will open from Monday November 18, 2013 through Friday December 6, 2013.
When exactly will my instructors be able to see the results of each survey?
Typically, faculty will be able to see the results of their SmartEvals for each term the day after final grades are posted.
For Fall 2013, SmartEvals results will be open for faculty viewing December 25, 2013.
What exactly will my instructors be able to see when they review their teaching effectiveness survey results?
Instructors will be able to see summative information about each of their courses in a form that will allow them to understand how effective their teaching was in each of their courses. SmartEvals does not provide instructors with an option to identify individual responders.
However, course surveys are likely to include free-text, open-ended questions whose responses instructors will be able to see in their entirety. Therefore, students are advised not to enter information that could be used to identify them.
SmartEvals: Student Perspective
SmartEvals is an online-only system, which means that, other than through the use of a computer, smart phone, tablet, or similar computing device, there is no alternative way for students to complete their student rating of teaching effectiveness surveys. Peper forms will no longer be an option at RIT.
From a student’s perspective, SmartEvals is quite simple and consists of three main parts:
- Initial communication. At the beginning of the 13th week, you will receive a customized email from SmartEvals alerting you that your course surveys are open for responses. This email will include a link to RIT’s SmartEvals page plus a reference to each of the courses in which you are registered at the time that the surveys of teaching effectiveness process starts.
- Reminders. As you complete your course surveys, SmartEvals will take note and, during the following three weeks you may receive a reminder that one or more of your courses may still need their surveys completed. These reminders are not sent frequently and will only remind you of those courses whose surveys you have not completed. After you complete all your surveys, SmartEvals will not send you reminders anymore.
- Final reminders. About three days before the surveys close, students who have not completed all their course surveys will get a final reminder. After the survey completion period concludes, students will no longer be able to access their course surveys.
