Site-wide links

SmartEvals

IMPORTANT DATES FOR SPRING 2013 - 2014
Event Opening Date Closing Date Duration
Faculty can add their own questions to SRATE surveys 12:00 AM
2/17/2014
11:59 PM
4/19/2014
62 days
SRATE surveys open for students 12:00 AM
4/21/2014
11:59 PM
5/15/2014
25 days
SRATE results available for faculty, department heads and deans 12:00 AM
5/28/2014
No closing date forever

http://www.smartevals.com/ritevals
RIT Faculty Login Required

In the Spring quarter of 2013, the Academic Affairs Committee (AAC) of the Academic Senate recommended that SmartEvals, a third party online platform, be used to implement RIT’s new anonymous student rating of teaching effectiveness system. This recommendation was endorsed by both the Academic Senate and the Provost, who announced the decision to move forward with the implementation of SmartEvals ready for first use in the Fall semester of 2013.

Overview

SmartEvals is an online-only system, which means that, other than through the use of a computer, smart phone, tablet, or similar computing device, there is no alternative way for students to complete their student evaluation of teaching effectiveness surveys.

The initial SmartEvals cycle consists of five steps.  Once we setup SmartEvals for first use, future semesters will be much simpler to work with.  For example, Step 3, Survey Setup, will not need to be repeated at the university-wide and college levels, unless a decision is made to change any of the university-wide or college level questions.  Faculty, however, will need to add their own questions for each course each semester.  The five initial SmartEvals steps are:

1. Database Setup

This step is transparent to faculty and is managed by the SmartEvals campus coordinator in collaboration with SmartEvals personnel, ITS and the Registrar’s office.  During this stage, we create a database that consists of three basic components: Student data, course data and course registration data.

2. Setup of Emails to Faculty and Students

This step is completed by the SmartEvals campus coordinator in collaboration with RIT’s colleges and SmartEvals personnel.  It consists of constructing the emails that faculty and students will get throughout the rating of teaching effectiveness process.  These emails are customized to their intended targets and include an initial email sent out at the beginning of the evaluation period and subsequent reminders to encourage students to complete their evaluations.  Students only get reminders for courses whose surveys they have not completed.  Therefore, when a student completes all his/her surveys, he/she receives no more SmartEvlas reminders. Likewise, faculty also get reminders only for those courses where students have not completed their evaluation survey. The timing and frequency of these emails are built into the process.

3. Survey Setup

This step completed by RIT’s Student Rating of Teaching Effectiveness Office, college representatives, and individual faculty.  Our students know that each course survey will take 10 or fewer minutes to complete.  To meet that goal, our surveys consist of at a maximum of 20 questions per course.  Each survey includes:

  1. Seven university-wide Likert scale items (approved by the Academic Senate, see AACStudentRatingsPresentationPPt_03-21-2013.pdf)
  2. One Yes / No question (approved by the Academic Senate, see AACStudentRatingsPresentationPPt_03-21-2013.pdf)
  3. Two open-ended, free-text questions (approved by the Academic Senate, see AACStudentRatingsPresentationPPt_03-21-2013.pdf)
  4. Up to ten college and faculty-specific questions.  College-specific questions are decided by each college.  Faculty-specific questions are added by each faculty member and, unless the faculty member allows it, no one else will see the results or any summative data derived from these questions. Faculty who wish to provide access to the results of these questions to their supervisors, must do so through the SmartEvals system.  The questions themselves are visible by the supervisor.

4. Survey Completion

Surveys open for student to complete. By recommendation of the Academic Senate’s Academic Affairs Committee, this will typically be weeks 13-15 of each semester, weeks 9-10 of the summer term, and during days 13-15 of the January intersession.  A customized invitation to complete their surveys is sent to each student by the SmartEvals system.  A customized email is also sent to faculty advising them that course surveys are open.  During the evaluation period, students are reminded via email which specific course surveys they still need to complete. Students that complete all their course evaluations receive no further reminders from SmartEvals. Faculty may also receive email reminders for those courses with less than 100% survey completion.

5. Results Analysis

SmartEvals provides an extensive set of tools to help faculty view and analyze the results of their surveys.  SmartEvals keeps all your historical data to help you make decisions based on trends and other indicators.

Documents

 

This document was created by SmartEvals and is designed to provide deans, department chairs and department heads with a basic understanding of the functionality of the SmartEvals environment.  Because SmartEvals used real data to create this document, it was necessary to delete from the screenshots all information that could be used to identify specific departments, colleges or individuals.

AAC - Student Rating Presentation to the Academic Senate

March 21, 2013

AAC - A University-wide system of student ratings of teaching effectiveness

March 21, 2013

AAC - Supplemental report: Steps to establishing an effective system of student ratings

March 21, 2013

Provost announcement to move forward with the implementation of SmartEvals

 

SmartEvals Ascend™ Normative questions

FAQs

I was unable to attend a SmartEvals training session at RIT. Where can I learn about SmartEvals?

You can visit our Self-Training Videos page.  You can also view a pre-recorded RIT training session here.  Also, once you login to SmartEvals, you have the option to visit SmartEvals’ extensive library of help videos and FAQs.

What is SmartEvals?

SmartEvals is an online-only system which, as of Fall 2013, is the official student rating of teaching effectiveness of record for annual performance reviews and tenure and promotion processes.

Will teaching effectiveness data produced with systems prior to SmartEvals be available for promotion and tenure decisions?

Yes. All historical teaching evaluation data obtained with current and older systems will still be available as input to the faculty evaluation processes, as required by data timelines and process needs.

Can old student ratings of teaching effectiveness systems be used?

No. SmartEvals will be RIT's official student rating of teaching effectiveness system in place starting Fall 2013.

Where can I learn more about SmartEvals?

There is an extensive library of self-training SmartEvals videos available to faculty. Please take a look at the "Self-training faculty videos" tab in this page.

What do I need to login to the SmartEvals system?

A valid RIT login ID and your RIT password.  Go to http://www.smartevals.com/ritevals to take your to RIT's customized SmartEvals webpage.

I cannot login to SmartEvals. What can I do?

There may be several reasons (keep in mind that for fall 2013, SmartEvals will become available for faculty access starting September 9):

  1. It may not be September 9, 2013 yet.  Please wait.
  2. You may not have a valid RIT login ID.  Please note that SmartEvals uses your RIT login ID to authenticate you.  Please go to http://www.rit.edu/its/help and proceed from there.
  3. You may have a valid RIT login ID which you may not have used in a long time.  Please go to http://www.rit.edu/its/help and proceed from there.

Note that the SmartEvals database runs independent of RIT's systems.  Changes made to our systems may take time to propagate to SmartEvals.  While the SRATE office and ITS personnel are working diligently to keep SmartEvals up-to-date, changes done on RIT systems each week (like enabling your RIT login ID) will be reflected on SmartEvals by the end of weeks 2, 3, 4, 5 and 6 of each term.  A final update will be made at the end of week 12 in preparation for opening SmartEvals to our students by the start of week 13 each semester.

I can login to SmartEvals but I do not see all the labs and courses I am teaching. Why?

There may be several reasons:

  1. Certain labs that run concurrently with a course are not assigned a separate final grade.  Instead, the course and its lab are issued a combined final grade.  SRATE surveys will not be produced for the lab component of these courses.  You may be scheduled to teach this kind of labs this term.  Check with your department chair or program coordinator.
  2. You may not be scheduled to teach any courses this term.  Check with your department chair or program coordinator.
  3. You may be teaching courses but your department may not have updated the SIS system yet.  Contact your department chair or program coordinator.

Note that the SmartEvals database runs independent of RIT's systems.  Changes made to our systems may take time to propagate to SmartEvals.  While the SRATE office and ITS personnel are working diligently to keep SmartEvals up-to-date, changes done on RIT systems (like updating RIT's SIS course schedule) will be reflected on SmartEvals by the end of weeks 2, 3, 4, 5 and 6 of each term. A final update will be made at the end of week 12 in preparation for opening SmartEvals to our students by the start of week 13 each semester.

Is my RIT password safe?

Yes. RIT is providing full authentication services to our SmartEvals services; your RIT password is not being shared with or stored in SmartEvals authentication services. If you update your RIT password on RIT authentication services, the new password will provide you access to our customized RIT SmartEvals webpage.

How do I login to SmartEvals?

Use the following link: http://www.smartevals.com/ritevals

At the prompt, enter your RIT login ID and password.

When and for how long will surveys be open for students?

By recommendation of the AAC, surveys will be open weeks 13-15 of the fall and spring semesters, respectively; weeks 9-10 of the summer term and days 13-15 of the January intersession.

Does SmartEvals provide online help?

Yes. SmartEvals has a robust online help feature designed to provide faculty quick and to-the-point answers. When you login to RIT's customized SmartEvals page, click on "Help" and type your question, keyword or terms that the online search system can use to identify items of relevance.

Can promotion and tenure committees see results from college, department and university-wide questions?

Yes. When colleges are ready, they will provide the SmartEvals campus coordinator with the necessary data to setup tenure and / or promotion committee access rights. Tenure and promotion committee members will only be able to view evaluations from those individuals whose documents the committee is reviewing

Can promotion and tenure committees see results from my faculty-specific survey questions?

Only if you grant them access through RIT's customized SmartEvals page.

How many questions can I add to each of my student rating of teaching effectiveness surveys?

You can add up to five questions to each of your surveys.

What kind of faculty-specific questions will I be able to add to my surveys?

The SmartEvals environment allows faculty to add a wide variety of question types such as Likert scale items, multiple choice, yes / no, and open ended free text questions. Likert scale items, multiple-choice and yes/no questions are highly recommended because they are easier to analyze and provide summative data over time.
Before adding your own questions, please take the following into account:

  1. Each survey will include ten core university-wide questions; seven of these questions focus on your effectiveness as a teacher.
  2. Each of your surveys may also include a number of college-specific questions provided by your college's leadership.  These questions may focus on other aspects of instructional effectiveness or the teaching/learning context.
  3. You may have been using questions of your own with previous surveys of teaching effectiveness. You should consider whether or not some of those questions may be duplicative of either the university-wide or college-specific questions.
  4. Asking the same or similar question more than once within the same survey may cause students to lose interest in that survey.

Does SmartEvals provide sample faculty-specific questions I can use?

Yes.  SmartEvals provides a data bank of 40 Ascend™ questions.  While RIT does not necessarily endorse the use of these questions, using them will provide youwith nation-wide comparative data, which allows you to compare yourself with results from other schools. Follow this video link to learn more about SmartEvals Ascend™ Normative questions.  The questions can be found here.

Self-Training Videos

RIT Training Session

http://youtu.be/isJCD_yd7nM

This video was recorded at RIT during one of the SmartEvlas training sessions we offered early in the Fall 2013 semester.

Course evaluation instructor training page

www.smartevals.com/evalcenter

This page is designed to help instructors familiarize themselves with the student rating of teaching effectiveness system.  To access this page you will need to first login to SmartEvals using your RIT login and password.

Main faculty activity page

www.smartevals.com/evalcenter
At the prompt, enter your full RIT email

This 3-minute video demonstrates the SmartEvals faculty page. Here you will find out how to add questions, send out emails, find out survey begin and end dates, and get your results.

Or watch the video on YouTube:
https://www.youtube.com/watch?feature=player_embedded&v=53_KJvIGy30

Increase your student response rate

www.smartevals.com/responserates
At the prompt, enter your full RIT email

Or watch the video on YouTube:
https://www.youtube.com/watch?feature=player_embedded&v=EOAfbuSiXDU

This 4-minute video offers testimonials from schools using the online rating of teaching effectiveness system as to how they achieved higher response rates.

Please note that while some of the techniques highlighted in this video include incentivizing students with extra points or other academic rewards, this will not be followed at RIT based on the recommendation from the Academic Affairs Committee of RIT’s Academic Senate.

General concerns

https://www.smartevals.com/reporting/ShowVideos.aspx?y=Vv9wyoZC-_A&e=jfnvse@rit.edu&access=1 
At the prompt, enter your full RIT email

Or watch the video on YouTube:
https://www.youtube.com/watch?feature=player_embedded&v=Vv9wyoZC-_A#at=79

This 5-minute video addresses general concerns you may have on SmartEvals online rating of teaching effectiveness system.  It also provides faculty with an overview of the tools that SmartEvals provides to analyze data.

Tips

Taking just five minutes of a single class to explain to students why SRATE surveys are important, you can increase the response rate for that class by 10% or more.  These tips represent years of experience using SmartEvals at many universities.  Faculty are encouraged to share with the SRATE office their own tips for future inclusion in this list.

Tip # 1

Set aside five minutes at the beginning of each class to talk to your students about the value of the student rating of teaching effectiveness process. Let your students know that:

  • The rating of teaching effectiveness process has begun.
  •  Each survey will take at most 10 minutes to complete.
  • They will receive emails from SmartEvals encouraging them to complete their surveys.
  • Their responses are anonymous and that you will only see the results of the surveys after you have submitted final grades. And that you will only see summative information from their responses.
  • SmartEvals is a third-party organization that RIT has contracted with to administer and manage our student ratings of teaching effectiveness process.

Tip # 2

Did you know that an email asking your students to complete their surveys can increase your response rate by up to 15%? Here is how this works in SmartEvals:

From within the SmartEvals environment, you can email personalized reminders to those students who have not completed their survey.  The recipients of the emails you send are not known to you.  To edit these emails, login to your SmartEvals course page and click the hyperlink in the "Not Resp" column.  This will display a window titled "Edit Mails To Be Sent Out" where you can edit the email that will be sent ONLY to those students who have not completed any of your surveys.  Once you have made the changes, click the "Update Message" button to save.  Press the "Send" button and the rest is done by SmartEvals!

The SmartEvals system is initially set to "Do not automatically send this email next semester".  We recommend that the first time you use the system you keep this option.

Tip # 3

It sometimes makes sense for an instructor to reserve a few minutes of a lab or class meeting as teaching effectiveness rating period.  Remember: each survey will take at most 10 minutes to complete. Some classes are already taught in a computer lab environment making it easier to allow time to complete the survey as part of the class period. If the class is not taught in a computer lab or if it is not possible to easily schedule a computer lab space have students use their laptops, tablets or smartphones to complete their surveys during a designated class period.

Tip # 4

Did you know that SmartEvals is programmed to send reminders to students who have not completed their SRATE surveys approximately every 72 hours?  Just reminding your students that as soon as they complete their surveys the emails will stop, will motivate them to complete them.

You can also prevent the standard SmartEvals reminders by sending a personalized reminder from you by using SmartEvals email feature. Tip # 2 explains how this works.

Tip # 5

Students who have not completed their surveys may be getting a reminder from SmartEvals.  Faculty may want to remind them that as soon as they complete all their surveys, SmartEvals will stop sending them reminders.

Tip # 6

Share with your students some of the benefits you get as a result of their responses.  Perhaps there is a class you taught in a previous term, which you changed as a result of student feedback; or perhaps there was class project you did not think had gone well but your students told you otherwise in their surveys.

Tip # 7

Take a few minutes of each of your classes and thank your students for taking the time to complete their surveys.

Tip # 8

Remind your students that each survey should take at most 10 minutes of their time and that their answers can provide valuable information you can use to keep offering great courses.

Tip # 9

You may want to remind your students that results of these surveys may help you improve not only this class but other classes you teach as well.

For Students

At RIT we are committed to providing our students with a high quality education; evaluation of teaching effectiveness is a critical part of that commitment.  Each semester, during weeks 13 through 15, you will be invited to complete a short 10-minute teaching effectiveness survey for each course in which you are registered.  Success of the teaching evaluation process depends on your participation and on the quality of the answers that you provide.  We encourage you to take the time to provide your faculty with thoughtful answers.

Starting in Fall 2013, SmartEvals will allow students to evaluate each of their professors using a uniform online platform with a common look and feel experience across all courses.  Each survey should take no more than 10 minutes to complete.

What is SmartEvals?

SmartEvals is an online-only system which, as of Fall 2013, is the official student rating of teaching effectiveness of record for annual performance reviews and tenure and promotion processes.

Can I use my smart phone or tablet to rate my teachers?

Absolutely. RIT’s SmartEvals webpage can be accessed from a variety of web-ready smart phones and tablets.

How long will each course survey take to complete?

Each survey consists of at most 20 questions, most of which ask you to select an answer from up to five available options, and at least two open-ended, free text questions.  SmartEvals has advised us that surveys of this length and characteristics should require no more than 10 minutes to complete each.

How will I access my course evaluations each term?

Sometime before at the beginning of week 13 of each semester, you will get a customized email from SmartEvals alerting you that student rating for courses you are registered for in that term are open for your input. The email will include a link to a customized RIT SmartEvals page which, when you click on it, will prompt you for your RIT email and your RIT password.

Is RIT sharing my password with SmartEvals?

No. RIT is providing SmartEvals with authentication services in a way that, when you login to RIT’s SmartEvals page, it is RIT – not the vendor— that authenticates you as an RIT student.

Are student ratings of teaching effectiveness anonymous?

Absolutely. SmartEvals is a system contracted by RIT to manage RIT’s evaluation of teaching effectiveness. At no time will survey results be shared with RIT faculty or administrators in ways that specific survey responders can be uniquely identified.

However, course surveys are likely to include open-ended free-text questions whose responses instructors will be able to see in their entirety. Therefore, students are advised not to enter information that could be used to identify them.

When exactly will my instructors be able to see the results of each survey?

Typically, faculty will be able to see the results of their SmartEvals for each term the day after final grades are posted.

What exactly will my instructors be able to see when they review their teaching effectiveness survey results?

Instructors will be able to see summative information about each of their courses in a form that will allow them to understand how effective their teaching was in each of their courses. SmartEvals does not provide instructors with an option to identify individual responders.

However, course surveys are likely to include free-text, open-ended questions whose responses instructors will be able to see in their entirety. Therefore, students are advised not to enter information that could be used to identify them.

I would like to re-submit an SRATE survey. Can I do that?

Yes. But you must first complete all your SRATE surveys before you are allowed to edit any of them.  Assuming that you completed all your surveys, login to your SmartEvals page, select the course whose SRATE survey you want to re-submit, and click on the “Retake Evaluation” button.

SmartEvals: Student Perspective

SmartEvals is an online-only system, which means that, other than through the use of a computer, smart phone, tablet, or similar computing device, there is no alternative way for students to complete their student rating of teaching effectiveness surveys. Peper forms will no longer be an option at RIT.

From a student’s perspective, SmartEvals is quite simple and consists of three main parts:

  1. Initial communication.  At the beginning of the 13th week, you will receive a customized email from SmartEvals alerting you that your course surveys are open for responses.  This email will include a link to RIT’s SmartEvals page plus a reference to each of the courses in which you are registered at the time that the surveys of teaching effectiveness process starts.
  2. Reminders.  As you complete your course surveys, SmartEvals will take note and, during the following three weeks you may receive a reminder that one or more of your courses may still need their surveys completed.  These reminders are not sent frequently and will only remind you of those courses whose surveys you have not completed.  After you complete all your surveys, SmartEvals will not send you reminders anymore.
  3. Final reminders.  About three days before the surveys close, students who have not completed all their course surveys will get a final reminder.  After the survey completion period concludes, students will no longer be able to access their course surveys.