Jump to Navigation
dept services image

ESCI Recommended methods and practices

How can our department best manage the logistics of handing out and collecting ESCI surveys?

The most important principle is to develop a workable system that takes minimal resources but protects the integrity of the information and thereby protects the interests of the students, the instructors and TAs being rated, and the University. Any such system must ensure that the person being rated a) does not have access, and b) is not perceived by students as having access, to the completed rating sheets at any time before they are returned to Instructional Development for tabulation, or to written student responses prior to grade submission. Anonymity of the raters, and confidentiality of the results at every step of the process, is essential.

Ideally the surveys should be distributed and collected by someone not affiliated directly with the course. Persons successfully use in this capacity have included departmental staff members, departmental honor society undergraduate students, graduate students who are not teaching assistants for the course, and temporary departmental employees hired for these purposes. If no other practical or economic alternatives are available, trustworthy volunteer students enrolled in the course have been used. Any person administering and/or collecting the surveys in class and/or transporting back to the department should be instructed to place the surveys in an envelope to be immediately sealed in the presence of another person, and return them as soon as possible to the department.

There are five steps in the overall process: getting packets to the classroom, handing out the survey materials, giving instructions to the students, collecting the response sheets, and returning the response sheets to the department office. Each of these steps typically can be carried out by any of several people. Let’s examine the steps sequentially.

1. Get the packets to the right classroom at the right time

Since this task is a logistical one, and the security risks are small (in terms of the likelihood that tampering with the forms will not be detected), there’s not a compelling reason to require the delivery of the unused packets to be done by a “neutral” party unaffiliated with the course. Several departments have considerable experience with instructors and TAs taking packets to their own courses or sections, but in no case do these persons being rated have access to the completed response sheets. Persons who could do this include:

  • Departmental staff (either permanent or casual, hired for this purpose)
  • Grad students not affiliated with the course being rated
  • TAs or Readers affiliated with the course
  • Departmental honor society undergraduates
  • Volunteer or paid undergraduates, arranged ahead of time, not affiliated with the class
  • Volunteer undergraduates, arranged ahead of time, enrolled in the class
  • The instructor or TA being rated

2. Hand out the Questionnaires and the Response Sheets

A. How many things?

The material to be passed out to students frequently (but not always) consists of two sheets of paper: a Questionnaire that contains the questions to which students are asked to respond (other than campus-wide Questions A and B, which are printed on the optical-scan Response Sheet), and an optical-scan Response Sheet on which they record their ratings.

An infrequently used alternative is to present questions by overhead projector or data projector. This saves costs of reproducing paper copies, but does raise issues of how to handle response sheets for open-ended questions, which cannot be recorded or processed on the optical-scan response sheets.

It is common for departments and individual instructors to include some open-ended questions in their surveys, and/or to encourage students to elaborate their “bubbled” ratings with narrative comments. As noted above, the optical-scan response sheets cannot accept handwritten or printed responses, since there is not room on the form and in any case the system cannot “read” or electronically process them. Therefore, students must respond to open-ended questions on some medium other than the optical-scan response sheets. The simplest and most frequently chosen solution is to have students write their comments and responses to open-ended questions on the Questionnaire, and to collect both Questionnaires and Response Sheets. (Click HERE for recommendations on best practices in handling student written comments.)

Thus, for courses following this typical model, the packet should contain, and the administrator should pass out, two sheets of paper to each student: a Questionnaire and an optical-scan Response Sheet.

In those courses for which the instructor desires to add open-ended questions to the survey, it may be desirable to either a) pass out an additional sheet containing the questions and space to respond, or b) project the additional questions via overhead or data projector, or write them on the board, and ask students to respond on paper that is either handed out or that the students supply. In either case, this becomes a third item to collect as the rating process is finished, and may be a third item to hand out.

B. How should they be passed out?

The method of passing out the materials does not matter much, so long as each student receives a copy of each form in unwrinkled or unfolded condition, in a timely way. In large classes, particularly in large lecture halls, it is common to give a small stack of each form to students sitting on the aisles and let them pass the forms along the row. Ideally there are enough monitors or volunteers to make sure that each student gets a form, that no forms are left over or wasted, and that no one has multiple forms to fill out.

C. Who should pass them out?

Students’ perception of the institutional and personal importance of the ratings can be an important factor in how seriously they attend to the task of rating. The instructor’s attitude and approach to the rating process is one of the important signals to the students. In general, it is less important whether the instructor is involved in physically passing out the surveys than whether the instructor tells the students via words and actions that this process is important, is taken seriously, and that their honest opinion is valued. Thus, it is recommended that the instructor, or a TA or reader, solicit cooperation from the class, whether or not this occurs as part of the brief instructions discussed below in 3., and whether or not this person actively passes out forms during the comments or instructions.

Persons who could do this include:

  • Departmental staff (either permanent or casual, hired for this purpose)
  • Grad students not affiliated with the course being rated
  • Departmental honor society undergraduates
  • Volunteer undergraduates, arranged ahead of time, not affiliated with the class
  • Volunteer undergraduates, arranged ahead of time, enrolled in the class
  • Students assigned, or who volunteer, at the time of survey administration in class
  • TAs or Readers affiliated with the course
  • The course instructor

It is essential to the integrity of the ratings system that any person being rated NOT have access to, or be perceived as having access to, the raw ratings and comments or any summaries thereof prior to the submission of course grades. See 3. below for suggestions on how to communicate such intent.

3. Give brief instructions on filling out the forms

As mentioned above, students’ perception of the importance of the ratings can affect how seriously they attend to the task of rating. Brief instructions serve the functions both of reinforcing the importance of the ratings and clarifying some logistics.

By the time the first students begin to fill out their response forms, a survey administrator should get the attention of the entire class and convey a few brief points.

A.The purposes of the overall survey (typically, to help the instructor improve the course, and to help the department know how well teaching is being carried out). A statement that this rating process is important, is taken seriously, and that students’ honest opinion is valued.

B. A reminder to write the instructor’s name, course information, and quarter on both the optical-scan Response Sheet and any other sheets to be used for written comments.

C. A reminder to answer questions A and B on the Response Sheet, since these are particularly important for the academic personnel process (IF the department uses these items).

D. The purposes of other sections of the survey, if the departmental or course-specific questions are clustered into meaningful themes.

E. Assure students that the system is designed to protect their anonymity. The instructor will not see statistical reports until grades are submitted. The instructor will not see written comments until grades are submitted (and, if departmental practice, written comments will be word processed to eliminate handwriting as a means of identification of authorship). Remind students not to put their names on the response sheets.

F. Instructions on what to do with their forms when completed (e.g. “pass them to the aisle,” or “hand them to me at the door,” or “give them to a volunteer,” or “put them face down on the table at the front of the room.”)

Person who could do this include:

  • Departmental staff (either permanent or casual, hired for this purpose)
  • Grad students not affiliated with the course being rated
  • Departmental honor society undergraduates
  • Volunteer undergraduates, arranged ahead of time, not affiliated with the class
  • Volunteer undergraduates, arranged ahead of time, enrolled in the class
  • Students assigned, or who volunteer, at the time of survey administration in class
  • TAs or Readers affiliated with the course
  • The course instructor

It is essential to the integrity of the ratings system that any person being rated NOT have access to, or be perceived as having access to, the raw ratings and comments or any summaries thereof prior to the submission of course grades.

Two very clear ways to signal this intention are for the person being rated to announce that s/he will leave the room during the actual rating, and also to mention the method that will be used to deliver the completed forms to the department office.

4. Collect All Response Forms

Collect all forms that have been used to record student opinions, including optical –scan Response Sheets with “bubbled” responses, Questionnaires with comments or narrative responses, and other pages with comments or narrative responses.

One or more persons not directly involved with teaching the course (e.g. not a person being rated, and not a course instructor, TA or reader) should be responsible for placing the forms in the envelope(s) or packet(s) provided by the departmental ESCI Contact Person. Seal the packet(s) if possible. As indicated below, the next step will be to return the packet(s) as pre-arranged to the department.

Note that only one (preferably sealed) envelope/packet will typically be necessary if the only response form used by students is the ESCI optical-scan Response Form. If, however, the students also respond with written comments or other narrative responses on the Questionnaire or on a separate response sheet, the department may wish to have those returned in one or more separate envelopes/packets. It may be physically possible to return all response forms in one envelope/packet, but the department may prefer that the response forms be sorted by kind in the classroom by the person(s) collecting the forms, or sorted later by the ESCI Contact Person. These details are at the discretion of the department, but should be clearly communicated to the person collecting the response forms by the ESCI Contact Person.

It is highly desirable that two or more persons be involved in collecting the completed response sheets and sealing them into the packet(s), so that both the actuality and the appearance of opportunity for misbehavior (i.e. alteration of response sheets) is minimized.

Person who could do this include:

  • Departmental staff (either permanent or casual, hired for this purpose)
  • Grad students not affiliated with the course being rated
  • Departmental honor society undergraduates
  • Volunteer undergraduates, arranged ahead of time, not affiliated with the class
  • Volunteer undergraduates, arranged ahead of time, enrolled in the class
  • Students assigned, or who volunteer, at the time of survey administration in class

It is essential to the integrity of the ratings system that any person being rated NOT have access to, or be perceived as having access to, the raw ratings and comments or any summaries thereof prior to the submission of course grades. This is the stage of the ratings process where actual and perceived access to raw information becomes a potential issue.

5. Return All Response Forms to the Department

Prior to the administration of the surveys in class, the person(s) responsible for collecting and returning the response sheets should have a clear understanding of the expected and acceptable method(s) of transporting the sealed packet(s) of response sheets to the department. Presumably the acceptable method(s) would be communicated by the ESCI Contact Person.

The response forms collected in each class should be transported to the department via the acceptable method. Such methods may include:

  • immediate hand-delivery
  • next-day hand-delivery (for evening classes)
  • US Postal Service
  • other prepaid commercial delivery service
  • Note: campus mail typically is not accessible for drop off after the same business hours that the department is open, so typically would not provide a feasible and secure delivery solution.

It is expected that packets would be addressed to the ESCI Contact Person in the academic department, and that this person would open the packets and process the response sheets.

Persons who could return the response forms to the Department include:

  • Departmental staff (either permanent or casual, hired for this purpose)
  • Grad students not affiliated with the course being rated
  • Departmental honor society undergraduates
  • Volunteer undergraduates, arranged ahead of time, not affiliated with the class
  • Volunteer undergraduates, arranged ahead of time, enrolled in the class
  • Students assigned, or who volunteer, at the time of survey administration in class

Records Retention for Departments

Per record retention policy in section IX - 18 of the Red Binder, UCSB Campus Policies and Procedures For Academic Personnel:

I. Retention of files
Documents are to be maintained as follows:

Personnel files*:
Senate faculty: Academic Personnel maintains files for Senate faculty indefinitely. If a Department or College is keeping a secondary file, that file must be maintained until the employee separates from the University.


Non-Senate Academics: Academic Personnel, as the office of record, maintains files for 5 years after separation.  Departments must retain files until the employee separates from the University.


*Items that are stored electronically on line by the Academic Personnel office and are accessible to the department or college do not need to be separately maintained in the department or college.


Applicant Files:
Files of applicants who do not become employees are to be maintained by the department for 3 years.


Faculty appointment cases that are put forward but for review, but are ultimately unsuccessful recruitments will be maintained by Academic Personnel for 3 years.


Teaching evaluations (student comments and ESCIs) are to be maintained for the longer of:
1) until used in a review file, or
2) as long as a need is present

You can access this document at the Academic Personnel website (https://ap.ucsb.edu/).

 

consultation contacts

George Michaelsexecutive director2130 Kerr Hall
work805-893-2378
lisa berrysenior instructional consultant1130 Kerr Hall
work805-893-8395
mindy colininstructional consultant1130 Kerr Hall
work805-893-2828
Mary Lou Ramos database and ESCI administrator1130 Kerr Hall
work805-893-3523
Aisha Wedlaw ESCI assistant1124 Kerr Hall
work805-893-4278
Breana Barakoffice manager 1130 Kerr Hall
work805-893-2972
faxfax: 805-893-5915