Jump to Navigation
dept services image

Report Description

The ESCI report has been designed to provide a clear and concise summary of student ratings data for an individual course in a given quarter. The layout is also designed to provide comparative information between the current course in the current quarter and the academic department in the current quarter, the academic department over time (past five years), and the campus over time. Below are described the key concepts underlying the report, followed by an explanation of the statistics reported, as well as the categories into which the four lines of statistics are divided. Following that are examples of actual output and what is reported in each portion of the report. If you have any questions about the format of the report, please contact Instructional Consultation staff at x-2972. An explanation of how best to interpret ESCI results is provided in the document "Interpreting ESCI Results", which is available from the main menu or by clicking on the title above.

An example ESCI report can be found at the bottom of this page

Proper interpretation of the ESCI Report relies upon an understanding of the following concepts:

Norms

It can be challenging to interpret the results of ratings on items that use response categories that are "relative," as opposed to categories having some kind of "absolute" meaning. For example, a question about the number of hours spent on homework in the average week lends itself to a fairly unambiguous interpretation; but a question like campus item A using response categories of "Excellent" through "Poor" can be interpreted more richly if results from one course can be compared to results from other courses.

In order to make possible this kind of relative comparison, ESCI calculates and reports "norms" for EVERY (fixed-response) item on EVERY ESCI survey. Without going into technical detail, the term "norms" refers to a presentation of results for an item that reflect what ratings were made in some comparison group of courses - in other words, what would "normally" or "typically" be expected. Using this information, one can better see whether a particular set of results reflect truly outstanding ratings, problematic ratings, or ratings in the normal, to-be-expected range.

Norms are calculated for the entire set of users within the instructor's department both for the current quarter and over time, meaning the most recent five year period. Norms are also calculated for the entire campus over time, again meaning the most recent five year period. Norms are calculated separately for faculty, associates, and teaching assistants. Thus, for a particular item, an instructor would have information not only on the performance of her own class but also average current and long-term departmental results, as well as overall long-term campus-wide results.

Item based

For each course, departmental and campus norms are available for every item.

Course Level

ESCI reports data separately for undergraduate (UG) and graduate (GR) courses.

Rank of Instructor

The norms are calculated separately for faculty, associates, and teaching assistants. Thus, for a particular item, an instructor would have information not only on the ratings of his/her own class but also on average current and long term departmental ratings, as well as ratings for overall campus-wide use of the item by colleagues.

Departmental Norms

Norms are calculated both for the current quarter and across past quarters for the last five years by department. Separate departmental norms are calculated for undergraduate (UG) courses and graduate courses (GR).

Norm types

Student-Weighted undergraduate (UG), Course-Weighted undergraduate (UG), Student-Weighted graduate (GR), or Course-Weighted graduate (GR).

Student-Weighted Norms

In brief, student-weighted norms are what you are used to seeing in surveys: each student's "vote" counts the same, whether s/he is in a 200 student course or a 15 student course. This is equitable from many standpoints, but it does have the implication that when data are aggregated for an entire department (either for the current quarter or over time), the results for the large course will "swamp" the results for a smaller course. The larger courses will tend to set the norms, due to their larger enrollments.

Course-Weighted Norms

The course-weighted norms, on the other hand, aggregate data so as to "weight" each course equally. The results for a small course will count the same as the results for a large course. The raw counts for each course are converted to percentages, and these percentages are aggregated for the entire department. By aggregating percentages instead of raw counts, the results for a small course will count the same as the results for a large course. This allows another view of the response distribution from the perspective of "How would the results look if class size were factored out?".


The Labeling of the Summary Lines of Statistics

This course current quarter
Relative frequency distribution (expressed in terms of percentages), mean and median for this course current quarter.
Dept XXXXX current quarter
Relative frequency distribution (expressed in terms of percentages), mean and median for all instructors in the department of the same rank as the instructor teaching the course (Faculty or Associate or TA) who used that item this quarter. This norm would reflect undergraduate courses if "(UG)" is displayed or graduate courses if "(GR)" is displayed.
Dept XXXXX over time (most recent five year period)
Relative frequency distribution (expressed in terms of percentages), mean and median for all instructors of the same rank (Faculty or Associate or TA) in the department who have used the item within the last five years (including the current quarter). Recall, however that if an item has only recently been added to departmental surveys, its departmental norm may only reflect one or a few quarter's data. This norm reflects undergraduate courses if "(UG)" is displayed or graduate courses if "(GR)" is displayed.
Campus over time (most recent five year period)
Relative frequency distribution (expressed in terms of percentages), mean and median for all instructors of the same rank (Faculty or Associate or TA) in the department who have used the item within the last five years (including the current quarter). Recall, however that if an item has only recently been added to departmental surveys, its departmental norm may only reflect one or a few quarter's data. This norm reflects undergraduate courses if "(UG)" is displayed or graduate courses if "(GR)" is displayed.


Response weighting: 1 2 3 4 5   Blank Total Total    
    (a) (b) (c) (d) (e)   Response Students Courses Mean Median
This course current quarter 81% 16% 3%       0 37 1 1.2 1.0
    An Arrow An Arrow An Arrow                
    relative frequencies                

DESCRIPTION of "This course current quarter"
Relative frequencies The relative frequencies - expressed in terms of percentages - are of those students responding to the item. It does not take into account persons not responding to that particular item.
Blank Response This is a simple frequency count of students who turned in survey forms but did not respond to this item, or whose responses were not in the available response categories.
Total students This represents the total number of response sheets received for this survey, not the total number of students taking the course. To derive the base number for the relative frequencies , subtract the "Blank Response" number from the "Total Students" number. This number, in turn, may be used to derive the absolute number of students responding to each "response alternative" by multiplying the response alternative percent by its "response weighting" and dividing by 100.
Total courses Indicates the number of courses for which the data are being reported. For the line "This course current quarter" the total number of courses should always be 1.
Response Weighting Indicates the relative weight of each response alternative in computing the mean and median. The series of weights displays a "direction" or a "directional relationship" between the marked response alternatives. Note that lower numbers are usually "better".
Mean and Median These are based on:

1) only the number of students responding to the available response categories;
2) the response weightings.

Note that lower numbers usually "better".

Available DownloadsSize
example_ESCI_report.pdf104.2 KB
description_of_ESCI_output.pdf124.61 KB

consultation contacts

George Michaelsexecutive director2130 Kerr Hall
work805-893-2378
lisa berrysenior instructional consultant1130 Kerr Hall
work805-893-8395
mindy colininstructional consultant1130 Kerr Hall
work805-893-2828
Mary Lou Ramos database and ESCI administrator1130 Kerr Hall
work805-893-3523
Aisha Wedlaw ESCI assistant1124 Kerr Hall
work805-893-4278
Sarah Koepkeoffice manager 1130 Kerr Hall
work805-893-2972
faxfax: 805-893-5915