1.0 Introduction

1.1 Membership of the Committee

The revisited study committee on student ratings of instruction reconvened in Fall 1998. The committee included original members Dan Good, Bob Haney, Barry Joyner, Sue Moore, Jane Page, Fred Richter, and Cathy Swift plus new members June Alberto, Cordelia Douzenis, Chuck Johnson, Jeanette Rice, and Dan Webster. An additional member, Bryan Griffin, was added in 1999. Bob Haney was the original chair of the committee but he removed himself from the committee during fall semester 1999. Barry Joyner was asked to become the chair.

1.2 Committee’s Charge

The committee was charged with reviewing the current practice of student ratings of instruction including the instrument, administrative guidelines, and potential uses, and making recommendations about a new or revised instrument and about policy and practice in this area.

2.0 Analysis of Current Instrument

2.1 Rationale

The current instrument had been used for two years with no analysis on the quality of the instrument. Bryan Griffin from the College of Education agreed to provide analyses for the committee.

2.2 Internal Structure of the Current Ratings Instrument

Data from the current instrument were used to explore the internal structure of the current instrument for dimensionality. The current instrument was designed to measure instructor enthusiasm, clarity of presentation, instructor preparation, concern for students, fairness in evaluation, and instructor motivation of students. Factor analysis was performed to determine if the structure of the current instrument supported these dimensions. Results from almost 200,000 ratings from over 9,000 classes showed the concept of multiple dimensions was not supported. The current instrument appears to be measuring only a single dimension. The Executive Summary from the report can be found in Appendix A to this report.

2.3 Faculty Survey on Student Ratings of Instruction

Bryan Griffin surveyed faculty to determine their views of the student ratings process. Over 300 faculty returned surveys. In general, results showed that faculty do not believe the current instrument provides information to improve classroom instruction and most do not believe the instrument is a valid measure of teaching effectiveness. The Executive Summary from the report can be found in Appendix B to this report.

2.4 Chair Survey on Student Ratings of Instruction

Bryan Griffin surveyed department chairs to determine how they view and use student ratings. Eighteen chairs responded to the survey. Results showed that more than half of the chairs think the current instrument is a valid or partially valid measure of teaching effectiveness. Almost all think that student ratings are helpful in making administrative decisions. The Executive Summary from the report can be found in Appendix C to this report.

3.0 Conclusions and Recommendations

3.1 Recommended Instrument

3.1.1 The committee thinks a revision of the current instrument is needed. After reviewing much of the literature on student ratings and taking input from the faculty and chairs surveys, the committee recommends the University pilot an instrument which includes the following: 1) seven items related to the course; 2) eleven items related to the instructor or instruction; 3) seven background items; 4) items selected by the instructor or department from an item bank; and 5) two open-ended items. (See Appendix D.)

3.1.2 The committee recommends the instrument be piloted this summer (Summer, 2000) and that reliability and validity studies be conducted periodically.

3.2 Recommended Faculty Use of Student Ratings

3.2.1 Use the seven items related to the course and the eleven items related to the instructor for improving their teaching. Faculty members may wish to refine teaching and learning activities and syllabi on the basis of information gained from the student ratings.

3.2.2 Use the eleven items related to the instructor as documentation of teaching effectiveness. These may be used as evidence of teaching effectiveness for merit raises and promotion and tenure reviews. Student ratings may provide a record of growth by the faculty member over time. The changes made by the faculty member can also be documented.

3.2.3 Use the background items to help identify student perceptions prior to the course. Specifically, the first background item may help explain negative responses. Items two, three, four, and five may help explain why ratings were positive or negative. Items six and seven may identify student expectations about grading practices by the faculty member.

3.2.4 Additionally, the committee recommends that faculty members informally request feedback from students throughout the course and use their suggestions to improve the course that semester. One format could be to use the open-ended items from the instrument. These formative evaluations are recommended for improvement purposes, and not to be understood as an institutionally approved document. Also, the committee suggests the use of peer evaluations to supplement student ratings in determining teaching effectiveness. Faculty members should take advantage of the Faculty Mentor Program for teachers, especially recently appointed teachers.

3.3 Recommended Department Chair Use of Student Ratings

3.3.1 As an institution that values quality education for its students, Georgia Southern University has identified superior instruction as a priority. The Student Ratings Committee has developed an instrument to assist with evaluation of teaching effectiveness of faculty. While the committee recognizes differences in academic disciplines regarding what constitutes effective teaching, it recommends the following general guidelines to assist department chairs with the interpretation and use of these student ratings of instruction.

3.3.2 Use multiple inputs to judge teaching effectiveness. While the committee recognizes the importance of student ratings, other methods of identifying exemplary teaching should be used to supplement this information (e.g., course syllabi, peer review, self-evaluation, etc.).

3.3.3 Consider the two-fold purpose of student ratings of instruction; this instrument provides information to administrators for decision-making purposes (e.g., merit pay, tenure and promotion) and gives instructors diagnostic feedback for the improvement of teaching. Selected global items on the instrument (e.g., Items seven and eighteen) might be more appropriate for summative decisions while other specific items (e.g., Items 1 - 6) might be used strictly for faculty development.

3.3.4 Consider student ratings of instruction for a particular course in light of several factors, including level of class, class size, perceived difficulty of subject matter, method of delivery (distance learning, off-campus, internet, etc.), required or elective, etc. In addition, chairs should avoid penalizing faculty for innovation as experience dictates that changes in instructional techniques are often initially associated with lower student ratings. Faculty members should be given the opportunity to respond to their ratings to perhaps explain the context of an unusual set of ratings.

3.3.5 View student ratings of instruction in a criterion referenced rather than a norm referenced fashion (i.e., every faculty member within a department should be compared to the same standard, not relative to each other). For example, based on student ratings and other indices, a department might develop criteria for placing faculty members in categories to represent levels of teaching effectiveness - outstanding, satisfactory, and unsatisfactory. Due to the tendency for evaluators to focus on numbers, and because mean scores on items for student ratings are only one of several factors that represent teaching effectiveness, we strongly recommend that mean scores on items for student ratings not be seen outside of the department.

3.3.6 Discuss with faculty the role of student ratings of instruction in administrative decisions so that all faculty are aware of the amount of importance placed on ratings in situations such as tenure and promotion and merit pay decisions.

3.3.7 Recognize that flexibility is key with regard to use of student ratings of instruction. Each department has the best information regarding the meaningfulness of student ratings for its respective faculty. The faculty and chair of each department should work together to determine how student ratings will be used in that particular department, and all faculty should be made aware of resulting departmental guidelines regarding use of these ratings.

3.4 Administration of the Student Ratings Instrument

3.4.1 The committee recommends that the current practice of administering the instruments be continued. Ratings are given within the last two weeks of the semester. The instructor should not be present during the administration. A faculty colleague or graduate assistant administers the ratings during the first 20 minutes of class and not at the end of a class session. The responses should be given to the faculty member early the next semester. Ideally, all written comments should be typed before given to the faculty member.