Evaluations Procedures

The Bridges Curriculum evaluation approach is designed to be iterative and builds on UCSF’s current rigorous evaluation program. The Bridges Curriculum evaluation plan uses both short- and long-term processes and includes both quantitative and qualitative collection. The data collection methods are represented in the figure above.

General Questions

Q. What are the characteristics of effective feedback?

A. Feedback is focused on behavior rather than on the person. It is important that the evaluator refer to what a person does rather than to what s/he thinks the person is. Feedback should be specific rather than general. To be told that a small group facilitator is "dominating" will probably not be as useful as to say that "She was not listening to what the others said, but I felt I had to agree with her since she is an instructor."

Feedback should only be provided to help, not hurt, the person receiving it. It should be directed toward behavior that the receiver can improve. Feedback is destructive when it serves one's own needs, such as venting.

Feedback is about what or how something is said or done, not why it is done. It is risky to assume that we know why a person says or does something, or what he "really" means, or what she is "really" trying to accomplish. If we are uncertain of the person's motives or intent, this uncertainty itself is feedback and should be revealed. For more examples on providing constructive feedback, please review the F1 Evaluations Orientation.

Q. What kinds of questions and ratings scales are used in the UCSF SOM evaluation forms?

A. Evaluations are composed of objective questions with a 5-point response scale and a section for comments. Our instructors and course directors strive to earn ratings of 4 and above on this scale. Any score below 4 is considered an area for improvement. The following is a description of the 5-point scale and how the school interprets each value. While "3 = Good", an educator interprets a score of 3 as equivalent to a solid 75 (just above passing) on a 100 point exam. Remember to be honest and thoughtful in your evaluations and to give scores that are appropriate. A rating of "1 = Poor" means that the instructor or course-related item is not conducive to student learning or wellbeing. A rating of "5 = Excellent" means that the content or instructor quality is the best it can be. Ratings of 2 and below are considered low and will trigger notifications to course and clerkship directors alerting them to the low rating. 

Q.  What is the software used for evaluation purposes?

A.  MedHub is the online evaluation tool used to manage evaluations and schedules in Foundations 2 and Career Launch. MedHub is accessible on your MyAccess Portal using your Single Sign-On login credentials. 

eValue is the online evaluation tool used to manage evaluations for the Foundations 1 phase. eValue generates e-mail notifications specific to each student with a hyperlink to a list of evaluation assignments.

In some instances, Qualtrics is another online survey tool that will be used for evaluation purposes. In such instances, the evaluations will bear the following statement: This survey is endorsed by the School of Medicine for the purpose of improving our programs and services.

Q. Is it possible for an instructor to get the name of a student who completed a specific evaluation?

A. Instructors can never find out who the student is, UNLESS a student chooses to identify her/himself. However, preceptor-student relationships are one-on-one and thus preceptors will usually know who the student evaluator is even though student names are not on the completed evaluation.

Q. What happens to our evaluations after they are submitted? Do course directors and faculty actually listen to our comments?

A. Yes.  Course and faculty evaluations are taken very seriously by educational leadership for ongoing course improvement, as well as by departments for faculty promotion and tenure decisions. For example, faculty are not asked to return to teach if their low teaching scores do not show improvement. And, to ensure optimal learning, course content has been moved to other blocks and other times of the year as a result of student feedback on course evaluations.

The School is committed to addressing the issue of mistreatment of students by residents and faculty.  The Associate Dean for Student Affairs monitors evaluations of residents and attendings who receive low scores ("1" and "2") by students on the two Respect questions ("I was treated with respect by this attending" and "I observed others (students, residents, staff, patients) being treated with respect by this attending").  These low scores are brought to the attention of the departments in order to provide feedback to individual residents and attendings, and to address systems issues. A rating of "3" or below on the Respect questions will require student to provide an explanation so that the School can better address any mistreatment issues.

Q. Students feel reluctant to give honest evaluations for fear of individually directed retaliation.  What systems are in place to ensure that students are not subjected to repercussions from writing constructive comments in evaluations on faculty and residents?

A. Students' fears of repercussions have been heard. That is why the course and clerkship directors and the Educational Evaluations unit have developed ways to ensure that students are protected. For starters, faculty cannot see student names on evaluations.  Secondly, clinical faculty cannot see their evaluations until they have completed a reciprocal evaluation about the student. Third, a confidential comment box is available on clinical educator forms to allow for comments students do not feel comfortable sharing directly with the person being evaluated. In eValue, student evaluations are not available to the faculty within a designated series of days in order to enhance the anonymity of the evaluations by aggregating more ratings and comments. Additionally, students are encouraged to use the SAFE (Supporting A Fair and Equitable Environment) form to report mistreatment that they do not feel comfortable sharing in eValue or MedHub. This information is shared with the Associate Deans and clerkship/course director(s).

Q. Can students evaluate faculty members they are not formally assigned to evaluate?

A. While the School of Medicine takes measures to reduce the evaluation burden on students, students are provided an opportunity to evaluate all faculty members and residents with whom they work. There are many ways for students to provide feedback but it may be difficult, at times, to navigate through the process. On occasion, a student will work with a stellar or problematic faculty member or resident but is not asked to evaluate that individual, as each course and clerkship may have slightly different procedures for evaluation assignments. In that situation, students can contact the course or clerkship coordinator to request for an evaluation.  Students can also turn to the SAFE (Supporting A Fair and Equitable Environment) form to share their concerns confidentially or anonymously. This information on the SAFE Form is sent to the course or clerkship director and the Associate Deans.This information is sent to the course or clerkship director and the Associate Deans.

Q. What will happen to students’ eValue and MedHub accounts upon graduation?

A. Upon graduation student access to eValue and MedHub will be deactivated. It is recommended that all students download and save to their own records evaluations that they might need in the future.

Foundations 1

Q. What is expected of students in terms of evaluations in Foundations 1? How does the assignment of evaluations work in Foundations 1?

Students are expected to complete all mandatory evaluations assigned to them. Students who complete less than 70% of the evaluations assigned to them during a course will be contacted by the Educational Evaluations Unit and encouraged to complete at least 70% of their evaluations by the end of the subsequent course. Please see the F1 evaluations orientation.

All students must evaluate their small group leaders, coaches, CMC experience, and other program evaluations.  However, other evaluations are structured to minimize the number of evaluations a given student has to complete.

Specifically, the overall class is divided into randomized subsets to complete the full set of evaluations for each Foundations 1 course, which includes the overall course, lecturer, and lab instructor evaluations. Randomized subsets of students will also be asked to evaluate educational interventions for Foundations 1.

The Educational Evaluations Unit is responsible for monitoring each student's response rate and ensuring that each student meets the minimum 70% evaluation completion requirement. While they can monitor what percentage of evaluations a student has completed, they do not have access to student names associated with evaluations when reviewing this compliance. Students’ evaluation compliance is regularly shared with Coaches and Associate Deans for the purpose of supporting students to meet the Professionalism competency. If a student continues to fall below the 70% threshold by the second course, the Associate Dean(s) will contact the student and strategize with the student about how best to meet this competency of Professionalism.

Methods by which students can provide feedback in the Bridges Curriculum.

• Report Medical Student Mistreatment/Work Hour Violations using the SAFE form
• Connect directly with your School of Medicine Deans
• Share curriculum feedback using the SAFE form

Clinical Years: Foundations 2 and Career Launch

Q. How do evaluations work in the clinical years?

A. For a general overview of teaching and clerkship evaluations in the clinical years, please review the orientation to evaluations. In addition, each clerkship will have its own specific procedures on evaluations that you will learn from the clerkship. Evaluations are done in MedHub.

Q. When students submit evaluations are they sent directly to the clinical educator and is our confidentiality protected?

A. During clinical training, evaluations are set to be "reciprocal" between students and clinical educator, such that the educator cannot view their own evaluation until they have completed a reciprocal assessment of the student. Student names are never revealed to educator on their evaluations unless student chooses to identify themselves. The Clinical Educator form includes a confidential comment box for feedback if students do not feel comfortable sharing directly with the person being evaluated. Confidential comments are visible to the clerkship director and the program director with specific oversight.

Q. How many students typically evaluate an attending?

A. The number of students who complete an evaluation is clerkship dependent. Each clerkship has rules for how they assign evaluations.

Q. When an attending gets feedback, is it marked as being from a student vs. resident vs. intern vs. sub-I?

A. Educators can view their teaching evaluations in an aggregate view. The evaluation data are distinguished by the form types (i.e. student of faculty vs. resident of faculty) and the courses and programs with which the evaluation data are associated, but the evaluation data do not reveal who the student evaluator is.

Q.  How are student comments made anonymous and reported to residents and attendings?

A. Residents and attendings see all the evaluations completed about them. The students' names are suppressed however, so they don't know which student said what when they view their aggregated comments.

Confidential Reporting Resources

General Education Feedback
In addition to formal evaluations in eValue and MedHub, students are encouraged to share their positive or negative experience in the Bridges curriculum in an ongoing basis via the Bridges Real Time Feedback. The comments provided on this survey will be anonymous unless students choose to identify themselves. These comments will go directly to the Associate Deans and other curriculum leaders of the specific areas that students designate.

Click here to fill out the SAFE form.

Reporting Duty Hours Violations and/or Medical Student Mistreatment

Use the SAFE (Supporting A Fair and Equitable Environment) form—a single form for you to confidentially report mistreatment and/or duty hours violations. This reporting mechanism supports the RESPECT questions distributed through our course evaluation system and managed by your course curriculum team.

For all methods for students to provide feedback: http://tiny.ucsf.edu/methodsforstudentfeedback

Accountable Dean or Director: Director of Program Evaluation

Approval Date and Governing Body:  August 6, 2018, Executive Committee

Last Updated: July 10, 2020

---

If you have any questions regarding School of Medicine evaluations, please contact our Educational Evaluations help desk at [email protected]