Implement a guideline regarding the use of effective feedback in the School of Medicine curriculum.
The Bridges Curriculum evaluation approach is designed to be iterative and builds on UCSF’s current rigorous evaluation program. The Bridges Curriculum evaluation plan uses both short- and long-term processes and includes both quantitative and qualitative collection. The data collection methods are represented in the figure above.
Q. What are the characteristics of effective feedback?
A. Feedback is focused on behavior rather than on the person. It is important that the evaluator refer to what a person does rather than to what s/he thinks the person is. Feedback should be specific rather than general. To be told that a small group facilitator is "dominating" will probably not be as useful as to say that "She was not listening to what the others said, but I felt I had to agree with her since she is an instructor."
Feedback should only be provided to help, not hurt, the person receiving it. It should be directed toward behavior that the receiver can improve. Feedback is destructive when it serves one's own needs, such as venting.
Feedback is about what or how something is said or done, not why it is done. It is risky to assume that we know why a person says or does something, or what he "really" means, or what she is "really" trying to accomplish. If we are uncertain of the person's motives or intent, this uncertainty itself is feedback and should be revealed. For more examples on providing constructive feedback, please review the Educational Evaluations Orientation Screen Lesson for F1 students here.
Q. What kinds of questions and ratings scales are used in the UCSF SOM evaluation forms?
A. Evaluations are composed of objective questions with a 5-point response scale and a section for comments. Our instructors and course directors strive to earn ratings of 4 and above on this scale. Any score below 4 is considered an area for improvement. The following is a description of the 5-point scale and how the school interprets each value. While "3 = Good", an educator interprets a score of 3 as equivalent to a solid 75 (just above passing) on a 100 point exam. Remember to be honest and thoughtful in your evaluations and to give scores that are appropriate. A rating of "1 = Poor" means that the instructor or course-related item is not conducive to student learning or wellbeing. A rating of "5 = Excellent" means that the content or instructor quality is the best it can be. Ratings of 2 and below are considered low and will trigger notifications to course and clerkship directors alerting them to the low rating.
Q. What is the software used for evaluation purposes?
A. We are in the process of migrating from E*Value to MedHub. For more information about the migration, please see the MedHub project wiki.
MedHub is the online evaluation tool for Bridges, starting with the Class of 2020 in Foundations 2. MedHub is accessible on your MyAccess Portal using your Single Sign-On login credentials.
E*Value is the online evaluation tool used to manage evaluations for the School of Medicine until fall 2018. E*Value generates e-mail notifications specific to each student with a hyperlink to a list of evaluation assignments. In addition to email notifications, students can access E*Value from the Current Students homepage. You can retrieve your login on the E*Value homepage via the “Forgot Password?” link and instructions on the next page.
In some instances, Qualtrics is another online survey tool that will be used for evaluation purposes. In such instances, the evaluations will bear the following statement: This survey is endorsed by the School of Medicine for the purpose of improving our programs and services.
Q. Is it possible for an instructor to get the name of a student who completed a specific evaluation?
A. Instructors can never find out who the student is, UNLESS a student chooses to identify her/himself. However, preceptor-student relationships are one-on-one and thus preceptors will usually know who the student evaluator is even though student names are not on the completed evaluation.
Q. What happens to our evaluations after they are submitted? Do course directors and faculty actually listen to our comments?
A. Yes. Course and faculty evaluations are taken very seriously by educational leadership for ongoing course improvement, as well as by departments for faculty promotion and tenure decisions. For example, faculty are not asked to return to teach if their low teaching scores do not show improvement. And, to ensure optimal learning, course content has been moved to other blocks and other times of the year as a result of student feedback on course evaluations.
The School is committed to addressing the issue of mistreatment of students by residents and faculty. The Associate Dean for Student Affairs monitors evaluations of residents and attendings who receive low scores ("1" and "2") by students on the two Respect questions ("I was treated with respect by this attending" and "I observed others (students, residents, staff, patients) being treated with respect by this attending"). These low scores are brought to the attention of the departments in order to provide feedback to individual residents and attendings, and to address systems issues. A rating of "3" or below on the Respect questions will require student to provide an explanation so that the School can better address any mistreatment issues.
Q. Students feel reluctant to give honest evaluations for fear of individually directed retaliation. What systems are in place to ensure that students are not subjected to repercussions from writing constructive comments in evaluations on faculty and residents?
A. Students' fears of repercussions have been heard. That is why the course and clerkship directors and the Educational Evaluations unit have developed ways to ensure that students are protected. For starters, faculty cannot see student names on evaluations. Secondly, clinical faculty cannot see their evaluations until they have completed a reciprocal evaluation about the student. Third, a confidential comment box is available on clinical educator forms to allow for comments students do not feel comfortable sharing directly with the person being evaluated. In E*Value, student evaluations are not available to the faculty within a designated series of days in order to enhance the anonymity of the evaluations by aggregating more ratings and comments. Additionally, students are encouraged to use the Supporting A Fair Environment (SAFE) form to report mistreatment that they do not feel comfortable sharing in E*Value or MedHub. This information is shared with the Associate Deans and clerkship/course director(s).
Q. Can students evaluate faculty members they are not formally assigned to evaluate?
A. While the School of Medicine takes measures to reduce the evaluation burden on students, students are provided an opportunity to evaluate all faculty members and residents with whom they work. There are many ways for students to provide feedback but it may be difficult, at times, to navigate through the process. On occasion, a student will work with a stellar or problematic faculty member or resident but is not asked to evaluate that individual, as each course and clerkship may have slightly different procedures for evaluation assignments. In that situation, students can contact the course or clerkship coordinator to request for an evaluation. Students can also turn to the SAFE (Supporting A Fair Environment) form to share their concerns confidentially or anonymously. This information on the SAFE Form is sent to the course or clerkship director and the Associate Deans.This information is sent to the course or clerkship director and the Associate Deans.
Q. What will happen to students’ E*Value and MedHub accounts upon graduation?
A. Upon graduation student access to E*Value and MedHub will be deactivated. It is recommended that all students download and save to their own records evaluations that they might need in the future.
Q. What is expected of students in terms of evaluations in Foundations 1? How does the assignment of evaluations work in Foundations 1?
Students are expected to complete all evaluations assigned to them. Students who complete less than 70% of the evaluations assigned to them during a course will be contacted by the Educational Evaluations Unit and encouraged to complete at least 70% of their evaluations by the end of the subsequent course.
All students must evaluate their small group leaders, coaches, CMC experience, and other program evaluations. However, other evaluations are structured to minimize the number of evaluations a given student has to complete.
Specifically, the overall class is divided into randomized subsets to complete the full set of evaluations for each Foundations 1 course, which includes the overall course, lecturer, and lab instructor evaluations. Randomized subsets of students will also be asked to evaluate educational interventions for Foundations 1.
Melissa Nakapa'ahu, in the Educational Evaluations Unit, is responsible for monitoring each student's response rate and ensuring that each student meets the minimum 70% evaluation completion requirement. While she can monitor what percentage of evaluations a student has completed, she does not have access to student names associated with evaluations when reviewing this compliance. Students’ evaluation compliance is regularly shared with Coaches and Associate Deans for the purpose of supporting students to meet the Professionalism competency. If a student continues to fall below the 70% threshold by the second course, the Associate Dean(s) will contact the student and strategize with the student about how best to meet this competency of Professionalism.
Q. How do evaluations work in the clinical years?
A. For a general overview of teaching and clerkship evaluations in Bridges Foundations 2, please review the orientation to evaluations in Bridges Foundations 2. In addition, each clerkship will have its own specific procedures on evaluations that you will learn from the clerkship. Evaluations are done in MedHub.
Q. When students submit evaluations are they sent directly to the clinical educator and is our confidentiality protected?
A. During clinical training, evaluations are set to be "reciprocal" between students and clinical educator, such that the educator cannot view their own evaluation until they have completed a reciprocal assessment of the student. Student names are never revealed to educator on their evaluations unless student chooses to identify themselves. The Clinical Educator form includes a confidential comment box for feedback if students do not feel comfortable sharing directly with the person being evaluated. Confidential comments are visible to the clerkship director and the program director with specific oversight.
Q. How many students typically evaluate an attending?
A. The number of students who complete an evaluation is clerkship dependent. Each clerkship has rules for how they assign evaluations.
Q. When an attending gets feedback, is it marked as being from a student vs. resident vs. intern vs. sub-I?
A. Educators can view their teaching evaluations in an aggregate view. The evaluation data are distinguished by the form types (i.e. student of faculty vs. resident of faculty) and the courses and programs with which the evaluation data are associated, but the evaluation data do not reveal who the student evaluator is.
Q. How are student comments made anonymous and reported to residents and attendings?
A. Residents and attendings see all the evaluations completed about them. The students' names are suppressed however, so they don't know which student said what when they view their aggregated comments.
Confidential Reporting Resources
Bridges Curriculum Debrief Survey
In addition to formal evaluations in E*Value and MedHub, students are encouraged to share their positive or negative experience in the Bridges curriculum in an ongoing basis via the Bridges Curriculum Debrief Survey. The comments provided on this survey will be anonymous unless students choose to identify themselves. These comments will go directly to the Associate Dean for Curriculum and other curriculum leaders of the specific areas that students designate.
Click here to fill out the Bridges Curriculum Debrief Survey.
Reporting Duty Hours Violations and/or Medical Student Mistreatment
Use the “SAFE” (Supporting a Fair Environment) reporting mechanism—a single form for you to confidentially report mistreatment and/or duty hours violations. This reporting mechanism supports the RESPECT questions distributed through our course evaluation system and managed by your course curriculum team. tiny.ucsf.edu/safe
If you have any questions regarding School of Medicine evaluations, please contact our Educational Evaluations help desk at [email protected]