Equity in Assessment Guidelines and Checklist

Purpose:Assessment for learning

To outline good assessment practices and strategies to avoid harmful bias in assessments

Background:

Written assessments of student performance are important to document learning, provide feedback, determine achievement of expected objectives and milestones, and contribute to the residency selection process. Unfortunately, research shows that assessments of performance commonly reflect biases that disadvantage women and learners from groups underrepresented in medicine (UIM). Faculty and resident adherence to best practices in learner assessment and conscious effort to avoid harmful bias are essential to assess all students fairly, accurately, and equitably.Medical students do blood pressure checks and and dental students do screenings with community members at the Southeast Community Health Fair.  Med student Dereck Paul talks to participant Mary Crenshaw before her blood pressure screening.

Getting Started:

  • Participate in training about bias in assessment, diversity, equity, and inclusion
  • Reflect on your own potential biases: Evidence shows that bias based on gender and race/ethnicity negatively affects performance assessments. Drawing awareness helps you to avoid the negative effects of bias
  • Establish a relationship and aim to know your student as an individual. Know their name and how they want to be addressed, ask about their learning goals and background, invite them to share something about themselves and what they bring to medicine if they wish, and share something about yourself
  • Observe your student with patients and the team, not just on rounds

For more information, contact Karen Hauer, MD, PhD; [email protected].

Use the checklist below as a guide. Additional sections after provide best practices and useful strategies. All are available for download.Carolina Berg, NP, Urology with Samuel Washington, Resident in Urology at Benioff Children's Hospital, Mission Bay

Review Equity in Assessment Checklist
Equity in Assessment Checklist (PDF)

Problems that Lead to Biased, Inequitable Assessment

Recommendations: Checklist of Good Practice for Assessment of Students

Performance expectations are unclear to students

Supervisors do not observe students’ activities with patients or recognize the unique contributions of UIM students to their patients

  • Describe observed behaviors in written evaluations: after you watch the student with a patient or with the healthcare team, or review the student’s written work

  • Watch for and ask about student’s unique contributions in patient care: advocacy for patients, relationships with patients and families, ‘behind the scenes’ work outside of rounds

Comparing students to other students creates competition and inequity

  • Describe student performance based on the expectations for your clerkship; use competency-based language and reinforce the goal that all students achieve competence

Characterizing performance based on effort, personality, student background or interpersonal traits. For non-majority learners, evaluations may lack skills-based language.

  • Use competency-based language: describe patient care, knowledge, communication, professional behaviors, responsiveness to feedback, work in the healthcare system, interprofessional interactions

  • Review your evaluations for language that may promote stereotypes based on student background or characteristics and remove it.

Growth and improvement are seen as signs of weakness

  • Appreciate what students have learned and value growth: ask the student about their learning goals, describe their achievement of those goals

Narratives are shorter for women or minority students; length cues favoritism

  • Written comments should be about one paragraph

Muted language (faint praise such as ‘solid’ or ‘adequate’) and differential use of superlatives for women and UIM students

  • Pay attention to your use of adjectives. Use superlative language consistently across all students with descriptions of performance

Commenting upon personal life (a tendency for describing women and UIM students) without reference to clinical performance

  • Focus on observed behaviors in the clinical context; highlight unique contributions to patient care. If relevant, comment on how a student may demonstrate a particular strength such as resilience or time management.  

Writing High-Quality Evaluations of Student Performance: Best Practices and Examples
Writing High-Quality Evaluations of Student Performance: Best Practices and Examples (PDF)

Best practices in writing evaluations:

  • Discuss expectations when you start working with a student.
  • Know the School’s competencies and the expectations for students in your clerkship.
  • Keep personal notes while working with the student of what you have observed and feedback you have discussed, to use when writing your summary evaluation.
  • Complete your written evaluations promptly, within a week of working with the student.
  • Describe specific behaviors and concrete examples in your evaluation.  
  • Discuss midpoint feedback using competency-based language. 
  • Build upon your midpoint feedback in your written evaluation and comment on student achievement of the expectations you discussed.

Well-written evaluations include comments that:

  • Use competency-based language
  • Are appropriate length: one paragraph
  • Illustrate behaviors with specific examples
  • Highlight growth and learning as strengths

(see description of what to avoid in written evaluations)

Examples of well-written evaluations 

  • Student brought a very good fund of knowledge and especially strong common sense and clinical-reasoning skills to her evaluations. She knew our inpatients very well and was fluent with the most important information. She was especially helpful to the team, with a keen perception of procedural skill that allowed her to participate in operating room procedures far above the usual skill level expected of a student. She brought uncommon maturity and situational awareness and these allowed her to function with an uncommon level of independence in the daily management of our patients. Although the statement is frequently overused, student did actually function in all respects at the level of a good intern. She was very efficient. Her H&Ps were remarkably thoughtful and captured even the complex nuances of lengthy discussions with our patients.
  • Student demonstrated an exceptional fund of knowledge and she applied her understanding at the bedside in managing our patients with the level of skill we typically see in a second-year resident. She was well prepared with all the information needed to formulate treatment plans and her ability to prioritize and integrate bits of evidence into a cohesive and logical story was especially noteworthy. Student also has a sensitive ear for elements that don't fit into the accepted narrative and digs in to uncover the truth when needed. She initiated literature searches to supplement her understanding and to address questions raised by the team. She has a high level of emotional intelligence and this helps her connect effectively with a wide variety of patients, and also to function as an outstanding team member. Student was both efficient and thoughtful in her management of our inpatients and in the busy clinics. She was generous in teaching other students and nurses in a kind, respectful manner. She was an eager learner and a quick study with deft technical ability. Her performance was exemplary.
  • Student is a bright, hardworking, and capable medical student who approached this rotation with diligence and attention. Patients responded well to his calm demeanor, and his interpersonal skills reflected his maturity and thoughtfulness. I greatly appreciated his enthusiasm for the work, and he asked excellent questions to enhance his understanding of his patients, systems issues, and the consultation model. He demonstrated a strong fund of knowledge throughout the rotation, and his thoughtfulness about clinical care was consistently reflected in his notes. His assessments accurately and coherently laid out the differential diagnoses for his patients and the reasoning behind them. He gave an extremely interesting and thought-provoking presentation on XX disorder. He was a true asset to our team, and an absolute pleasure to work with during this rotation.
  • Student was an undeniable asset to our team as a sub-i. Her enthusiastic and proactive attitude led her to engage in many patient care endeavors during her rotation, during which she proved herself to be a bright, capable and empathetic provider. She was always open and seeking feedback, and she was quick to incorporate it. Her presentations on various topics were always well researched, succinct and high yield for all those listening. She knew her patients well and was always a helpful and pleasant member of the team. For a patient with a complex discharge plan, she coordinated with the case manager, family and team members to ensure a safe discharge to home. We were particularly impressed by her assessment of a patient's literacy level, and not only accommodating in a very sensitive way by providing more picture-based patient education materials, but also bringing this to the attention of the team. She was always advocating for patients and reminding us of our inherent biases and assumptions in caring for and educating patients. She will definitely be a strong intern at whatever program is lucky enough to have her.
  • Student did a stellar job on her rotation. Attending physicians she worked with commented on her teamwork, broad knowledge, and impressive procedural skills. She thoroughly reviewed patient records ahead of time, gathered appropriately focused histories, performed a targeted physical examination, presented in a well-organized and thoughtful manner, and wrote excellent notes. She was able to independently synthesize the information to formulate her own assessment and plan, often requiring little modification by the attending. Her knowledge level exceeds expectations. She worked well with all members of the team.
  • Student performed exceptionally well during his rotation. His teams universally praised his performance, noting that he exhibited independence and skill at the level of an intern in the workup and management of his patients. He confidently led the team when rounding on his patients, and served in the role of the primary provider. His patient-care skills were accurate and reliable; his history-taking and physical examinations were appropriately detailed and thoughtful. For example, he reported only the pertinent positives and negatives in his oral presentations and regularly tailored his physical examinations to the chief complaint rather than simply running through a screening exam. Student was highly motivated, performed a tremendous amount of reading about his patients' disorders, and regularly used evidence from the medical literature in his patient management plans. Additionally, he added significant value to team discussions by bringing to rounds what he had learned; for example, when managing a patient with X condition, he read extensively on the disorder and how to best manage the patient, which he then taught to his team. He is poised, compassionate, and a true team player.


Avoid these comments in written evaluations of students:

  • comments are very brief and don’t cite specific, competency and behavior-based skills 
  • comments refer to a student’s future subspecialty or career path
  • comments include references to grading or test scores
  • comments are just a list of quotes or fragments of sentences and have not been synthesized into a well-written paragraph

Examples to avoid (as the only narrative comment)

  • Overall outstanding job on service. One of our best Sub-Is this year.
    • Problems: too brief. No performance-based comments.
  • Smart, enthusiastic, responsible.
    • Problems: too brief. No performance-based comments.
  • Student was professional and diligent during her rotation.
    • Problems: too brief. Does not address other competencies beyond professionalism.
  • Student was engaged and interested; she asks relevant questions.
    • Problems: too brief. Does not address important competencies including patient care.
  • Very helpful to the team. Integrated will with the residents and faculty. Will be a great (specialist) We are excited that he is pursuing (our field) as a specialty interest.
    • Problems: focuses more on predicting future and on interest in the field than on performance.
  • Excellent work on service. He was enthusiastic, considerate, and thoughtful.
    • Problems: does not explain what behaviors were excellent.
  • Student performed admirably during his rotation in our department and will make an outstanding house officer and eventually a superb medical practitioner. I sincerely hope that he will eventually pursue a career in (our specialty). He will be a clinician we can all be proud of!!??????
    • Problems: focuses more on predicting future and on interest in the field than on performance.  Does not explain what behaviors were ‘admirable’ or address competencies.
Avoiding Stereotypes and Bias in Assessment of Learner Performance
Avoiding Stereotypes and Bias in Assessment of Learner Performance (PDF)

Narrative evaluations can contain bias. Focusing on traits associated with certain groups based on race/ethnicity, gender, or other characteristics reinforces stereotypes and can ignore other aspects of learner performance. Avoid these stereotypes by focusing on observed behaviors and describing competency-based performance. Scan your written evaluation to check for any unintended stereotypes.

Caution: avoid this language Analysis Instead, consider using this language
‘She was quiet yet participatory.’ ‘Quiet’ can be associated with gender or race/ethnicity. ‘She listened well; she participated thoughtfully.’
’She was warm, caring and empathic.’ (as the only descriptors) These are strengths but are also the characteristics that evaluators tend to focus more on for women than men; the evaluator should comment on other competencies as well. Describe clinical skills, knowledge, and interactions with patients and team.
‘He worked hard through the rotation.’ Effort is commendable; it is also important to describe the performance and connect effort to accomplishments. ‘Due to his hard work creating a discharge plan, our team was able to discharge the patient safely to home.’
 

‘He did well despite his many outside responsibilities.’

‘Her knowledge grew as she balanced outside family responsibilities.’

Qualified language detracts from the student’s accomplishments.
 

‘He contributed to the team by doing….’

‘She gained knowledge and applied it to her clinical practice.’

‘Compared to other people with his background, his performance was…’ Avoid focus on comparing students to other students particularly in a stereotyped way; focus on observed performance. ‘He successfully performed the following key clinical tasks for this clerkship….’
‘He contributed a lot to the team despite being gone from the rotation for weekly appointments.’
 
Student with approved accommodation to attend medical appointments cannot be penalized for this.
 
Focus narrative on what student did on the service, not when the student was not there.
 
‘After initially not having a very strong fund of knowledge of our specialty, they seemed to read a lot over the course of the rotation.’
 
The evaluator may be trying to comment on improvement but the focus should be on the student’s achievement by the end of the rotation, not just on the deficit. ‘The student read a lot and achieved the expected fund of knowledge by the end of the rotation. They applied their reading effectively to patient problems in their notes and case discussions.’

Watch for and ask the student and other team members about student contributions you may not have observed.

Student Contribution Competencies Description
Spending time with a patient explaining a diagnosis that was unclear to the patient on rounds Interpersonal and communication skills
Patient care
‘The student spent extra time with the patient explaining his diagnosis and answering questions to ensure his understanding and provide reassurance.’
Working on discharge planning to ensure that the patient will receive all of her medications, have secure housing, and understand her follow up appointments Systems-based practice
Interprofessional collaboration
‘The student coordinated discharge planning for a complex patient discharge by working with the with the pharmacist, case manager, and resident to ensure that the patient understood the discharge plans. This included finding a pharmacy to provide all of the prescribed medications, securing temporary housing and explaining the plans to the patient to confirm her understanding.’
Answering questions from a patient’s family about an upcoming procedure Interpersonal and communication skills ‘When a patient’s family had questions about an upcoming procedure, the student listened to their questions and concerns with empathy. The student then coordinated with the resident to answer all of their questions accurately and thoroughly.’

References

  1. Addressing Harmful Bias and Eliminating Discrimination in Health Professions Learning Environments. Macy Foundation Conference Recommendations 2020.  https://macyfoundation.org/assets/reports/publications/jmf_2020_confsummary_fin.pdf
  2. Lucey CR, Hauer KE, Boatright D, Fernandez A. Medical education’s wicked problem: achieving equity in assessment for medical learners. Academic Medicine, 2020. 
  3. Mueller AS, Jenkins TM, Osborne M, Dayal A, O’Connor DM, Arora VM. Gender differences in attending physicians’ feedback to residents: A qualitative analysis. J Grad Med Educ. 2017;9:577–585.
  4. Rojek AE, Khanna A, Yim JWL, Gardner R, Lisker S, Hauer KE, Lucey C, Sarkar U. Differences in narrative language in evaluations of medical students by gender and under-represented minority status. J Gen Intern Med. 2019;34:684–691.
  5. Ross DA, Boatright D, Nunez-Smith M, Jordan A, Chekroud A, Moore EZ.  Differences in words used to describe racial and gender groups in medical student performance evaluations. PloS One. 2017;12(8):e0181659.  
  6. Teherani A, Perez S, Muller-Juge V, Lupton K, Hauer KE. A narrative study of equity in clinical assessment through the anti-deficit lens. Academic Medicine, 2020.   

Review module on Bias and Microaggressions in Feedback

Additional Assessment, Evaluation, and Feedback Resources