Equity in Assessment Guidelines and Checklist

Purpose:Assessment for learning

To outline good assessment practices and strategies to avoid harmful bias in assessments

Background:

Written assessments of student performance are important to document learning, provide feedback, determine achievement of expected objectives and milestones, and contribute to the residency selection process. Unfortunately, research shows that assessments of performance commonly reflect biases that disadvantage women and learners from groups underrepresented in medicine (UIM). Faculty and resident adherence to best practices in learner assessment and conscious effort to avoid harmful bias are essential to assess all students fairly, accurately, and equitably.Medical students do blood pressure checks and and dental students do screenings with community members at the Southeast Community Health Fair.  Med student Dereck Paul talks to participant Mary Crenshaw before her blood pressure screening.

Getting Started:

  • Participate in training about bias in assessment, diversity, equity, and inclusion
  • Reflect on your own potential biases: Evidence shows that bias based on gender and race/ethnicity negatively affects performance assessments. Drawing awareness helps you to avoid the negative effects of bias
  • Establish a relationship and aim to know your student as an individual. Know their name and how they want to be addressed, ask about their learning goals and background, invite them to share something about themselves and what they bring to medicine if they wish, and share something about yourself
  • Observe your student with patients and the team, not just on rounds

For more information, contact Karen Hauer, MD, PhD; karen.hauer@ucsf.edu.

Use the checklist below as a guide. Additional sections after provide best practices and useful strategies. All are available for download.Carolina Berg, NP, Urology with Samuel Washington, Resident in Urology at Benioff Children's Hospital, Mission Bay

Review Equity in Assessment Checklist
Equity in Assessment Checklist (PDF)

Problems that Lead to Biased, Inequitable Assessment

Recommendations: Checklist of Good Practice for Assessment of Students

Performance expectations are unclear to students

Supervisors do not observe students’ activities with patients or recognize the unique contributions of UIM students to their patients

  • Describe observed behaviors in written evaluations: after you watch the student with a patient or with the healthcare team, or review the student’s written work

  • Watch for and ask about student’s unique contributions in patient care: advocacy for patients, relationships with patients and families, ‘behind the scenes’ work outside of rounds

Comparing students to other students creates competition and inequity

  • Describe student performance based on the expectations for your clerkship; use competency-based language and reinforce the goal that all students achieve competence

Characterizing performance based on effort, personality, student background or interpersonal traits. For non-majority learners, evaluations may lack skills-based language.

  • Use competency-based language: describe patient care, knowledge, communication, professional behaviors, responsiveness to feedback, work in the healthcare system, interprofessional interactions

  • Review your evaluations for language that may promote stereotypes based on student background or characteristics and remove it.

Growth and improvement are seen as signs of weakness

  • Appreciate what students have learned and value growth: ask the student about their learning goals, describe their achievement of those goals

Narratives are shorter for women or minority students; length cues favoritism

  • Written comments should be about one paragraph

Muted language (faint praise such as ‘solid’ or ‘adequate’) and differential use of superlatives for women and UIM students

  • Pay attention to your use of adjectives. Use superlative language consistently across all students with descriptions of performance

Commenting upon personal life (a tendency for describing women and UIM students) without reference to clinical performance

  • Focus on observed behaviors in the clinical context; highlight unique contributions to patient care. If relevant, comment on how a student may demonstrate a particular strength such as resilience or time management.  

Avoiding Stereotypes and Bias in Assessment of Learner Performance
Avoiding Stereotypes and Bias in Assessment of Learner Performance (PDF)

Narrative evaluations can contain bias. Focusing on traits associated with certain groups based on race/ethnicity, gender, or other characteristics reinforces stereotypes and can ignore other aspects of learner performance. Avoid these stereotypes by focusing on observed behaviors and describing competency-based performance. Scan your written evaluation to check for any unintended stereotypes.

Caution: avoid this language Analysis Instead, consider using this language
‘She was quiet yet participatory.’ ‘Quiet’ can be associated with gender or race/ethnicity. ‘She listened well; she participated thoughtfully.’
’She was warm, caring and empathic.’ (as the only descriptors) These are strengths but are also the characteristics that evaluators tend to focus more on for women than men; the evaluator should comment on other competencies as well. Describe clinical skills, knowledge, and interactions with patients and team.
‘He worked hard through the rotation.’ Effort is commendable; it is also important to describe the performance and connect effort to accomplishments. ‘Due to his hard work creating a discharge plan, our team was able to discharge the patient safely to home.’
 

‘He did well despite his many outside responsibilities.’

‘Her knowledge grew as she balanced outside family responsibilities.’

Qualified language detracts from the student’s accomplishments.
 

‘He contributed to the team by doing….’

‘She gained knowledge and applied it to her clinical practice.’

‘Compared to other people with his background, his performance was…’ Avoid focus on comparing students to other students particularly in a stereotyped way; focus on observed performance. ‘He successfully performed the following key clinical tasks for this clerkship….’
‘He contributed a lot to the team despite being gone from the rotation for weekly appointments.’
 
Student with approved accommodation to attend medical appointments cannot be penalized for this.
 
Focus narrative on what student did on the service, not when the student was not there.
 
‘After initially not having a very strong fund of knowledge of our specialty, they seemed to read a lot over the course of the rotation.’
 
The evaluator may be trying to comment on improvement but the focus should be on the student’s achievement by the end of the rotation, not just on the deficit. ‘The student read a lot and achieved the expected fund of knowledge by the end of the rotation. They applied their reading effectively to patient problems in their notes and case discussions.’

Watch for and ask the student and other team members about student contributions you may not have observed.

Student Contribution Competencies Description
Spending time with a patient explaining a diagnosis that was unclear to the patient on rounds Interpersonal and communication skills
Patient care
‘The student spent extra time with the patient explaining his diagnosis and answering questions to ensure his understanding and provide reassurance.’
Working on discharge planning to ensure that the patient will receive all of her medications, have secure housing, and understand her follow up appointments Systems-based practice
Interprofessional collaboration
‘The student coordinated discharge planning for a complex patient discharge by working with the with the pharmacist, case manager, and resident to ensure that the patient understood the discharge plans. This included finding a pharmacy to provide all of the prescribed medications, securing temporary housing and explaining the plans to the patient to confirm her understanding.’
Answering questions from a patient’s family about an upcoming procedure Interpersonal and communication skills ‘When a patient’s family had questions about an upcoming procedure, the student listened to their questions and concerns with empathy. The student then coordinated with the resident to answer all of their questions accurately and thoroughly.’

References

  1. Addressing Harmful Bias and Eliminating Discrimination in Health Professions Learning Environments. Macy Foundation Conference Recommendations 2020.  https://macyfoundation.org/assets/reports/publications/jmf_2020_confsummary_fin.pdf
  2. Lucey CR, Hauer KE, Boatright D, Fernandez A. Medical education’s wicked problem: achieving equity in assessment for medical learners. Academic Medicine, 2020. 
  3. Mueller AS, Jenkins TM, Osborne M, Dayal A, O’Connor DM, Arora VM. Gender differences in attending physicians’ feedback to residents: A qualitative analysis. J Grad Med Educ. 2017;9:577–585.
  4. Rojek AE, Khanna A, Yim JWL, Gardner R, Lisker S, Hauer KE, Lucey C, Sarkar U. Differences in narrative language in evaluations of medical students by gender and under-represented minority status. J Gen Intern Med. 2019;34:684–691.
  5. Ross DA, Boatright D, Nunez-Smith M, Jordan A, Chekroud A, Moore EZ.  Differences in words used to describe racial and gender groups in medical student performance evaluations. PloS One. 2017;12(8):e0181659.  
  6. Teherani A, Perez S, Muller-Juge V, Lupton K, Hauer KE. A narrative study of equity in clinical assessment through the anti-deficit lens. Academic Medicine, 2020.     
Writing High-Quality Evaluations of Student Performance: Best Practices and Examples (PDFs)

F1 SMALL GROUPS

F2-CL CLINICAL CLERKSHIPS

More Resources

Review module on Bias and Microaggressions in Feedback

Additional Assessment, Evaluation, and Feedback Resources