top of page
  • Avni Gulrajani

A Step Forward: How a New Evaluation System is Reducing Bias in Medical Training

Introduction: The medical profession, revered for its dedication to saving lives and promoting health, is not immune to the challenges of bias and inequality. A recent study from Harvard Kennedy School has shed light on a significant shift in the landscape of medical training – a reduction in ratings bias against minority group residents following the implementation of a new evaluation system. This change marks a crucial step towards equity in medical education and, subsequently, in healthcare delivery.

The Problem of Bias in Medical Training: For years, Black, Latino, and Asian medical residents in the United States have reported experiences of bias during their residency evaluations. These biases not only affect their training experience but also have long-term implications on their career advancement and representation in leadership roles within the medical field. Before 2014, when the Milestone evaluation system was introduced, the performance evaluations of hospital residents were largely subjective, allowing personal biases to influence the outcomes.

The Milestone Evaluation System: The Milestone system, implemented in 2014, aimed to address these shortcomings by reducing reviewer discretion and introducing more structured measurements. This system placed greater emphasis on clinical competency committee ratings and provided clear performance expectations and guidelines to increase the number and diversity of evaluators.

Study Findings: The study compared bias trends in performance evaluations of nearly 60,000 residents before and after the new system's adoption. The results are telling:

  • During the pre-Milestone period (2008-2013), ratings biases against minoritized groups were significant. Underrepresented in Medicine (URiM) residents faced a bias of -0.40 standard deviations (SDs), U.S.-born Asian residents -0.24 SDs, and non–U.S.-born Asian residents -0.36 SDs.

  • Post-Milestone (2015-2020), these biases decreased significantly for all groups, except U.S.-born Black residents, who still faced considerable bias (-0.26 SDs), although lower than before.

These findings indicate a substantial deviation from the pre-Milestone linear bias trends, suggesting that the Milestone ratings system has been effective in reducing bias.

My Thoughts:

Reflecting further on the Harvard Kennedy School study's findings, several additional thoughts come to mind:

  1. Cultural Competency in Evaluation: The study highlights the need for cultural competency in evaluation processes. Evaluators should be trained not only to recognize their biases but also to understand and appreciate the diverse cultural backgrounds of residents. This could lead to more empathetic and accurate assessments of their skills and competencies.

  2. Impact on Patient Care: The implications of this study extend beyond the training environment. By reducing bias in the evaluation of medical residents, we can expect a more diverse pool of physicians, which is crucial for patient care. Diverse medical teams have been shown to better understand and treat patients from various backgrounds, potentially leading to improved health outcomes.

  3. Mentorship and Support Systems: The study also indirectly points to the importance of mentorship and support systems for minority medical residents. Institutions should consider enhancing mentorship programs, ensuring that underrepresented residents receive guidance and support from mentors who understand their unique challenges and experiences.

  4. Longitudinal Follow-Up Studies: To truly gauge the effectiveness of the Milestone system, it would be beneficial to conduct longitudinal follow-up studies. Such studies could assess the career trajectories of residents who trained under this system to see if reduced bias in training translates to more equitable opportunities in their professional careers.

  5. Replicability in Other Fields: The success of the Milestone system in reducing bias suggests that similar structured evaluation systems could be beneficial in other fields and industries. There's potential for this approach to be a model for other professions grappling with bias and inequality issues.

  6. Feedback Mechanisms for Residents: Incorporating robust feedback mechanisms for residents could further enhance the evaluation process. Allowing residents to provide anonymous feedback about their experiences with evaluators might offer insights into areas where bias still exists and needs to be addressed.

  7. Global Implications: The study, while focused on the U.S., has global implications. Medical training institutions worldwide can learn from these findings, adapting the Milestone system or similar approaches to their cultural contexts to reduce bias in their own training programs.

In conclusion, the Harvard Kennedy School study opens up several avenues for thought and action. It's not just about reforming evaluation systems but about fostering a more inclusive, empathetic, and effective medical training environment that ultimately benefits the entire healthcare ecosystem.


bottom of page