Use of the Behavior Assessment Tool in 18 Pilot Residency Programs

Authors

April D. Armstrong
Julie Agel
Matthew D. Beal
Michael S. Bednar
Michelle S. Caird
James E. Carpenter
S. Trent Guthrie, Henry Ford HealthFollow
Paul Juliano
Matthew Karam
Dawn LaPorte
J. Lawrence Marsh
Joshua C. Patt
Terrance D. Peabody
Karen Wu
David F. Martin
John J. Harrast
Ann E. Van Heest

Recommended Citation

Armstrong AD, Agel J, Beal MD, Bednar MS, Caird MS, Carpenter JE, Guthrie ST, Juliano P, Karam M, LaPorte D, Marsh JL, Patt JC, Peabody TD, Wu K, Martin DF, Harrast JJ, and Van Heest AE. Use of the Behavior Assessment Tool in 18 Pilot Residency Programs. JB JS Open Access 2020; 5(4).

Document Type

Article

Publication Date

10-1-2020

Publication Title

JB JS Open Access

Abstract

Background: The purpose of this study was to determine the feasibility and evaluate the effectiveness of the American Board of Orthopaedic Surgery Behavior Tool (ABOSBT) for measuring professionalism.

Methods: Through collaboration between the American Board of Orthopaedic Surgery and American Orthopaedic Association's Council of Residency Directors, 18 residency programs piloted the use of the ABOSBT. Residents requested assessments from faculty at the end of their clinical rotations, and a 360° request was performed near the end of the academic year. Program Directors (PDs) rated individual resident professionalism (based on historical observation) at the outset of the study, for comparison to the ABOSBT results.

Results: Nine thousand eight hundred ninety-two evaluations were completed using the ABOSBT for 449 different residents by 1,012 evaluators. 97.6% of all evaluations were scored level 4 or 5 (high levels of professional behavior) across all of the 5 domains. In total, 2.4% of all evaluations scored level 3 or below reflecting poorer performance. Of 431 residents, the ABOSBT identified 26 of 32 residents who were low performers (2 or more < level 3 scores in a domain) and who also scored "below expectations" by the PD at the start of the pilot project (81% sensitivity and 57% specificity), including 13 of these residents scoring poorly in all 5 domains. Evaluators found the ABOSBT was easy to use (96%) and that it was an effective tool to assess resident professional behavior (81%).

Conclusions: The ABOSBT was able to identify 2.4% low score evaluations (

Level of Evidence: Level II.

PubMed ID

33244509

Volume

5

Issue

4