INTRODUCTION: During surgical residency, trainees are expected to master all the 6 competencies specified by the ACGME. Surgical training programs are also evaluated, in part, by the residency review committee based on the percentage of graduates of the program who successfully complete the qualifying examination and the certification examination of the American Board of Surgery in the first attempt. Many program directors (PDs) use the American Board of Surgery In-Training Examination (ABSITE) as an indicator of future performance on the qualifying examination. Failure to meet an individual program's standard may result in remediation or a delay in promotion to the next level of training. Remediation is expensive in terms of not only dollars but also resources, faculty time, and potential program disruptions. We embarked on an exploratory study to determine if residents who might be at risk for substandard performance on the ABSITE could be identified based on the individual resident's behavior and motivational characteristics. If such were possible, then PDs would have the opportunity to be proactive in developing a curriculum tailored to an individual resident, providing a greater opportunity for success in meeting the program's standards.METHODS: Overall, 7 surgical training programs agreed to participate in this initial study and residents were recruited to voluntarily participate. Each participant completed an online assessment that characterizes an individual's behavioral style, motivators, and Acumen Index. Residents completed the assessment using a code name assigned by each individual PD or their designee. Assessments and the residents' 2013 ABSITE scores were forwarded for analysis using only the code name, thus insuring anonymity. Residents were grouped into those who took the junior examination, senior examination, and pass/fail categories. A passing score of >= 70% correct was chosen a priori. Correlations were performed using logistic regression and data were also entered into a neural network (NN) to develop a model that would explain performance based on data obtained from the TriMetrix assessments.RESULTS: A total of 117 residents' TriMetrix and ABSITE scores were available for analysis. They were divided into 2 groups of 64 senior residents and 53 junior residents. For each group, the pass/fail criteria for the ABSITE were set at 70 and greater as passing and 69 and lower as failing. Multiple logistic regression analysis was complete for pass/fail vs the TriMetrix assessments. For the senior data group, it was found that the parameter Theoretical correlates with pass rate (p < 0.043, B = -0.513, exp(B) = 0.599), which means increasing theoretical scores yields a decreasing likelihood of passing in the examination. For the junior data, the parameter Internal Role Awareness correlated with pass/fail rate (p < 0.004, B = 0.66, exp(B) = 1.935), which means that an increasing Internal Role Awareness score increases the likelihood of a passing score. The NN was able to be trained to predict ABSITE performance with surprising accuracy for both junior and senior residents.CONCLUSION: Behavioral, motivational, and acumen characteristics can be useful to identify residents "at risk" for substandard performance on the ABSITE. Armed with this information, PDs have the opportunity to intervene proactively to offer these residents a greater chance for success. The NN was capable of developing a model that explained performance on the examination for both the junior and the senior examinations. Subsequent testing is needed to determine if the NN is a good predictive tool for performance on this examination. (C) 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

BACKGROUND: The selection of residents for any program and their evaluation for success is an inexact science. Errors can prove costly, disruptive, and potentially damaging to training programs, and personal and professional setbacks can occur for resident applicants. A method was sought to determine the intangible characteristics of applicants to a general surgery residency program, particularly an assessment of behavior and motivation. The hypothesis was that such information could contribute to a more objective analysis of how well an applicant might fit into a program and its culture, and therefore improve the residents' chance for success and reduce the attrition rate.METHODS: Applications were screened by the Program Director and selection committee according to departmental standards. Those applicants who were offered the opportunity for interview were asked to complete an on-line survey that assessed behavioral style, intrinsic motivators, and dimensional balance. The assessment is known as the TriMetrix Personal Talent Report (TriMetrix; Target Training International, Ltd; TTI, Phoenix, AZ). An initial job. benchmark was constructed from data based on surveys of current residents and faculty, and from interviews held with Subject Matter Experts (SMEs) familiar with the demands of the position and the qualities necessary for success. Resident selection was carried out as has been done historically within the program. An independent contractor, who was blinded to the ranking by the program, presented an applicant list based on the candidate's Personal Talent Reports against the job benchmark. The ranking lists were then compared.RESULTS: Of the 535 applications received, interviews were offered to 112, and 77 interviews were conducted. Seventy-five online TriMetrix (TTI) assessments were completed by the applicants. Rank lists developed independently by the program and by the consultant were compared, with obvious discrepancies. Overall there was little concordance between the two lists, suggesting that the TriMetrix (TTI) assessment measures something different. The job benchmark identified different behavioral styles among the most successful of the current residents, suggesting that a diversity of natural behavior does not preclude success in the program.CONCLUSIONS: Objective data regarding an individual's personal style can be used to identify applicants who match with a training program's job benchmark. Factors predictive of success specific to our program include an independent desire for knowledge, a commitment to the service of others, and a view of the world with a sense of direction and purpose. The diversity of our current residents' styles as identified by this analysis indicates that many different individuals can be successful. While the instrument can provide important information regarding elements that contribute to successful performance, it is weighted as one essential component utilized in conjunction with other tools. (J Surg 68:534-541. (C) 2011 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)