Study Evaluates Exactech’s Predict+ Decision Support Tool

Exactech reports a new study that demonstrates its machine learning clinical decision support tool, Predict+, creates fair and accurate outcomes predictions for shoulder arthroplasty patients of different ethnicity, sex and age. 

To evaluate fairness, the study quantified the accuracy of Predict+ when predicting clinical outcomes for anatomic total shoulder arthroplasty and reverse total shoulder arthroplasty in more than 8,000 patients, stratified by multiple patient sub-groups of age, gender and ethnicity. Comparing differences in accuracy between patient groups, the authors were able to identify precisely which patient groups received fair predictions and which patient groups received unfair predictions. The study reported that Predict+ was deemed fair for 98.6% of regression predictions, 99.4% of substantial clinical benefit (SCB) classification predictions, and 100% of minimal clinically important difference (MCID) classification predictions.

Predict+, part of Exactech’s Active Intelligence portfolio of smart solutions, was released in 2020 and was designed to better inform surgeons regarding the expected outcomes that can be achieved after shoulder arthroplasty, based on the clinical experience documented within the world’s largest single-shoulder prosthesis outcomes database, consisting of more than 15,000 patients.

Predict+ is available to surgeons worldwide on a limited basis.

“This new machine learning study is important because it proposes a new statistical test methodology to evaluate the fairness of AI predictions,” said Chris Roche, Exactech’s Sr. Vice President, Extremities, “and because it establishes the clinically relevant criteria that determine acceptable differences in prediction accuracy between shoulder arthroplasty patients of different demographic status. Perhaps the biggest barrier to adoption of machine learning-based clinical decision support tools is the perception that they are not fair and accurate for all patients. This new clinical outcome study is the first of its kind in the orthopaedic literature to evaluate if an AI prediction is fair.”

“Bias in AI clinical predictions can adversely impact decision making,” said Vikas Kumar, Exactech’s Vice President of Machine Learning. “Biased or unfair predictions are more likely to occur in patient groups that are underrepresented in the training data. As a result, it is critical to train and evaluate machine learning prediction tools against diverse datasets that are representative of all potential patients. These results were only possible because of the large volume of high-quality clinical data Exactech has collected with the Equinoxe shoulder system. Using our proposed machine learning testing methodology, we aim to provide the statistical determination of what makes a fair or unfair prediction for all machine learning clinical decision support tools.”

Source: Exactech