In the last post, I shared a paper about extracting social determinants of health (SDoH) from EMR. Continuing with the SDoH theme, this week’s paper is about using them to improve accuracy and equitability of predictive models.
TL;DR: The researchers develop models to predict in-hospital mortality among heart failure patients. First, they show that simple, off-the-shelf ML tools are significantly more accurate than standard risk scores. Second, taking race and area-level SDoH like median household income improved performance further, particularly for Black patients.
Why should you care? When considering machine learning applications, healthcare organizations increasingly need to evaluate equitability and not just performance, and ensure that models don’t persist bias due to historical patterns. Incorporating SDoH can help improve both the accuracy and equitability of a model.
Worth your attention: Mortality is a rare outcome (low single digits) even among hospital patients. The number of observed deaths is small, which makes comparing models more challenging. This paper uses metrics targeted specifically for rare outcomes. This is good practice. Always ask which metrics are being used to assess model performance, especially if only standard-fare metrics like accuracy and AUC are being reported.
One caveat: SDoH typically come from a different data source than clinical data. Whenever two data sources are combined, special care is needed. Perhaps SDoH are more likely to be available for a certain patient population? This could affect both the accuracy and equitability of a model. In this particular work, more information on this topic would be helpful.