Welcome to Straight Talk about AI in Healthcare. In each post I explain a recent research paper in non-technical terms and highlight lessons for healthcare organizations. My focus is not necessarily the most inspirational work but the most practical insights: those that help you understand what you can do today and what should be on your radar for tomorrow.
Always feel free to reach out with any feedback and article suggestions, and if you are not on the mailing list, please subscribe!
Today’s post, like the previous one, is about work presented in December at the Machine Learning for Health workshop at the Conference on Neural Information Processing Systems (NeurIPS), arguably the most influential conference in the artificial intelligence world. The work, by a Finnish team, tackles the problem of predicting healthcare utilization using deep learning.
“Deep learning” is a family of algorithms inspired by the way neurons work in the human brain, which is why they’re also called “artificial neural networks”. All an “artificial neuron” does is check if a simple calculation on the data is positive or negative. In deep learning, artificial neurons are stacked in several layers (hence the term “deep”), each one using the output of the previous layer as its input.
The difficult part is figuring out the right configuration of layers and the specific calculation that each neuron should perform. That takes massive amounts of data and computational power, but once you have them, deep learning methods can vastly outperform more traditional approaches, especially in tasks involving understanding of images or text. So in recent years, with abundant data and cheap computational power, deep learning methods have become extremely popular.
However, deep learning applications in healthcare have had mixed results so far. They have been extremely successful analyzing medical imaging data, but applications to medical text generally don’t achieve the same boost as outside of healthcare. Deep learning has also not been particularly successful with other types of healthcare and biological data, such as omics and claims.
In this particular work, the research team attempts to predict healthcare utilization using diagnosis codes. This task is especially for payers, because it is the basis for risk adjustment. But, as I mentioned above, deep learning methods usually don’t perform very well on structured healthcare data alone. Here, the researchers try to attack this problem by borrowing a new deep learning technique in text processing called “attention”.
To understand a word in a sentence, one needs to identify its context. In standard deep learning methods, the context of each word is taken to be a few of the words around it. “Attention” is a way of organizing the neurons so they can decide which of the surrounding words is most relevant. The seminal paper on the topic uses the example: “the law will never be perfect, but its application should be just”. Interpreting the word “its” requires “paying attention” to the word “law”, despite the distance between the two. (Want to go deeper? Here’s a gentle technical introduction). In the past couple of years, applications of attention have led to amazing improvements in text understanding: that’s how gmail suggests sentence completions while you type.
In this case, the researchers wanted to see if using attention could improve the prediction of healthcare utilization. And the answer is... not really. Simple methods using decision trees perform just as well. To some degree, this is reassuring: the main risk adjustment method used in the US is the Hierarchical Condition Category (HCC), which is basically a decision tree curated by health actuaries. So, the current methods are probably about as good as they get.
What’s the lesson for healthcare organizations? All that glitters is not gold. Deep learning is very exciting, and being aware of the newest developments in the field is worthwhile. But for tasks where deep learning hasn’t been shown to perform well, you’re likely better off with simpler, tried and true methods.
Not on the mailing list? Subscribe to this newsletter!