THANK YOU FOR SUBSCRIBING
Medical Tech Outlook Weekly Brief
Be first to read the latest tech news, Industry Leader's Insights, and CIO interviews of medium and large enterprises exclusively from Medical Tech Outlook
The latest method will inspire machine learning practitioners to get more creative in integrating real-time human expertise into their algorithms.
FREMONT, CA: Researchers from MIT's Computer Science and Artificial Intelligence Lab (CSAIL) develops a machine learning system that can either make a prediction about a task or defer the decision to an expert. Most importantly, it can adapt when and how often it defers to its human collaborator, based on factors such as its teammate's availability and level of experience.
The team has trained the system on multiple tasks, such as looking at chest X-rays to diagnose specific conditions such as atelectasis (lung collapse) and cardiomegaly (an enlarged heart). In the case of cardiomegaly, they found that their human-AI hybrid model performed 8 percent better than either could on their own (based on AU-ROC scores).
In medical environments where doctors do not have many extra cycles, its not the best use of their time to have them look at every single data point from a given patient's file. In that sort of scenario, its important for the system to be especially sensitive to their time and only ask for their help when absolutely necessary. The system has two parts: a classifier that can predict a certain subset of tasks, and a rejector that decides whether a given task should be handled by either its own classifier or the human expert.
Through experiments on tasks in medical diagnosis and text/image classification, the team showed that their approach not only achieves better accuracy than baselines but does so with a lower computational cost and with far fewer training data samples. The algorithms allow users to optimize for whatever choice they want, whether that's the specific prediction accuracy or the cost of the expert's time and effort. Moreover, by interpreting the learned rejector, the system provides insights into how experts make decisions, and in which settings AI may be more appropriate, or vice-versa.
The system's particular ability to help detect offensive text and images could also have interesting implications for content moderation. In future work, the team plans to test their approach with real human experts, such as radiologists, for X-ray diagnosis.
Read Also
