Can a computer be actually better at diagnosing and treating conditions than a human doctor? As more and more patients try to self-diagnose diseases on their own by searching their symptoms on the Internet or with new app-based checkers, new diagnostic software is tested every day to answer to this ever-increasing demand for answers. Human physicians, on the other hand, are all but infallible, so digital technology as well as computers that could make sense of symptoms to make accurate clinical diagnoses may represent the future of modern medicine. Although other studies already compared the diagnostic accuracy of information technology vs doctors for reading electrocardiograms (ECG), this new research published in JAMA Internal Medicine is the first one that measured if the accuracy of computer-based symptom checker algorithms could outperform, or even just match, human intelligence.
Researchers from the Harvard Institute of Medicine compared the answers coming from the most famous 23 web- and app-based symptom checker platforms to those provided by a group of 234 internal medicine physicians. Each doctor had to evaluate 45 clinical scenarios (called “vignettes”) with clinical conditions of varying degrees of rarity and severity. For each case, the healthcare provider had to identify the most probable diagnosis but was able to choose two additional diagnoses. Each clinical case was solved at least 20 times, and “errors” were computed by either providing a wrong diagnosis or not providing it in a timely manner. The results left no space for doubts: humans vastly outperformed machines in recognizing diseases, and their answers were correct 72 percent of the time when listing the first diagnosis, versus just 34 percent for the computer software. When checking the correct choice in the top three alternatives, humans identified the right one 84 percent of the time, compared with 51 percent for digital platforms. This difference was even more substantial for less common and more severe conditions, further highlighting how human judgment could be critical in life-or-death situations.
Senior investigator Ateev Mehrotra and the rest of his research team concluded that although computer-based symptom checkers are still clearly inferior to real doctors in terms of accuracy, their future development is still crucial to help physicians in their future endeavors. To date, doctors still make errors in about 10-15 percent of cases, and the upcoming digital technologies may help them fill that gap.
Article by Dr. Claudio Butticè, Pharm.D.
- Semigran, H L, et al. Comparison of Physician and Computer Diagnostic Accuracy. JAMA Internal Medicine, 2016; DOI:10.1001/jamainternmed.2016.6001