Experts Say IBM Watson’s Flaws are Rooted in Data Collection and Interoperability

Despite being backed by a significant budget and the marketing power of a major technology company, IBM Watson appears to be falling short of expectations when it comes to revolutionizing cancer care. The shortcomings of IBM’s premier artificial intelligence system—made famous by its appearance on Jeopardy in 2011 and later co-pted to provide support for oncologists—are linked to a number of factors, according to an in-depth investigation by Stat that included interviews with doctors and artificial intelligence experts from around the world.

IBM Watson promised to revolutionize cancer care. So far, experts say it's fallen well short.It’s not the first time health experts have questioned the impact of Watson. Last year, researchers pointed out Watson’s high costs weren’t meeting expectations. Earlier this year, venture capitalist Chamath Palihapitiya called Watson “a joke” and said the company is leaning on marketing prowess rather than its software. Palihapitiya later told CNBC he "should have been more careful" with his words, but maintained smaller AI companies are getting "somewhat outmarketed" by larger companies like IBM.

But two key failures of the system, pitched by IBM’s marketers as a tool that can generate new insights and approaches to cancer care, are some of the same problems that currently plague most health IT systems: interoperability and data collection. A specific issue that came up during IBM Watson’s partnership with MD Anderson Cancer Center—one of the first providers to partner with IBM—was navigating the incongruent data points within medical records. Some noted that inputting data could be time-consuming and onerous for a relatively small payoff in which Watson’s recommendation aligned with the physician. MD Anderson backed out of its partnership with Watson earlier this year...