Artificial intelligence in medicine: Is the genie out of the bottle?

“Code-based regulation – especially of those who are not themselves technically expert – risks making regulations invisible. Controls are imposed for particular policy reasons, but people experience these controls as nature.”

– Lawrence Lessig, 2006¹

Bruce L. WiLder, MD, MPH, JdIt is probably a given that artificial intelligence (AI) will become an integral part of healthcare delivery and of our public health infrastructure.² What is not a given is that we will easily reach that point, and maintain progress in a way that maximizes its effectiveness in achieving the goals we have come to expect of it – efficient and improved healthcare and public health systems. In other words, making the health of people better in a cost-effective way. Responsible commentators have already begun to question the value of AI in medicine.³

People may differ on what is and what is not AI. I choose to define the concept in general terms that are inclusive of its various sub-categories. AI can be thought of as a method of automating human thought processes (by creating algorithms⁴) to exploit information by making or at least suggesting (augmented intelligence⁵) a course of action, whether it be an end result, or modifying the algorithm based on the end result, to become more accurate (machine learning). AI assumes that a person accurately perceives what those thought processes are, and that the algorithm they create accurately reflects those thought processes as perceived. In other words, there are three points where the possibility of human error can come into play: understanding human thought processes (including how data are selected and presented), reducing that understanding to an algorithm, and the process (another algorithm, if you will) by which the human-made algorithm is auto-modified.

The transformation of the paper chart to the electronic health record (EHR), jump-started by the HITECH Act of 2009, has geometrically increased the amount of easily accessible data accumulated on an individual patient in the healthcare setting. This data comes to us in somewhat structured form, including that required by the design of the particular EHR, as well as unstructured, such as free text. The former has given rise to much dissatisfaction on the part of clinicians, as well as the unanticipated tendency to collect and propagate inaccurate information in the EHR. The use of clinical natural language processing (cNLP) has enabled the extraction and structuring of data from free text, but it is not infallible, and its methods are still evolving. Moreover, rapidly evolving technical advances in diagnostic and monitoring techniques have added to the mountains of data acquired in the care of a patient.

If one adds the increases in the sheer volume of data accumulating almost daily in the medical sciences that is relevant to the care of a particular patient, the need for reliable artificial intelligence systems, including up-to- date clinical decision support (CDS), to maintain an acceptable standard of care seems undeniable.⁶

AI has already been shown to improve diagnostic accuracy in body imaging and microscopic specimens. Predictive analytics is increasingly being used in assessing risk of re- admission, early detection of changes in the status of a patient and advances in personalized medicine. The elephant in the room, though, is the potential for use as a business tool by healthcare organizations and the attendant possibility that conflicts with what is best for the individual patient will emerge.

So, the task before us requires some forethought on how AI systems will be integrated into healthcare and public health practice with a minimum of unanticipated adverse consequences. Let’s start with the supposition that physicians, to a large degree, need to be in control of the process.

The traditional model of makers of different products competing for the business of healthcare providers (institutions, physicians, etc.) doesn’t work very well for AI systems, since a “customer” is hardly in a position to compare products so as to choose the “best” one, since the product is constantly changing, and the logistics of changing to a new system can be prohibitive. Healthcare is more like a public good than a consumer product, and thus AI systems ought to be transparent. Moreover, developers of AI systems face the marketing pressures of creating a product that appeals to the business needs of an institution/ potential customer. Michael Crichton could as well have been talking about AI in medicine as he was molecular biology when he wrote “The commercialization of molecular biology is the most stunning ethical event in the history of science . . . ” and “Genetic research continues, at a more furious pace than ever. But it is done in secret, and in haste, for profit.”⁷

Medical software is generally considered to be a device and thus subject to regulation by the FDA.⁸ But who is going to be liable when things go wrong and a patient, or group of patients, is harmed? New ethical/legal issues relating to patient privacy and informed consent are sure to arise. Medical device manufacturers are ever on the lookout to avoid product liability lawsuits and tend to include

in their contracts with providers “hold harmless” clauses (the provider agrees to indemnify the manufacturer if it is a defendant in a lawsuit), or “gag” clauses (where the customer agrees not to disclose defects in the product). If an algorithm is public and crowd-sourced, users will be likely to equate its processes with the standard of care, and help eliminate or ameliorate the “black box” problem.⁹ The American Medical Association has called for user-centered design and transparency of AI systems. It remains to be seen how that policy is implemented.

Progress in medical science is fast-moving and AI algorithms will likely need constant tweaking in order to function accurately. A better paradigm (than strict IP protection) would be one that exploits crowdsourcing, i.e., input from the entire community of users. Such a scheme would require some central governing entity that would consist of healthcare providers, experts with technical proficiency (developers) in artificial intelligence, government regulators and legal experts attuned to issues of liability and intellectual property. It also would require public algorithms, a relatively new concept in the field of AI, generally. “[Algorithms] reflect existing biases in our data and society and in the very questions asked of them. Algorithms can reinforce and even accelerate existing discrimination patterns.”¹⁰ In the healthcare setting, hidden biases and assumptions in algorithms can lead to medical misadventure as well.

In summary, there is potential for wrong turns on the road to success- fully implementing and integrating AI into our healthcare system, and in the case of patient care, necessarily into the EHR – bringing about the trans- formation of the EHR into something else. For now, let’s call it the electronic healthcare module. In any event, this inevitable marriage of the EHR and AI is sure to produce something that necessarily radically transforms how we practice medicine. In order to achieve the goals of user-centered design and transparency in AI, we will need algorithms to be public. It is, therefore, critical that we not allow the potential for commercialization and venture capital interests to co-opt the process and rush us to premature adoption.

References

1. Lessig, L, Code 2.0, Basic Books, New York, p. 138 (2006).

2. Davenport T, and Kalakota R, The potential for artificial intelligence in healthcare, Future Healthc. J. 2019 Jun; 6(2):94-98, www.ncbi.nlm.nih.gov/pmc/articles/ PMC6616181/ (access 3/14/20). See also the AMA Journal of Ethics, February, 2019 (Vol 21, No. 2:E119-197 (Multiple articles), https://journalofethics.ama-assn.org/ sites/journalofethics.ama-assn.org/files/2019-01/joe- 1902_0.pdf (access 3/14/20).

3. Emanuel, EJ, Artificial Intelligence in Heath Care: Will the Value Match the Hype, JAMA 2019;321(23):2281-2282. See also Allen A, AI was supposed to save health care.

What if it makes it more expensive?, https://www.ncbi.nlm. nih.gov/pmc/articles/PMC6616181/ (access 3/14/20).

4. See www.merriam-webster.com/dictionary/algorithm (access 3/14/20).

5. AMA Policy H-480.940, Augmented Intelligence in Health Care (2018), https://policysearch. ama-assn.org/policyfinder/detail/augmented%20intelligence?uri=%2FAMADoc%2FHOD.xml-H-480.940.xml (access 3/14/20).

6. Wilder BL, On the Need for a Universal Health Record, Open Health News, 1/1/17, http://www.openhealth- news.com/story/2017-01-01/need-universal-health-record (access 3/14/20).

7. From the Introduction to Jurassic Park, Alfred

A. Knopf, New York, 2006. The novel was originally published in 1990.

8. Guidances with Digital Health Content, www. fda.gov/medical-devices/digital-health/guidances-digital-health-content (access 3/14/20).

9. “Black box” in the AI world refers to inability of users, and sometimes even developers to understand how an AI system reaches a result. See generally, AMA Journal of Ethics, February, 2019 (Vol 21, No. 2:E119-197, https://journalofethics.ama-assn.org/sites/ journalofethics.ama-assn.org/files/2019-01/joe-1902_0. pdf (access 3/14/20).

10. Pittsburgh Task Force on Public Algorithms, www.cyber.pitt.edu/algorithms (access 3/14/20).

This article was originally published in the April, 2020 issue of the Bulletin of the Allegheny County (Pennsylvania) Medical Society. It is reprinted with permission.