Highlights

Summarized by Arne Sonar

Highlights of the VEIL

The seventh and last Virtual Innovation Lecture of this term featured Robert Sparrow, professor of philosophy at Monash University (Australia). His main research interests focus on political philosophy and applied ethics, e.g., bioethics, environmental ethics, media ethics or the ethics of science and technology. The talk outlined the ideas of the article “High Hopes for ‘Deep Medicine’?” written with co-author Joshua Hatherley, which consist in a critical examination on the plausibility of hopes, promises and expectations associated with the use of AI within medicine.

Introduction

Robert Sparrow began with the provocative statement that some of the potential benefits stated in association with the use of AI in healthcare are naive, especially in relation to the economical as well as organizational demands of the corresponding environment. The main point rests on the observation that the social context within which these applications are developed and deployed impedes/hinders the realization of their full potential. As a representative of positivistic opinion on the topic of AI in medicine, Robert Sparrow bases his critical reflections on excerpts from Eric Topol's "Deep Medicine - How Artificial Intelligence can make healthcare humane again" (2019.

Topol’s Thesis #1: Ubiquitous datafication will improve medicine and health care services 

Sparrow comments that the term ‘deep’ in deep medicine corresponds especially to the data-mining of patients, using all data available within the health care context for improving medical diagnosis and treatment. Referencing work by Liu et al. (2019), Sparrow notes that despite frequent far reaching claims, AI in medicine lacks broader evidence of the superiority of AI systems over human capabilities. With Liu et al. showing that from the 31.581 identified and potentially relevant papers only 14 provide an explicit comparison on capabilities of deep learning models and health care professionals, there is hardly any reliable empirical long-term evidence on AI’s effectiveness with respect to improving clinical outcomes. However, Sparrow concedes that systems eventually outperforming humans is a likely prospect. Nevertheless, Sparrow pleads for a little more realism and reflection within the current discourse in relation to the current technical possibilities and a little more awareness on the point, that most of the often discussed aspects belong to a technology in the future.

Regardless of available evidence, but rather in response to frequent claims that AI should not replace physicians, Sparrow highlights the view once machines really prove to be better, there is an obligation to replace human doctors. Accordingly, these must not only be seen as systems in themselves, but should also be seen as replacements in narrowly defined areas or particular roles (where there is evidence of their superiority). However, transitional phases in which machines and humans are similarly capable can be problematic here, as confidence in one's own judgement can sometimes be undermined by possible uncertainties vis-à-vis machine capabilities or an increasing pressure to consult technology.

Topol’s Thesis #2: Adopting AI’s use as a partnership fosters physicians-patients-relationships

As Sparrow states, a frequently shared expectation or promise of AI in medicine is that doctors will have more time for their patients. As he lays out, medicine has evolved from a humanistic practice to a technocratic one, e.g., through the impact of electronic health records. Such positive promises therefore seem to be a natural rhetoric for advertising AI technology in medicine.

Even though such positive claims and attractive visions for a future AI-assisted medicine sound quite appealing to Sparrow, they appear rather naive due to a supposed neglect of the institutional context of medicine and its everyday practice. Sparrow suspects that the efficiency gains are more likely to lead to an institutionally induced rationalization. He locates the primary lines of argumentation or motives for this in a triad of strive for profit, creating expanded access and the analysis of measurable data metrics. 

Topol’s Thesis #3: AI could squeeze physicians to more activism & performance pressure

According to Sparrow, Topol is not unaware of the potential importance of economic pressures – that’s why he mostly writes his expectations as a hypothetically ‘might be’ –, calling also to contest a future in which efficiency dominates. In this regard, Topol’s book could also be understood as a kind of ‘call to arms’ against those developments. Sparrow himself is not optimistic here, because history does not give much cause for hope. 

As Sparrow states further, Topol assumes physicians (taking on political action) will contest with their own institutions in a period that is characterized by 

• massive disruption to medical practice and professions, which could negatively influence physicians’ self-confidence and their subjectively felt (political) empowerment;
• (fear of) job losses, which, e.g., might have negative impacts on physicians’ willingness to go on strike and to demand that things need to be done differently;
• deskilling, which leads physicians (and other clinicians or skilled professionals) to worry that skills seen as core principles of their profession are being taken over by technology;
• disempowerment, resulting from an outsourcing of physicians’ former core functions (and skills) to technical applications (machines) and human-machine-combinations;
• surveillance, resulting from gathering data and tracking more and more of physicians’ and other health care professionals’ performances.

In contrast to Topol, these are substantial reasons for Sparrow to be rather pessimistic on the political ambitions of doctors and about future developments within the health sector. 

Topol’s Thesis #4: Physicians have allowed the care relationship to erode 

As Sparrow points out, Topol seems to be well aware that doctors, as actual critical gatekeepers, sometimes contribute to the erosion of their own relationships with patients and their self-experience as physicians. Evidence can be found from the the introduction of electronic health records or other computer-based systems within medicine. Hence, for Sparrow it is necessary to ask, why the introduction of AI in medicine should prove a different case. Rather, it may be reasonable to assume that AI will contribute to these developments.

Above all, in Sparrow’s view a possible drowning in data proves to be a main aspect that needs to be problematized, especially addressing Topol’s vison that many data collection processes will and should be automated. A crucial part of Sparrow’s plea for skepticism here is the need for an awareness that 'sensing' the world even considering all technological developments, has been proven to be more difficult than initially thought. To gain a holistic picture of patients in the way physician-led anamnesis can has been proven difficult to automate.

Black-Box AI & AI’s claim to improve patient-physician relationships & practice of care

In Sparrow’s opinion, the claim that AI will improve the patient-physician relationship is a strange one, because of the strong relationship between trust and care as well as between care and the sense of the agency of physicians –more reliance on algorithms, which, however, are hardly understood by the physicians themselves, could undermine their authority of physicians as perceived by patients and may therefore prove extremely disruptive for the physicians-patients relationship. Any attempt to overcome this dynamic, e.g., by insisting on the doctor’s responsibility for  overlooking the AI’s output, exacerbate the physicians’ workload and, accordingly, pressure on them. To avoide this, it may turn out inevitable to assign more and more tasks to AI, which, in turn, amplifies the problem of physicians’ loss of authority. In conclusion, according to Sparrow, the hope that AI is going to free up doctors is just wishful thinking.

Q&A Session

Questions being raised (and addressed) in the aftermath of the talk revolved around the potential needs to be considered for a responsible collection of data, thoughts on the education of medical professionals in the present and in an ideal (future) scenario, the thesis of potential disempowerment and human-in-the-loop design approaches as well as  the thesis of a need for the system’s ability to efficiently explain itself. Another question also addressed, whether AI could change medical business models (e.g. health- or disease-related) in health care.

Summary

Although the use of AI (especially) within medicine comes with many promises (e.g. enhancing physicians) and expectations, Rob Sparrow highlights that these should always be aware treated as just that: Promises and expectations; not as proven facts. Despite all the supposedly good intentions, promises and expectations in and of the development of AI-based (support) systems, significant economic and institutional considerations must always be taken into account, especially in the medical field, which can and will determine the way in which they are implemented and used – not always necessarily for the benefit of physicians and patients.

Literature

Liu X, Faes L, Kale A. U., Wagner S. K., Fu D. J., Bruynseels A., Mahendiran T., Moraes G., Shamdas M., Kern C., Ledsam J. R., Schmid M. K., Balaskas K., Topol E. J., Bachmann L. M., Keane P. A., Denniston A. K. (2019): A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. In: Lancet Digital Health; 1, 271-297. DOI: 10.1016/S2589-7500(19)30123-2

Sparrow R., Hatherley J. (2020): High Hopes for “Deep Medicine”? AI, Economics, and the Future of Care. In: Hastings Center Report; 50 (1),14-17. DOI: 10.1002/hast.1079

Topol E. (2019): Deep Medicine - How Artificial Intelligence can make healthcare humane again. ‎New York, Basic Books. ISBN: 978-1541644632