A Study of Physicians’ Explanatory Reasoning in Re-diagnosis Scenarios: Investigating Explanation Strategies for AI Diagnostic Systems
Event Type
Poster Presentation
TimeThursday, April 152:04pm - 2:05pm EDT
LocationDigital Health
DescriptionArtificial Intelligence (AI) has the potential to revolutionize digital healthcare, and one potential area is an initial clinical diagnosis and first contact with patients. Researchers have known since the 1970s that transparency and explainability are necessary recurs to trustworthy healthcare systems, and this has been one of the largest impediments to the success of these systems and it remains one of the main challenges for these systems. Without sufficient explanations, it is difficult for a physician or patient to understand how AI makes its decision, and thus whether to trust it. In order to develop appropriate AI diagnostic explanations, it is important to understand the strategies physicians use to explain their diagnoses to their patients. Understanding physicians’ explanatory reasoning may help create systems that make the patient-AI communication better, help patients comprehend the diagnosis process, and help physicians place trust in these systems as an aid for initial patient contact. To support this, we report on an interview study with physicians in which we identified explanation strategies during diagnosis. Based on these interviews, we will summarize themes of their explanation strategies to improve existing and future AI medical diagnostic systems and provide some design recommendations for patient-facing AI diagnostic systems.
We interviewed seven physicians with a variety of specialties and experience, with a focus on identifying incidents in which they made and changed diagnoses. We used an adapted Applied Cognitive Task Analysis (ACTA) technique to conduct incident-based interviews. All methods were approved by the MTU institutional review board human subjects committee. Participants gave oral consent before the interview and agreed to have their interview audio recorded. Interviews were conducted either via phone/internet video or in-person and lasted for 45-70 minutes. After initial background questions, we focused on 1-2 cases per physician that involved a re-diagnosis and had them discuss how they communicated this to the patients. The goal of these interviews was to understand the methods physicians used to communicate with patients to explain their decisions, changes in diagnosis, and their reasoning strategies. We implemented three-step coding for the qualitative analysis of the data: 1) Initial coding, 2) Card sorting, and 3) Hierarchical clustering. Five rough hierarchical themes emerged from the clustering analysis, along with the 24 base codes.
Theme: Preparing Patients for Later Possibilities
Physicians often provided an initial provisional diagnosis based on the symptoms and history. This not only included the most-likely condition but also often included other possibilities. Thus, this kind of explanation prepares the patient to accept and understand possible future changes in diagnoses. Some AI systems do show probability distributions across different possibilities, which supports this same function. However, they are much less likely to do this in order to create a narrative that will be followed up on later in the diagnosis.
Theme: Tailoring Information to the Audience
Physicians also reported that they often tailored their explanation to the individual, based on either socio-economic or cultural status, the intellectual level of their patients, their current emotional state, and other concerns, all of which were dependent on the patients and their ability to understand the information. There has been advocacy for user models, but tailoring can be done in simple ways as well, such as having the user select the complexity of the explanation they want.
Theme: Using Case Information
Physicians often generated their diagnoses over time using emerging information about the case. They then walked patients through the case scenario to help the patients understand the diagnoses. This diagnosis mode is similar to the explanation scripts initially explored by developers of expert diagnostic systems, which would create text-based descriptions of the logical steps by which a diagnosis was reached.
Theme: Using Test Results and Logical Constructs to Support Diagnosis
Physicians reported that they frequently used test results and medical records of the patients to support the diagnoses, which formed the basis for justifying and explaining diagnoses. The interviewees mentioned using an X-ray, CT scan, endoscopy, angiogram reports as visual aids, as well as medical records and test results (e.g. blood tests) to explain their diagnoses. Physicians also reported several higher-level strategies, including logical arguments, examples, analogies, metaphors, and counterfactuals to help patients understand a diagnosis. These correspond roughly to many forms of explanation being explored. For example, the explanation given by a physician pointing out a critical area of an X-ray image is similar to the LIME algorithm that highlights critical features in an image.
Theme: Build Emotional Connection and Rapport
Physicians often considered the emotional aspects of communication with the patient and their families. These were not always about providing explanations or information but involved empathetic strategies to ensure their patients knew the physicians listened and cared. Physicians suggested that patients might initially be anxious and not in a condition to understand the reasoning and explanation, and their explanations at this point differ from later explanations. This may be an area where AI will never match the empathetic abilities of human diagnosticians. Nevertheless, researchers have investigated ways in which we treat computers as social actors, which suggests that it may be possible to build social and emotional rapport between a human and a machine.
The first generation of medical diagnostic systems based on the 1980s expert systems framework failed. Many observers at that time rightly pointed to a lack of explainability as one of their main weaknesses. Yet explanations in those systems were relatively simple to identify, as they came directly from human-generated rules. Today’s diagnostic systems are becoming more difficult to understand, making explanations even more necessary. But the current explainable AI (XAI) approaches remain algorithm-focused, without accounting for or modeling the explanation patterns of human physicians. The present study helps identify some of the goals and methods of explanation among human diagnosticians that may be applied for the development of AI diagnostic systems. Some of the things we discovered really are the core of XAI systems- use of examples, counterfactuals, visual aids. But these are the pieces of explanation, it is not the whole explanation that patients need. AI systems in healthcare need to put it together at the right time, tailoring it for different patients at different points of diagnosis to ensure proper utilization of these systems.