Design and development of a prototype heads-up display: Supporting context-aware, semi-automated, hands-free medical documentation
Event Type
Oral Presentations
TimeWednesday, April 142:00pm - 2:20pm EDT
LocationDigital Health
DescriptionMilitary and civilian medical personnel across all echelons of medical care play a critical role in evaluating, caring for, and treating casualties. Accurate medical documentation is critical to effective, coordinated care and patient outcomes. Traditional medical record systems used for documentation are not appropriate for challenging military operational environments, such as Tactical Combat Casualty Care’s (TCCC) TCCC card. Improved methods are needed to support efficient medical documentation during treatment and to provide relevant information in situ to support effective medical care, without detracting provider attention or action.

We will describe our prototype Context-Aware Procedure Support Tools and User Interfaces for Rapid and Effective Workflows (CAPTURE). Leveraging human factors and user-centered design methods, and advanced natural language processing (NLP) and computer vision capabilities, CAPTURE was designed to enable TCCC providers to more efficiently and effectively input critical medical information across a range of operational environments through hands-free interaction techniques and semi-automated data capture methods.

During the course of this project we encountered challenges towards effective design which fall into three main categories: (1) challenges related to designing novel multimodal interfaces; (2) technical challenges related to software and hardware development to meet design needs; and (3) challenges as a result of domain characteristics and operational constraints. We will discuss how we addressed some of these challenges and lessons learned, as well as provide additional considerations necessary for future research regarding next generation technology design for medical documentation in the field.

Target audience:
The target audience for this presentation are practitioners and developers interested in next generation medical device design and development, specifically augmented reality and other hands-free interface and interaction methods. In particular, our discussion regarding challenges faced, lessons learned and next steps will be helpful for those looking to perform similar work. Researchers will also benefit from this presentation to learn the current state of the art and gaps where future research would be beneficial (e.g., developing more in-depth augmented reality guidelines through interaction design studies, expansion of voice to text libraries in the medical domain, and exploring new approaches to facilitate data capture and computational power in austere environments).

Uniqueness and Impact:
CAPTURE is a novel system that provides medical care providers with medical documentation support in a form that is appropriate for the field environment (e.g., Tactical Combat Casualty Care; TCCC). This system provides hands-free, heads-up interaction methods and semi-automated data capture that support medical documentation without detracting attention from the main task of performing medical care. The novel interfaces minimize interaction costs by combining multimodal capture methods, vision technologies, and efficient hands- and eyes-free interaction mechanisms. Because documentation does not necessarily occur in a single phase of medical care, we explored direct input methods, passive documentation, semi-automated documentation, and no documentation but artifact creation, to support the flexible and effective capture of necessary information while providers are simultaneously providing medical care. The introduction of this technology approach ensures providers can effectively document in support of the care continuum via new documentation methods that do not detract from the medical care they are providing. The care provider can rely on semi-automated documentation of many elements as well as capture of key artifacts (e.g., audio notes and photos to support cueing for later documentation) to support complete and accurate post-care documentation

The nature of medical care in the field (e.g., TCCC, pre-hospital care) results in difficulty with performing medical documentation in a timely and accurate manner. Care providers are first attending to casualties’ time sensitive medical care needs and once those needs are met, they are then able to complete medical documentation. As a result, medical documentation can be incomplete or inaccurate due to challenges of prospective memory, recall, and time constraints, or even damaged or lost as with current paper-based methods like the TCCC card. Our CAPTURE work provides a means for care providers to complete medical documentation without detracting from medical care, improving the effectiveness and efficiently of this documentation and care, as well as information accessibility across echelons of care (e.g., provider’s own reference during continued care, to facilitate hand off).

This work is supported by the US Army Medical Research and Materiel Command under Contract No.W81XWH-18-C-0099. The views, opinions and/or findings contained in this report are those of the authors and should not be construed as an official Department of the Army position, policy or decision unless so designated by other documentation.