Rapid prospective risk assessment of mobile products in healthcare
TimeThursday, April 1512:30pm - 12:50pm EDT
LocationPatient Safety Research and Initiatives
There is much hope and hype surrounding the development of innovative technologies, but are we as motivated and effective in assessing potential risks? There is a difference between the level of risk associated with ordering a book online compared to the inherent risks of using a mobile application to manage healthcare needs. This paper describes the work of designing and performing a rapid prospective risk assessment when new functionality or changes to design are proposed to a mobile product. Prior to initial release, a prescription refill mobile product was evaluated using a comprehensive risk analysis framework for identifying latent design issues (Wood, et al., 2014). Because changes and new functionality were proposed, the business owners requested a rapid assessment to evaluate the potential risks of proposed changes within the context of use.
The mobile product described in this paper is prescription refill mobile application and is designed to help patients manage their prescriptions. Using this application, patients can request prescription refills, view prescription history, and track prescription deliveries. Recent software development activities created changes to the Electronic Health Record (EHR) data stream and proposed including barcode scanning functionality. Prescription label scanning using in-phone cameras enables patients to more easily locate and act on their prescriptions. This application uses data originating from the EHR. It was important to consider mobile application dependency on other health information systems along with changing states of each.
This case study provides a framework for human factors (HF) practitioners to conduct rapid risk assessments, especially practitioners working in Agile mobile application development environments. A rapid risk assessment could also be used as an initial assessment to determine if a more in-depth assessment is warranted.
Preliminary testing of the prescription refill mobile application uncovered several usability issues. Because of concerns about identified issues, the team deferred release of changes to the functionality and requested a prioritized list of what they should fix. A risk assessment was suggested to first define potential areas of risk and then prioritize the highest potential hazards for mitigation. The business owners requested a rapid turnaround time of two weeks to fit in the Agile product development cycle.
Deciding on and designing an approach for a prospective hazard analysis concerning the prescription refill mobile application was a challenge given the time constraints and complexity of the functional components. Considering these limitations, we decided on a strategy rather than a single method. The strategy was to draw from different approaches to design a plan to meet the goal of identifying latent design issues. We discussed several approaches for system level risk analysis such as Failure Modes and Effect Analysis (FMEA), Fault Tree Analysis (FTA), and Hazard Identification (HAZID). We drew from different approaches in designing a hybrid plan. The goal was to identify latent design issues within the specified constraints. In other words, we considered approaches from safety and resiliency engineering while accommodating needs within a complex organization within a rapid timeline.
We adapted and followed the steps suggested by Stamatis (1995) and DeRosier and colleagues (2002) in performing an FMEA as follows:
• Define the project scope
• Assemble a multi-professional team
• Describe the process, technology, system
• List all potential failure modes
• Perform hazard analysis
• Prioritize failure modes based on hazard analysis
• Develop an action plan
Steps are provided in the form of a list, but this process is dynamic and not completely linear. This approach facilitated an inquiry into the characteristics of and relationships between persons, technology, and tasks. Represented in this paper are custom processes developed to help understand potential failure modes and hazards. The deliverables took the form of a fault tree and a cognitive aid.
The FTA helped the risk assessment team organize and document potential risks associated with the scanning component into themes or areas of potential failure. The team identified three themes: the characteristic and conditions of the mobile phone camera, the phone user, and the prescription label. To address the data stream changes, the team assessed each module of the mobile product across the three phases of the EHR transition described as current, hybrid, and future states. We designed a cognitive aid to organize and guide the conversation. The cognitive aid consisted of a table listing the modules vertically and the EHR transition phases horizontally. In each corresponding cell of the table, we described the characteristics and potential failure modes and performed a hazard analysis to consider and assign potential severity and likelihood of occurrence to each section. The team develop a report documenting the approach and findings of the rapid risk assessment and presented findings to the mobile application development team to establish priorities for future work.
There are two levels to our findings. One level is concerned with the overall approach to rapidly assessing risk in an agile environment. Our initial approach was to use an FTA to describe and document the risk. In walking through the mobile application with the team, we found that the FTA was useful for describing the scanning component, but it was not the right tool for describing the data stream changes. For the data stream changes, we found that a matrix provided a better representation of the changes over time. A lesson learned is to be open to using more than one tool in an analysis of a product. We found fault trees were useful for assessing risk of static processes. In this case, we found it was easier to assess risk to changes in information flows using a table format.
The second level findings focused on the specific latent design issues in the proposed mobile application functionality. For example, a critical failure mode identified through this process is that the mobile product does not display active prescriptions available for refill that have been migrated to the new EHR. Attributed to this failure mode is that patients accustomed to seeing refillable medications in the refill module will only see the medications that are refillable in the legacy system. In other words, prescriptions are missing from the refill section because the migrated prescriptions are no longer marked as an “Active” status. This finding suggests the potential for a high-risk hazardous situation by introducing a “blind spot,” which may lead to patients inadvertently not refilling required medications potentially contributing to a catastrophic outcome. Latent design issues can contribute to unfortunate outcomes in pharmacotherapeutic delivery. Prospective risk assessments can identify latent design issues reducing the chance of patient harm.
Wood, S. D., Chapman, R., Taylor, L., Wright, P., & Scott, J. (2014, June). Identifying Latent Design Issues in Mobile Products to Prevent Patient Harm. In Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care (Vol. 3, No. 1, pp. 222-229). SAGE Publications.
Stamatis, D. H. (1995). Failure mode and effect analysis: FMEA from theory to execution. ASQ Quality Press. Milwaukee, WI
DeRosier, J., Stalhandske, E., Bagian, J. P., & Nudell, T. (2002). Using health care failure mode and effect analysis™: the VA National Center for Patient Safety’s prospective risk analysis system. The Joint Commission journal on quality improvement, 28(5), 248-267.
Stalhandske, E., DeRosier, J., Wilson, R., & Murphy, J. (2011). Healthcare FMEA in the veterans health administration. Journal of Systems Safety, 47(1), 24.
Guideline, I. H. T. (2005). Quality risk management. Q9, Current step, 4, 408.
Provan, D. J., Woods, D. D., Dekker, S. W., & Rae, A. J. (2020). Safety II professionals: how resilience engineering can transform safety practice. Reliability Engineering & System Safety, 195, 106740.