Applying human factors methods to improve healthcare risk management tools
TimeThursday, April 152:00pm - 3:00pm EDT
The Healthcare Insurance Reciprocal of Canada (HIROC) is a trusted healthcare safety advisor, committed to offering a full spectrum of insurance products and healthcare safety and risk management services. As a reciprocal, HIROC is governed by their subscribers—a group of over 600 diverse healthcare organizations across Canada. Healthcare risk management and patient safety are the cornerstones of the work that is provided. HIROC currently offers two risk management tools to their subscribers—the Risk Assessment Checklist (RAC) and the Risk Register (RR). The RAC is a web-based self-assessment tool enabling healthcare organizations to systematically self-assess compliance with a number of actionable mitigation strategies for top risks leading to malpractice claims. The RR is also a web-based tool for healthcare organizations to manage and track key organizational risks. As part of HIROC’s initiative to continuously improve value for subscribers, HIROC conducted a human factors evaluation to systematically identify and improve usability and content, as well as the experience of these tools.
Between July and August 2020, a human factors evaluation consisting of remote semi-structured interviews was conducted to identify usability areas of interest with HIROC’s RAC and RR tools. The research team consisted of an HF specialist, a Risk Management specialist, and a Risk Management coordinator. Interview participants (n=16) were recruited from nine HIROC subscribers working in risk management, quality improvement, and patient safety. The participants consisted of specialists, managers, and directors with experience in risk management or experience with the RAC or RR tools. In total, five participants provided feedback on the RAC tool and fourteen participants provided feedback on the RR tool.
The research team prepared a set of questions to understand each participant’s organizational risk identification process, identify opportunities for improving the RAC and RR tools, and determine how well existing reports align with their organizational needs and expectations. The interview questions included open-ended and Likert scale questions.
Once all the interviews were completed, the research team synthesized the participants’ responses into a list of identified usability findings for the RAC and RR tools. The research team then completed a thematic analysis to categorize these findings and identify common themes with each of the risk management tools. Design recommendations to address these findings were also developed.
For the RAC tool, most participants indicated that the tool is informative as it helps create risk management awareness across their organizations. Overall, some participants said the RAC is easy to use, and some specifically noted the submission process to be fairly simple. For improvement opportunities, participants identified a number of usability findings to be considered and addressed (e.g., inefficient data entry, challenges viewing the current year’s progress, and discerning between different sections within the tool). The last interview question was, “How satisfied are you with the Risk Assessment Checklists at helping you self-assess compliance with mitigation strategies for HIROC's costliest risks?” Using a Likert scale from 1-to-5 (1 = very dissatisfied, 5 = very satisfied), out of five participants: four (80%) provided a score of 4 (satisfied), and one (20%) provided a score of 5 (very satisfied).
For the RR tool, participants reported that the tool is simple and easy to use. They find value in having an electronic system that keeps them organized and provides a way to track and trend their progress. Some participants also utilize the reporting feature to generate reports that are shared with their Senior Leadership Team and their Board of Directors. For improvement opportunities, participants would like to see a modernized and updated user interface. They would also like more options for customizing data reports. Some participants also suggested a better audit trail in order to clearly see updates made within the risk records. At the end of the interview, participants were asked, “How satisfied are you with the Risk Register as a platform to systematically identify, assess, and manage key organizational risks?” Using a Likert scale from 1-to-5 (1 = very dissatisfied, 5 = very satisfied), out of 14 participants: one (7.1%) provided a score of 3 (neutral), nine (64.3%) provided a score of 4 (satisfied), three (21.4%) provided a score of 5 (very satisfied), and one participant (7.1%) did not provide a score.
Future Next Steps:
Although most participants provided a satisfaction score of 4 (satisfied) or 5 (very satisfied) with the risk management tools, they also identified a number of usability findings. The design recommendations are currently being assessed for feasibility and overall impact for the organization. All recommended design changes will be evaluated with end users to validate ease of use and to ensure unexpected usability interactions have not been introduced.