Selecting and Justifying User Surrogates for Human Factors Regulatory Submissions: A Framework for Analyzing and Measuring User Capabilities During Design, Evaluation and Validation Activities
TimeWednesday, April 142:00pm - 2:20pm EDT
LocationMedical and Drug Delivery Devices
DescriptionIn the medical product design and development process which requires regulatory submissions to the FDA and other notified bodies, manufacturers must understand, document, and recruit intended product users for human factors evaluation and validation activities. Intended users are the people who complete all or part of the intended use of the product. Design, evaluation and validation efforts typically focus on groups of similar users (i.e., user groups) due to challenges designing products for every type of individual user. Distinct user groups are typically comprised of users who perform different tasks (e.g., setup, calibration, measurement) or have different attributes (e.g., gender, age, product experience).
When considering the use of surrogate users (i.e., people used as a substitute or representative for intended users, in order to provide information in design meetings, user testing, etc.), it is critically important to focus on the meaningful differences in user capabilities and limitations that may impact the safe and effective use of a product (Usability First, 2021). For example, functional attributes of the user such as diminished hand strength for older nurses with arthritis, vision loss for adults with type 2 diabetes, and cognitive impairment for patients with heart failure may introduce the most use-related risk from the perspective of user attributes.
To date, practitioners have reported various challenges related to evaluating and validating user interfaces with some types of intended users. Challenges may be due to factors such as intended users being part of a rare disease population, intended user populations being limited in a particular geographic location, intended users being needed for formative testing so participants are more limited for validation recruitment, etc. To date, FDA also has been open to considering the recruitment of user surrogates (i.e., proxies) in certain instances, while various manufacturers have successfully recruited and justified surrogates in design evaluations as well as human factors pre-market submissions (i.e., including validation studies).
In order to understand, select, document and justify the involvement of appropriate representative intended users during the different phases of the design, development, and regulatory submission processes (e.g., determining user needs, developing design inputs, writing the validation protocol, developing the human factors regulatory submission), designers, human factors practitioners, and manufacturers need a broad and deep understanding of functional attributes, capabilities, and limitations of different types of users and potential surrogates. This is especially true when manufacturers want or need to use surrogates during the formative or validation phases of design and development. And a solid understanding of user disabilities and functional performance is critical to minimize the risks of confounded or irrelevant data collection, need for repeated usability studies, as well as to justify the use of surrogates to FDA or other notified bodies within a regulatory submission.
Building on decades of industry and consulting experience, accessibility and disability research expertise, and a tool-kit presented at the HFES Healthcare Symposium in 2020 (Liebel, 2020), our team developed a surrogate user framework to assist design teams, human factors practitioners, and manufacturers with the systematic analysis, selection, and measurement of critical user characteristics. Our surrogate user framework can be used throughout different phases of the medical product design and development process, and it ultimately helps stakeholders select, justify and recruit the most appropriate surrogate users when seeking to optimize and validate medical product user interfaces.
The surrogate user framework is built upon the well-known PCA analysis (Kaye, North, and Peterson, 2003), and it includes an analysis of the intended users, systematic assessment of the functional characteristics of the intended users, selection of appropriate functional assessments to measure and document user capabilities and limitations, and considerations of potential risks to the research and regulatory submission.
The PCA tool helps outline the perceptual, cognitive, and manual action characteristics of the intended users and user groups for the surrogate analysis. The PCA user characteristics are then used to help stakeholders determine the most critical intended user attributes to evaluate and measure during recruitment and testing of the user interface, which are prioritized when they are most likely to impact the safe and effective use of the overall user interface (or a subset of the product design features).
The surrogate user framework can be used throughout different phases of the design, evaluation, validation, and regulatory submission phases of medical product development. It provides a process and tools for improving how intended users are defined and participants are selected, justified and recruited throughout the product design and development phases. The surrogate user framework helps designers, researchers, human factors practitioners, and manufacturers who are considering the use of surrogates or who may be looking to prioritize recruitment of the most meaningful subset of users with functional limitations during different phases of the product development process. When implemented, the framework enables stakeholders to better represent intended user group characteristics during human factors testing, which ultimately results in more efficient optimization of safe and effective user interface designs.
Overall the surrogate user framework provides: (1) Categories of information relevant to specific perceptual, cognitive and manual action user characteristics, and (2)
Standardized, repeatable, reliable, and validated functional assessments that can be administered by individuals with minimal to no training or clinical expertise. To develop the framework, our team mapped the PCA characteristics to specific functional limitations (e.g., Manual Action > Mobility > Upper Extremity > Manual Dexterity). Then we identified options of potentially appropriate functional assessments (e.g., Purdue Peg Board, Box & Blocks, etc.) to help stakeholders document different categories and levels of user capabilities and limitations that are most relevant to various types of intended users. We aimed to include functional assessments in the framework based on the tools being: (1) easily administered by a non-clinician or someone with minimal training, (2) standardized, repeatable, reliable and validated, (3) administered in <15 min, and (4) linked to specific PCA characteristics. The surrogate user framework organizes key information related to the functional assessment, web-based information about the tool, price, short description, estimated time to complete the assessment with one user, and any required equipment.
The framework is currently being used with medical and combination product clients, and the presentation will include an overview of the framework as well as case study examples of its use.
• Usability First Glossary for ‘surrogate users’, retrieved on January 28, 2021 from https://www.usabilityfirst.com/glossary/surrogate-users/index.html#:~:text=people%20used%20as%20a%20substitute,have%20interacted%20w%2F%20users)
• Liebel, T, Considerations for Study Design: A Tool-Kit for Impaired Users, HFES Healthcare Symposium 2020.
• Kaye, R. D, North, R.A., and Peterson, M. K. (2003) UPCARE: An analysis, description, and educational tool for medical device use problems. Proceedings of the 9th Annual International Conference on Industrial Engineering Theory, Applications and Practice. Las Vegas, NV.