When, How, Why and Watch-outs for Knowledge Tasks
TimeThursday, April 153:30pm - 4:30pm EDT
LocationMedical and Drug Delivery Devices
DescriptionWhen critical task performance cannot be observed/evaluated through simulated use or are not otherwise associated with handling (e.g., device storage requirements, frequency of device component replacement), knowledge tasks must be used to understand whether the device user interface supports safe and effective use. In most cases, the knowledge tasks focus explicitly on the effectiveness of risk controls in the labeling, primarily in the IFU.
Some knowledge task questions are difficult to construct without sounding leading and can also be difficult to assess in terms of the ‘correctness’ of the response. While simulated-use study design methods are well understood and often discussed, little guidance exists around crafting effective knowledge task questions.
This panel will discuss topics pertaining to when to use knowledge tasks as a tool in human factors evaluations, including FDA’s expectations for knowledge tasks in human factors validation studies, and how to determine the need for and scope of knowledge tasks based on the use-related risk analysis. The panel will also discuss how to design knowledge task questions to ensure study objectives are met, sharing lessons learned from study experience and FDA feedback received on past protocols and study reports. The session will also discuss EMA’s expectations for labeling readability studies. Session attendees will be able to apply these learnings to their own human factors study design, particularly those seeking regulatory approval for medical devices and combination products.
Panelists include human factors experts with a wide array of background expertise/experience (spanning, surgical devices, combination products, consumer/medical devices and diagnostics), including:
• Robert Stephens (Medtronic): Robert Stephens holds a doctorate in cognitive systems engineering and human factors from The Ohio State University. He has over 10 years of professional experience in medical device usability, particularly in conducting and planning usability studies. For the last seven years, he has directed usability and user experience programs at Medtronic for a number of products, including a surgical robot, electrosurgical devices, and surgical hand tools.
• Molly Story (Sanofi): Molly Follette Story, PhD is a Senior Advisor in Sanofi’s Medical Device Development Unit. She was formerly Senior Director of Sanofi’s Global Usability Engineering and Risk Management group. Before joining Sanofi, Dr. Story was Human Factors and Accessible Medical Technology Specialist at the U.S. Food and Drug Administration, Center for Devices and Radiological Health, Office of Device Evaluation, where she had an appointment to the Senior Biomedical Research Service. Dr. Story is an expert in human factors engineering of medical devices, with particular expertise in universal design of products and in the accessibility and usability of medical devices. Dr. Story received a BSE from Princeton University, a MS from Stanford University, and a MS and a PhD from the School of Public Health at University of California, Berkeley.
• Limor Hochberg (Bose): Limor Hochberg is the Human Factors Engineering Lead for Bose's Health division, where she helps ensure their hearing healthcare products are safe and effective. Limor has a BA in Cognitive Science from Ben-Gurion University and an MS in Engineering Psychology from the Rochester Institute of Technology.
• Melanie Turieo (Cambridge Consultants): Melanie Turieo is Senior Vice President and Head of Cambridge Consultants' US MedTech business. She has over 20 years’ experience providing Human Factors expertise to the design and development of regulated and safety-critical items, including medical, military and consumer products, with the last 15 years focused exclusively on regulated medical development. Melanie has a Bachelor of Science in Human Factors Engineering with a minor in Engineering Management from Tufts University’s School of Engineering.
Key takeaways include:
• When to use knowledge tasks vs. other assessment methods (e.g. simulated use)
• FDA expectations for knowledge tasks in human factors validation studies
• EMA expectations for labeling readability studies
• Various methods to present knowledge tasks to reduce bias and artifact
• How to assess/score/rate knowledge tasks
• Considerations for conducting post-task interviews to support root cause analysis of incorrect/partially correct responses
• Common pitfalls in asking knowledge task questions and assessing responses - what can go wrong & how to avoid the pitfalls