How to Combat Negative Transfer of Learning during Medical Device Development
TimeThursday, April 152:00pm - 3:00pm EDT
LocationMedical and Drug Delivery Devices
Industry best practices to combat negative transfer of learning when designing a new device or making changes to an existing device.
The FDA is primarily concerned that devices are safe and effective for the intended users, uses, and use environments. Therefore, the goal of manufacturers is to eliminate or reduce to the extent possible, use errors resulting from aspects of the device user interface that contribute to or cause unsafe or ineffective use. Use errors may result if the device use is inconsistent with the user’s expectation about the operation of the device. The most effective strategy to reduce or eliminate such use errors is to design the device user interface to be logical and intuitive to use.
Transfer of learning is a concept that manufacturers should consider when designing a user interface to facilitate correct user actions and prevent use errors. Transfer of learning occurs when knowledge learned in one context is applied to a new context. Transfer of learning can be positive when it enhances the related performance in another context, or negative when it hinders the performance. Though negative transfer of learning is commonly used to explain use errors observed during usability testing, there is a lack of concrete methodology in the field of medical device development to identify sources of negative transfer of learning and to design to combat negative transfer.
The FDA acknowledges the importance of a well-designed user interface that aligns with the user’s expectations to reduce or eliminate use errors. The 2016 FDA Human Factors Guidance states:
“An important aspect of the user interface design is the extent to which the logic of information display and control actions is consistent with users’ expectations, abilities, and likely behaviors at any point during use. Users will expect devices and device components to operate in ways that are consistent with their experiences with similar devices or user interface elements. For example, users might expect the flow rate of a liquid or gaseous substance to increase or to decrease by turning a control knob in a specific direction based on their previous experiences. The potential for use error increases when this expectation is violated, for example, when an electronically-driven control dial is designed to be turned in the opposite direction of dials that were previously mechanical.”
This example is a case of transfer of learning, where a user takes their understanding of the function of a knob from previous experiences with similar devices and transfers that knowledge to a different device. If the knob is designed to function differently than expected by the user, this example would be a negative transfer of learning potentially leading to a use error. In some cases, negative transfer of learning can lead to serious harm. Therefore, actively seeking and designing-out sources of negative transfer will improve the overall safety and effectiveness of the device. This presentation further explores this concept by presenting methods to uncover the potential for negative transfer of learning and best practices to combat negative transfer of learning through design.
Our methodology for combatting negative transfer of learning includes three steps.
STEP 1: The first step to identify sources of negative transfer of learning is to assess the intended user’s knowledge and experience with similar devices, as well as willingness and motivation to learn a new device. The more experienced the user, the more rigid their knowledge tends to be which may lead to a false sense of confidence when using a new device. Over-confidence bias is commonly seen in usability testing with untrained users, leading to reduced likelihood of reliance on instructions and other labeling. Therefore, labeling is not an effective mitigation against negative transfer; modification to the device user interface is the most effective strategy to employ.
This talk will include our methodology for assessing users’ tendencies towards negative transfer of learning, based on user characteristics and contexts of use.
STEP 2: The next step is to compare the device to other on-market, similar products. This may include previous generation devices, predicate devices, or devices with similar user interface elements. Identifying similarities and differences in the user interface compared to other products can pinpoint aspects of the user interface which may lead to negative transfer.
Methods for performing a thorough comparative analysis include:
• Physical user interface comparison
• Labeling comparison
• Task comparison
This talk will include case studies describing the methodology we developed for uncovering negative transfer of learning via comparative task analysis.
The goal of identifying sources of negative transfer is to eliminate or reduce the potential for use error by making modifications to the device user interface. By employing this strategy early on, this information can inform the device design before the user interface is final as well as inform requirements for labeling and training materials. The process of identifying sources of negative transfer will inform how users are anticipated to interact with the user interface and their expectations of how the device functions. As modifications are implemented, this process should be conducted iteratively to assess the effectiveness of the design modifications and mitigations.
STEP 3: The final step is a heuristic evaluation of the user interface. This talk will explain heuristics for user interface design to combat negative transfer of learning, such as:
• Consistency with generally accepted standards or conventions (e.g., up arrow means increase, down arrow means decrease)
• Consistency across user interface (e.g., appearance and location of critical functions should be consistent)
• Warning screens or enabling actions to force acknowledgement by the user
• Warnings or cautions to highlight functions that may differ from the user’s expectations
• Automation of device functions that are prone to use error and removal of unnecessary functions where possible