High fidelity formative studies at low fidelity costs: Leveraging overlapping user groups between products to get more information for less
TimeThursday, April 152:00pm - 3:00pm EDT
LocationMedical and Drug Delivery Devices
This poster discusses the benefits of piggybacking secondary early-stage formative studies onto primary studies that are further along when different products share user groups. The study design, logistical benefits and challenges, timeline savings, and cost/time savings are broken down along with recommendations for identifying similar opportunities for future work.
When initially exploring different drug delivery options for a medication, getting information quickly and cost-effectively is highly valued. When the information being pursued is best captured with a user study, speed and cost - often inversely correlated - can be actualized together by leveraging a piggybacking strategy.
At the conclusion of a primary study studying Activase IFU updates, participants were asked if they were willing to participate in a secondary study to study different drug delivery options for TNKase for an additional hour with additional compensation. If they agreed, they were offered a short break prior to starting the second study. The secondary user study being presented required evaluating three device configurations with both emergency room nurses and pharmacists. The secondary study was moderated by one Genentech employee with a notetaker present to record real-time task performance. Participants were presented with each of the devices one at a time, asked to perform tasks according to barebones instructions, and scored on task performance. The study followed standard best practices of counterbalancing across subjects to avoid ordering effects, and all questioning and debriefing were saved until after all three delivery methods were tested.
Logistical Benefits and Challenges
The logistical benefits of leveraging overlapping user groups between products in this case included the ability to use the same facility and already-recruited participants, and not having to spend hours working on new requests for quotations (RFQs) and statements of work (SOWs). The main logistical challenges were limited space at the facility requiring the use of the staff kitchen for the study session, and variability in the study schedule due to dependence on the primary study. The variability in the study schedule meant that some sessions were held in rapid succession without breaks for the moderator and notetaker, and others would have a few hours in between.
By amending SOWs instead of drafting new contracts from scratch, a process that for some companies can take up to three months and many back-and-forths was instead accomplished with a few emails and phone calls over a week.
It is common for Genentech to work with 3rd-party human factors research consulting firms to formulate and run user studies. By completing all activities related to the secondary study without the use of 3rd-party vendors, initial savings of $66,066 were realized by avoiding costs from formative planning, study moderation, data collection and analysis, report writing, and vendor travel. Additional savings were realized by sharing the facility ($13,036) and not having to specifically recruit for the study ($9,996). While full-time equivalent (FTE) hours were required to set up the study, an estimated 40 FTE hours were saved in RFQ and SOW creation alone.
Conclusions and recommendations for identifying similar opportunities for future work
The ability to leverage a scheduled study for a secondary piggyback study is highly dependent on the overlap in user groups between products. If a company has multiple products in the same space, there are likely efficiencies that can be gained by communicating across project teams about upcoming work and periodic knowledge sharing. It is important to consider the logistical challenges, such as space constraints, that exist when trying to run piggyback studies as early as possible to increase the likelihood in which one can overcome these challenges. (For example, by booking additional space with enough advanced notice that you are not conducting research in the staff kitchen.) By establishing a good working relationship with vendors and understanding the mechanics of user studies, the appropriate accommodations can be made to ensure that both the primary and secondary studies are completed without negative impact on either. With increased collaboration and flexibility, usability studies can be run faster and cheaper to more quickly inform the product development process and ultimately reduce the cost to patients.