The My Diabetes Care Mobile Patient Portal Intervention: Design Sprint and Prototype Testing
TimeThursday, April 152:35pm - 2:36pm EDT
DescriptionBackground: Diabetes self-management can prevent or delay diabetes-related complications, yet patients struggle to consistently engage in recommended self-care behaviors. Patient portals offer a promising platform to increase patient engagement, improve patient’s understanding, and support self-management while overcoming the limitations of costly and difficult to scale face-to-face interventions. Although patient portals could be a highly effective way to engage patients and improve diabetes self-care, a lack of attention to design, usability, health literacy, and patient engagement has limited their impact. We recently applied Design Sprint methodology to develop a multi-faceted patient portal intervention, My Diabetes Care (MDC), designed to help patients better understand their diabetes health data as well as promote and support self-management. MDC was embedded within an established patient portal, My Health at Vanderbilt (MHAV), on Epic’s MyChart platform and was best viewed on desktop devices. In our prior studies of MDC, users indicated that a mobile-friendly version would increase their satisfaction and use of MDC.
Objective: The aim of this study was to apply Design Sprint methodology and task-based prototype usability testing to design and evaluate a mobile-friendly user-interface for MDC.
Methods: We applied 5-day Design Sprint methodology developed by Google Ventures (Alphabet Inc, Mountain View, CA) to create our initial MDC mobile-friendly prototype. This process included research on users’ needs and existing challenges, stakeholder interviews, a review of existing products and strategies, sketching and critiquing new solutions, selecting and storyboarding the preferred design solution, and creating a high-fidelity prototype. We then conducted task-based prototype usability testing with individual patients in-person then via the Zoom web conferencing platform due to the Covid-19 pandemic. To ensure representation of groups with distinct usability challenges, we purposively sampled so each round of testing included at least two individuals with limited health literacy and/or age of 65 or older. The facilitator instructed participants in “think aloud” procedures which allowed for rich qualitative data collection to better understand and track participants’ thought processes as they attempted to use the prototype. Measures included: (1) metrics of task performance on 11 common and standardized tasks, (2) participant feedback regarding areas for improvement and of satisfaction as well as participants’ overall tone and message, and (3) the System Usability Scale (SUS). We revised the prototype after each round of testing in response to usability findings until most participants successfully completed tasks, expressed high satisfaction with MDC usability (SUS score ≥ 80), and identified no new usability concerns (i.e., stop criterion was met).
Results: The sample (N=12) comprised six patients with diabetes in round one and another six patients with diabetes in round two at which point we reached our stop criterion. The participants’ mean age was 54 years, 25% (3/12) were age 65 or over, 58% (7/12) were female, 83% (10/12) were White, and 42% (5/12 ) had limited health literacy. Our Design Sprint yielded an initial MDC mobile-friendly prototype with six main components: (1) infographics to facilitate the understanding and convey significance of four measures of users’ diabetes health (e.g., HbA1C, LDL) including most recent and historic values; (2) info icons providing literacy-level appropriate descriptions of each measure; (3) comparisons of users’ values on diabetes health measures to ‘Patients Like Me’ (i.e., patients of the same age group, gender, and insulin use status); (4) literacy-level appropriate and tailored diabetes self-management educational materials; (5) an embedded link to the American Diabetes Association online support community; (6) notifications when new health data is available to view; and (7) pre-visit planning module encouraging patients to review their data and prepare questions or topics for discussion in advance of their doctor’s visit.
In round one, three tasks (find values, interpret values, pre-visit plan) were successfully completed by all six participants (6/6, 100%). Four tasks (access MDC, find patients like me, interpret patients like me, and view trends) were successfully completed by five participants (5/6, 83%). Three tasks (info icon, view self-management, view pre-visit plan summary) were successfully completed by four participants (4/6, 67%). None of the participants (0/6, 0%) were able to successfully complete the ‘new data’ task where they were asked to find new health data after receiving a notification on the mobile device (e.g., new HgbA1C lab result). Instead of going to MDC, participants searched the messages or results sections of the patient portal, which was where they indicated they normally go when they receive a message about new lab results. Participants statements indicated all found the app to be a positive user experience.
“So for me being diabetic, this, My Diabetes Care would be number one on my list [within the patient portal].”
The mean SUS score —from 0 (worst) to 100 (best)— was 90 (range: 75-100). Based on the results of round one, the interface was modified to better fit the users’ needs including wording, font, and graphical changes. In addition, we changed the way participants were able to access MDC new results by allowing them to directly link to the app screen from a secure message notification.
In round two, seven tasks were completed successfully by all six participants (6/6, 100%); two tasks (find values and pre-visit plan) were completed successfully by five participants (5/6, 83%), and two tasks (info icon and self-management) were completed successfully by four participants (4/6, 67%). One participant was not able to complete two tasks (find values and pre-visit plan) and was an outlier regarding task ratings and SUS results. Overall, participants statements suggested they saw value in the app and viewed it positively.
“I especially like the fact that it has those [past lab values] to look at, because usually when I look at my test results […] then I will look at the past scores immediately.”
The mean SUS score was 86.7 (range: 47.5-100). Without the outlier participant who scored the SUS at the minimum 47.5, the mean SUS score was 94.5 (range: 87.5-100).
Discussion: Our study demonstrates the value of a user-centered design approach and iterative testing in the development of mobile-friendly digital health applications. We were able to rapidly create a high-fidelity prototype and rigorously assess its performance with target users —including users more likely to experience usability challenges— prior to any programming. Iterative testing with a small number of participants and qualitative analysis of participants statements quickly identified usability challenges that led to improved task performance. Participants’ SUS scores demonstrated a high level of satisfaction with the usability of MDC. These findings suggest that the fully programmed version of the mobile-friendly MDC is likely to meet the needs of target users.