Technology Applications: Use of Digital Health Technology to Enable Drug Development
Purpose This pilot study developed and evaluated the feasibility, usability, and perceived satis- faction with an end-user mobile medical application and provider web portal. The two interfaces allowed for remote monitoring, provided daily guidance in the management of hypertension and diarrhea, and allowed for rapid management of adverse events during a clinical trial of olaparib and cediranib. Patients and Methods eCO (eCediranib/Olaparib) was designed for patient self-reported, real-time management of hypertension and diarrhea using remote monitoring. eCO links to a Bluetooth- enabled blood pressure (BP) monitor and transmits data to a secure provider web portal. eCO use was assessed for suitability, usability, and satisfaction after 4 weeks using a 17-item question- naire. Metrics regarding patient-reported BP and diarrhea events were analyzed. Results Sixteen patients enrolled in the pilot. A total of 98.2% of expected BP values were report- ed: 94.2% via Bluetooth and 5.8% entered manually. Twelve patients experienced 21 BP events (systolic BP > 140 and/or diastolic BP > 90 mmHg on two consecutive readings); data from cycle 1 were comparable to the study database. Thirteen patients reported diarrhea (more than one stool per 24 hours over baseline) categorized as grade 1 or 2, which was comparable to the study database. Survey analysis showed that patients had statistically significant, positive responses to the use of the eCO application. Patients indicated eCO use made them feel more involved in their care and better connected to their health care team. The only aspect of the application that did not show a statistically significant positive response was the process of reporting diarrhea. Conclusion The eCO application was designed to assist in managing acute treatment-related events most often associated with treatment discontinuation, need for drug holidays, or dose interruption. Hypertension and
INTRODUCTION
Technological advances have improved the practice of medicine in numerous ways. The use of telemedicine for patient management has been shown to increase patient involvement in care and in reporting observations while on treatment. Therapeutic software, or companion software, is a new area of digital health providing a link between mobile technology and therapies. The management of toxicities related to many new oncologic agents differs from the cytotoxic standards, in part because of continuous admin- istration schedules. Using technology to improve toxicity management for these new oncologic agents may reduce toxicities and subsequent drug discontinuation.Cediranib is a potent inhibitor of vascular endo- thelial growth factor receptors 1, 2, and 3,1 with the antiangiogenic class toxicity of hypertension. This hypertension can be of rapid onset and marked intensity, requiring medical intervention often on day 1 or 2 of cediranib treatment.2 It also causes moderate to severe diarrhea, which has been identified as a major cause of treat- ment discontinuation. International Collaborative for Ovarian Neoplasia (ICON6; ClinicalTrials. gov identifier: NCT00532194), a study of plat- inum-based chemotherapy for first-recurrence platinum-sensitive ovarian cancer, included arms containing concomitant cediranib or concomitant and maintenance cediranib.3,4 The addition of cediranib to chemotherapy resulted in improved progression-free survival; however, cediranibdiscontinuation for toxicity was observed in 27% and 39% of women in the concomitant and concomitant/maintenance arms, respectively. Highly effective symptom management is there- fore a high priority to minimize treatment discon- tinuation of cediranib due to adverse events and maximize clinical benefit.
Recently, the combination of olaparib and cediranib was shown to have an unexpectedly strong benefit for women with recurrent plat- inum-sensitive ovarian cancer, both those with germline BRCA mutations and those with wild- type BRCA status.5 These findings have now led to two ongoing phase III trials in recurrent plat- inum-sensitive (NRG GY004; ClinicalTrials. gov identifier: NCT02446600) and platinum- resistant (NRG GY005; ClinicalTrials.gov identifier: NCT02502266) ovarian cancer. One consider- ation regarding the effectiveness of combination olaparib plus cediranib in the real-world setting is the ability to effectively manage toxicities and prevent early drug discontinuation secondary to adverse effects. In the phase II trial demon- strating significant activity of the olaparib plus cediranib combination, an aggressive interven- tion plan for hypertension and diarrhea was implemented, resulting in only 11% toxicity- associated discontinuation in the combination arm. However, this implementation was feasible because of the limited number of participating sites, many of which had prior experience with the management of cediranib-related toxici- ties. The experiences from this trial and ICON6 emphasize the need for more immediate inter- vention and proactive symptom management; a significant challenge is in how to facilitate this type of management outside of the academic setting.
The electronic cediranib olaparib (eCO) app was therefore developed as an accessible technology-based intervention that could address this need.We hypothesized that the implementation of an end-user smartphone-based app to provide immediate feedback for hypertension on a twice- daily basis and diarrhea as it occurred would improve detection and intervention of these adverse events. We now report on the develop- ment, design, and initial pilot evaluation of the eCO app in women with advanced ovarian can- cer.The process for development of the app, eCO, is outlined in Figure 1. The most appropriate symptoms to address in real time using a mobile app to maintain safety and quality of care were identified as diarrhea and hypertension, on the basis of the results of ICON6.1,5-7 eCO was devel- oped to allow patient self-reporting and real-time triage for toxicity worsening and for documen- tation of these symptoms. Ease of use was a central focus and included minimizing patient training and allowing for use by patients with limited mobile technology experience and for those with reduced dexterity. As a result, eCO was developed to link directly to a Bluetooth- enabled blood pressure (BP) monitor (A&D Con- nected Blood Pressure Device UA651-BLE; A&D Medical, San Jose, CA, provision of which was funded by AstraZeneca) for direct uploading of required twice-daily BP measurements and to function as primary source data. Diarrhea data were collected on an as-needed basis, with stool quality and symptom presence requested to evaluate severity. Patients could opt to use a provided study iPhone 6 (Voluntis, Cambridge, MA) or to use their own iPhone (models 4 to 7).
Algorithms were created to give direct feedback to patients in response to BP measurements or patient self-reported diarrhea, according to study protocol management guidelines and National Cancer Institute (NCI) Common Toxic- ity Criteria for Adverse Events (CTCAE) version4. Feedback included immediate instructions regarding retesting and diet changes and, if appropriate, prompts to contact the health care team via a screen that allowed a one-touch direct phone call to health care contacts. Figure 2 shows examples of BP and diarrhea reporting and recommendations. eCO development was in accordance with the US Food and Drug Adminis- tration Guidance on Mobile Medical Applications and other international standards applicable to software medical devices, such as Interna- tional Electrotechnical Commission 62304 and International Organization for Standardization 13485.8 Risk analysis and risk control measures were implemented for cybersecurity, software failure modes, and use-related errors.A secure, encrypted, redundant, cloud-connected web portal was developed to allow health care providers the ability to view reports and analyze real-time patient data entered in eCO related to diarrhea and hypertension. The clinical team could review patient-specific graphs on home BP results and diarrhea entries, as well as recommendations provided to the patient through the app (Fig 3). E-mail alerts were sent to the patient’s clinical team for clinically criti- cal parameters requiring medical attention or if the patient missed a BP check or recheck. This allowed the study team to evaluate patient status in real time and to contact the patient as needed for appropriate intervention.eCO was determined to be a nonsignificant risk investigational medical device under Investiga- tional Device Exemption. NCI Central Institu- tional Review Board approval was obtained to pilot use of eCO in A Phase 2 Study of Olaparib and Cediranib for the Treatment of Recurrent Ovarian Cancer (ClinicalTrials.gov identifier: NCT02345265; NCI 9825).
The primary objec- tive was to evaluate the suitability, usability, and satisfaction of eCO use. Participating site insti- tutional review boards reviewed the study for individual institution appropriateness. Patients enrolled for treatment could opt to participate in this 4-week pilot study, starting with their cycle-1 day-1 study visit. After completion of informedconsent, patients were trained on the use of the eCO app for BP and diarrhea monitoring. User-Centered Design (Ashburn, VA) evaluated the eCO app/web portal for human factors engi- neering/usability, including a heuristic review and a user-based assessment before fielding. During the pilot, User-Centered Design conducted a 10-minute phone interview with patients after 1 week of use. After 4 weeks of use, patients completed a usability/satisfaction questionnaire (Tables 1 and 2). Health care professionals (HCPs) completed a usability/satisfaction ques- tionnaire at the completion of the pilot study.The patient questionnaire included 17 Likert scale elements focused on overall assessment of usability, usability of specific features (eg, measuring BP and understanding messages), and how eCO made them feel in relation to the clinical trial.
The HCP questionnaire included 14 Likert scale elements focused on the web por- tal and the eCO app, including overall usability assessment, usability of specific features of the app/web portal, and how use of eCO affected their workload. In both questionnaires, the data associated with the app’s usability over- all assessment were derived from the System Usability Scale (SUS), a standard, subjective assessment tool.9 These included aspects such as liking using the app/web portal and the user’s perception that most people would find the app/ web portal easy to learn how to use.Like theSUS, questions were worded both positively and negatively to ensure respondents were reading the questions. Unlike the SUS, which requires a large population to be valid, repeats assess- ments, and combines all data into a single rating for comparison between apps, the eCO ques- tionnaire used single measures to allow analysis of each question with a small population. The questionnaires were validated for respondent understanding using cognitive interviews before being used.Because the questionnaire collected subjective data, we analyzed the data using a Wilcoxon signed rank sum analysis, single population. User responses were converted to positive and negative weighted values. The Wilcoxon signed rank sum analysis analyzed the difference in means between positive and negative responses. A single-tailed test was used to determine statis- tical significance. eCO data output was analyzed using descriptive statistics.
RESULTS
eCO was offered as a technology option to women with recurrent ovarian cancer who were enrolled in the NCI 9825 olaparib plus cediranib study between July 2016 and July 2017. Sixteenpatients enrolled from four participating sites (Dana Farber Cancer Institute, Boston, MA; Mof- fitt Cancer Center, Tampa, FL; National Cancer Institute, Bethesda, MD; Ohio State University Comprehensive Cancer Care Center, Columbus, OH). All completed the pilot study (Table 3). The median age of participants was 58 years (range, 36 to 80 years). No app-related adverse events were reported.Patients using eCO recorded 98.2% of expected home BP values. A total of 94.2% were captured via the Bluetooth-enabled cuff, and 5.8% were entered manually. Twelve patients experienced a total of 21 elevated BP events, defined as two consecutive readings with systolic BP > 140 and/or diastolic BP > 90 mm Hg; six patients had one event each, three patients had two events, and three patients had three events (Table 3). The median duration of a BP event was 9.5 days (range, 3 to 28 days), during which time hypertension was managed by the health care team using the study guidelines. Seven patients reported no associated symptoms of hypertension; when reported, the most common symptoms included headache, change in vision, and shortness of breath. The eCO algorithm prompted the patient to contact the study team for elevated or low BP or for defined symptoms such as headache, chest pain, or shortness of breath. All 12 patients received the prompt to call at least once, and the majority of patients were prompted seven to 20 times on the basis oftheir BP entries. One patient received 54 recom- mendations to call; this patient was noted to be intermittently noncompliant with app recommen- dations to call providers.
CTCAE-graded hyper- tension events as reported in the study database were consistent with the BP events reported fromin the app, and the patient was prompted to call their health care team if they believed their symp- toms were still consistent with an experience of diarrhea. Seven of the 13 patients reported no associated symptoms; when reported, symptoms included severe cramping, diarrhea feels uncon-date of birth; eCO, electronic eCO, with 12 patients with treatment-relatedcediranib olaparib mobile hypertension. Three patients experienced gradeapplication; MR, medical re- 3 hypertension, requiring a maximum of 2 anti-trollable, and blood in stools. Twelve patients had treatment-related diarrhea reported in the study database, with the worst diarrhea being grade 2.cord; SYS, systolic. Adaptedwith permission.Thirteen patients reported a diarrhea event, defined as an increase of more than one total stools over baseline in 24 hours; of these, seven patients reported one event, four patients reported two events, one patient reported three events, and one patient reported four events (Table 3). Twenty-nine of 33 entries were CTCAE grade 1, and four entries were grade 2. No grade 3 or grade 4 entries were recorded. The median duration of each event was 2 days (range, 1 to 8 days). Diarrhea was managed according to study guidelines at each entry. Seven patients made 19 entries that were not diarrhea (number of stools entered less than or equal to baseline with no symptoms); when this occurred, the app pro- vided the patient with how diarrhea was defined Patient usability analysis.
Feedback was cap- tured after the first week of eCO use through a brief phone interview. This interview addressed patient understanding of eCO use and obtained initial feedback on ease of learning and ease of use. No patient reported difficulty with use of the app for BP monitoring. Specific questions reported by one or more patients included dif- ficulty learning how to discard and retake a BP reading and concern over the anxiety-provoking wording of some warning and feedback mes- sages, such as “Call your study team now” for an out-of-range BP. Some patients reported diffi- culty reading the graph format of the BP data in the app’s History section. Interview data showed inconsistent interpretation of 24-hour diarrheaevent recall and lack of awareness of all indi- cators of diarrhea. For example, some patients waited for 24 hours to report a diarrhea event once it started rather than when it occurred.Patients completed the questionnaire after 1 month of app use. The results, shown in Table 1, were consistent with the week 1 interview data. The analysis looked for a difference in median values between people who responded positively and those who responded negatively, providing two samples to compare. The Wil- coxon rank sum analysis single population isspecifically intended to test for differences and determines if the difference in these medi- ans are statistically significant or not. The data showed that patients had statistically significant, positive responses to the use of the eCO app in 16 of the 17 aspects analyzed. Patients reported finding various aspects of the app easy to learn and easy to use, and that use of eCO made them feel more involved in their own care and better connected to their health care team. The only aspect of the app that did not show a statistically significant positive response was the process of reporting diarrhea.
This assessment, as well asthe responses to the phone interview, suggested that this process may not be optimal.HCP usability. HCPs managing patients using eCO were surveyed about the use of the tool’s web-based summary data system (Table 2). Respondents included three physicians and eight research nurses/research coordinators from the four centers. Overall, HCPs were pos- itive about nine of the 14 aspects of the website and app, including learning to use and using the website and app, the availability of information, the value of the alerts, and the effectiveness of using this approach to collect data and moni- tor patients. Four questions showed data that tended toward positive but did not show statisti- cally significant positive responses. These ques- tions covered work efficiency, the nature of alerts generated, better association between alerts and patient instructions, and readability of the datapresented. Feedback comments elaborated on the negative aspects and included uncertainty about which patient triggered an alert, acknowl- edgment when a high BP reading was replaced with a normal BP reading, and a need for a for- mat for BP readings that could be emailed as text instead of as a graph. These recommendations, although not emanating from statistically concor- dant answers, were found to offer constructive directions for future app versions.
DISCUSSION
We hypothesized that use of a smartphone app with which patients would monitor BP and diar- rhea, directly and interactively with their health care team, would improve detection and inter- vention of those adverse events. The eCO app was piloted in a small cohort of patients with ovarian cancer receiving the combination ofolaparib and cediranib. Our experience demon- strated that this type of supportive companion app can be used to perform risk-based moni- toring of target-specific drug-related toxicities, acutely manage these events using protocol- derived embedded algorithms, and result in an enhanced user experience for patients. Patients reported high usability, both in terms of ease of learning and ease of use, with eCO. Hypertension and diarrhea events reported within the eCO app were comparable to the data reported by HCPs in the study database, with the highest CTCAE grades being grade 3 for hypertension and grade 2 for diarrhea. Use of eCO provided support to patients, who reported feeling more closely mon- itored and more connected to their health care team and had a greater involvement in self-care, leading to improved team-based quality patient care. The pilot experience with eCO yielded encouraging results; modifications to eCO will strengthen its supportive features and increaseits impact on patient care. First, the ability for practitioners to adjust alert parameters for BP in a patient-specific manner could increase the ability to provide individually titrated care. Sec- ond, real-time entry of diarrhea providing imme- diate guidance on self-management in a future version should improve on the initial benefits for diarrhea management.
Last, future changes in the app could be expanded to include broader integration of patient-reported outcomes, medi- cation diaries, or text alerts.With the development of new therapies in ovar- ian and other cancers, including oral medica- tions, new challenges in symptom and toxicity management have arisen. These oral therapies can increase patient freedom, because they can often be administered at home, but also give rise to the difficulties of managing adverse effects on a remote basis. Cediranib, for example, has proven to be an active agent in ovarian cancerand is currently being investigated in combina- tion with olaparib as an alternative to intravenous chemotherapy for relapsed disease. However, clinical experience has also shown that the cediranib adverse effects of hypertension and diarrhea require close management to preserve patient safety and avoid drug discontinuation due to toxicity.In early trials of cediranib across multiple dis- ease types, the most prominent adverse effects included fatigue, hypertension, and diarrhea.1,10-12 In the ICON6 trial, adverse effects from cediranib resulted in high drug discontinuation rates, rang- ing from 27% to 39%4. In our own experience of combined olaparib and cediranib, aggres- sive management of cediranib-related adverse effects was found to be critical; in one instance, delayed management of hypertension because of inconsistent patient self-report of elevated BPs resulted in hypertensive crisis, emphasiz- ing the importance of close communication and monitoring between the patient and the treat- ment team. Internal algorithms for managing hypertension, including initiation of antihyper- tensives immediately after onset of hypertension and weekly communications between treatment team and patients, were therefore developed and incorporated into studies to improve hyper- tension management.
Similarly, weekly com- munication with patients and patient education on immediate use of antidiarrheals and initia- tion of dietary modifications with any episodes of diarrhea were implemented. Although these interventions resulted in a much lower drug dis- continuation rate (11%) in the phase 2 trial of olaparib and cediranib, they also require active patient and health care provider involvement. The use of mobile digital health technology is therefore of great interest to support such close symptom management through a more struc- tured and systematic approach.Mobile health technologies have been moving into oncology and have demonstrated benefit to the patient and health care team. Initial studies of mobile phone technology for symptom man- agement in colorectal, lung, and breast cancers have generally reported good data compliance and positive reception from patients.13-16 Growing evidence supports the concept that use of digital health monitoring of patient-reported outcomes in cancer care results in improved outcomes.17-20 Improved overall survival was demonstrated inpatients with lung cancer who completed a web- based questionnaire on symptoms, which was sent to the HCP to determine if intervention was necessary.21 Evaluations of telemonitoring, tele- health including video conferencing, text mes- sages, and mobile phone technology in oncology have been completed or are currently under- way.22-26 These studies support the hypothesis that close monitoring of symptoms and attentive intervention with the support of digital health monitoring is of direct benefit to the patient, while also improving the course of their care.The US Food and Drug Administration now rec- ognizes the importance of remote monitoring and digital patient management. A Current Procedural Terminology code for remote monitoring has been established to support clinician use.
Patient use of an app to initiate protocol-driven symptom treatment on a device was recognized as one of the disruptive dozen for care of patients with can- cer, a group of treatments and modalities iden- tified by The Harvard Medical Group as having the potential to dramatically change medical and health care delivery in the next 10 years.27 Mobile approaches also add value to the execution of clinical trials. All members of the health care and clinical trial team can be made aware contempo- raneously of events occurring during treatment. The app-derived management harmonizes care across all providers and sites, keeping patient management uniform. These improvements could result from closer and more intensive monitoring of adverse events, allowing for interventions that prevent the development of serious toxicities or adverse effects that could be avoidable. The use of these devices may also diminish the number of clinical and hospital visits, thus decreasing the cost of health care delivery.
In addition, early intervention could ameliorate the degree of tox- icity experienced, allowing patients to remain on active and effective doses of therapies for pro- longed periods. Whether the use of this type of device-based intervention will lead to more effec- tive symptom management and overall improved disease outcome remains to be examined.Our experience in this pilot study of eCO demon- strated that a targeted symptom management app can support management of cediranib- related adverse effects and add positively to the patient experience. These findings support further development and exploration of how eCO use affects the ability to successfully managedrug-related toxicities and drug discontinua- tion rates. Patient usability feedback regarding eCO suggested that additional development and refinement regarding diarrhea reporting, such as incorporating measures of diarrhea beyond num- ber of stools alone (eg, stool quality or looseness, urgency) might further improve the patient experi- ence. Further development of eCO and additionalstudies using eCO are currently planned to sup- port the management of cediranib-related toxicities in ovarian and other malignancies.