Gathering high-quality patient data poses challenges for study teams across therapeutic areas, but pediatric trials are notoriously difficult in this regard. In studies with a pediatric cohort, it's not only the patients who are involved in the data collection process: The addition or substitution of caregiver-reported data adds yet another avenue of complexity that can compromise data accuracy. These unique challenges—which characterize, but are not confined to, pediatric studies—offer significant opportunities for the industry to address data subjectivity and quality by developing better ways to measure outcomes.
‍
ObvioHealth's experience has demonstrated the potential of digital instruments to mitigate these obstacles by improving data collection and assessment processes. By leveraging AI-assisted ePRO methods for audio and image capture, these instruments can collect more objective data. That data can then be augmented through Al-assisted expert rating. Adopting this "precision instrumentation" approach to create augmented ePRO can transform clinical trials across therapeutic areas and ultimately enhance clinical practice.

‍

Challenges to Accurate Data Collection in Pediatric Clinical Research

Barriers to Patient-Reported Outcomes

Pediatric research is replete with roadblocks that impede data accuracy.

Logistical challenges for caregivers‍
Participant burden is particularly acute for caregivers, who spend an estimated 40-plus hours each week attending to their children's needs. Recruitment and retention of caregivers in trials requires convenient, low-friction methods to maximize data reporting compliance and study task completion, particularly when multiple caregivers from the same family are involved in a study.

‍

Subjective reporting
Because participant-reported data has a human element, caregiver perceptions may vary, and caregivers and children may report similar data in different ways. Another complicating factor: Caregivers have emotional bonds with their children that can cloud their objectivity, especially when the children are ill. Caregivers may unknowingly inflate the baseline ratings or the impact of a therapeutic intervention, increasing the study's placebo effect.

‍

Observable versus projected behavior
‍When participating in a clinical trial, caregivers must adopt new skills and behaviors. They need to become neutral observers in order to gather and transmit observable data, rather than the projected outcomes or behaviors they may want to see.

‍

‍

‍

Benefits of BYOD ePRO: Improved Compliance ...

In response to these challenges, the industry has increasingly adopted “bring your own device” (BYOD) and other electronic patient-reported outcome (ePRO) models to improve data collection and assessment. The results are promising: ePRO has boosted compliance and accuracy among pediatric trial participants and caregivers. The following examples, among many others in medical literature, show that e-diaries and mobile devices can significantly improve data quality and adherence.

‍

A randomized trial of electronic versus paper pain diaries in children: Impact on compliance, accuracy, and acceptability

Tonya M.* Duare Valenzuela, Paul P. Stork

Study Design: 60 children with headaches or JIA (ages 8 to 16) randomized to complete e-diaries (smartphone) or paper diaries at home daily for 7 consecutive days.

Results: 83.3% of children with e-diaries and 46.7% of children with paper diaries were 100% compliant. Paper diaries contained significantly more errors and omissions than e-diaries (P<0.001).

‍

Brief Report: Assessment of Children’s Gastrointestinal Symptoms for Clinical Trials  

Lynn S. Walker, PhD, and Susan C. Sorrells, PharmD

Vanderbilt University School of Medicine and Glaxo SmithKline, Inc.

Study Design: Irritable bowel syndrome symptoms in pediatric population (6 to 10 years). Subjects (n=11) worked with parents to complete daily diaries using smartphones for 1 week.

Results: Subjects were 100% compliant. Parents reported that smartphones were enjoyable and easy to use. Parents and children willing to participate in a similar study in the future.

‍

… But ePRO Still Has its Limitations

The adoption of ePRO in decentralized or hybrid clinical trials can effectively mitigate the participant burden and, in turn, increase data accuracy. But, ePRO does have its limitations.

ePRO is ideal for measuring patient perceptions. When more objective reporting is required, however, ePRO falls short, impacting not only the accuracy of data reported during the trial but also the accuracy of baseline measurements—and outcome measurement in general.

For example, ObvioHealth conducted a study that used an audio recording device to monitor the duration of infant cries. The data from the device was compared to the frequency and duration of parent-reported crying via an e-diary. The results found that parents inflated the duration of crying by 2 to 4 times.

The findings of this study highlight the risks of relying too heavily on caregiver reporting in studies where objective, accurate outcome measurement is desired.

‍

The Promise of Digital Instruments

What is Augmented ePRO?

In seeking easier and more accurate ways to measure outcomes for pediatric decentralized clinical trials, ObvioHealth has identified opportunities to use digital tools, both on the front and back ends of studies, to augment ePRO and deliver on the promise of more accurate caregiver-reported data.
‍
On the front end, AI-enhanced algorithms can be incorporated into data capture tools to enable participants to record and submit source data with ease. On the back end, an augmented ePRO platform can also facilitate more accurate assessments of patient data by expert raters.

‍

AI-Assisted Audio and Image Capture

‍Al-assisted ePRO supports the capture and submission of unstructured data in the form of audio, images, and video. This advanced ePRO method transforms a caregiver's phone into a smart capture device that allows caregivers to gather observable—and therefore more objective—data in real time, much like a banking app guides its user to capture a clear, quality image of a check in order to make a mobile deposit into an account.
‍
Rather than expecting caregivers to precisely characterize their children's symptoms and behaviors, study teams can direct them to record and upload high-quality data with ease.

‍

AI-assisted audio capture

‍Advanced audio capture tools—both in the form of unique digital instruments or capture-support applications that leverage built-in smartphone features—allow study teams to gather and assess sounds that might be difficult to quantify or might otherwise go undetected. For instance, study teams can create an Al-supported audio capture feature that employs the microphone of a mobile device, designing it to automatically detect and record the nighttime crying of infants. The only burden left to caregivers is to submit the data via the app when prompted. For instruments recording constant, unstructured ambient noise, Al algorithms can be applied on multiple levels: to detect the onset of the audio outcome, to measure its duration or intensity, and to separate it from superfluous sound.

‍

AI-assisted auto-photography

‍During clinical trials with participant-submitted images, study teams frequently contact patients to request higher-quality images or to intervene when a photo includes protected health information. Low-resolution, out-of-focus, or underexposed images can affect the accuracy of clinical ratings. Al-driven capture-support applications leverage the built-in features of smartphones to take quality images. Smart-capture capabilities can correct poor lighting, autofocus the camera lens, and flag distracting objects in the background. And, Al-assisted image-capture tools, such as framing indicators, help participants take high-quality "selfies" by focusing the camera on the relevant area of the body, prompting them to take photos from multiple angles, and informing them of any face angle misalignments.

‍

Leveraging AI to Optimize Expert Rating

Of course, more data is not the end goal—better data is. In addition to streamlining data collection and submission, Al-assisted digital tools can help clinicians analyze and measure patient-reported and caregiver-reported data for optimized expert rating.

In the case of measuring infant cries, a study team may have 24 hours' worth of audio recordings to review. An Al-assisted platform allows the study team to cluster the important sounds together, filtering out unwanted noise. Human annotators can contribute to the training of Al and machine learning algorithms, which will ultimately streamline the expert rating process. With Al supporting the expert rating process, technology performs the repetitive "heavy lifting" involved in assessing participant submissions, so raters can focus on tasks that require their specific expertise.

Al-assisted rating of ePRO also helps to resolve inter-rater variability, addressing the human subjectivity inherent in expert scoring. An ObvioHealth study recently demonstrated the importance of objectivity in the expert rating of unstructured data. The study collected and compared stool ratings from three separate touchpoints in the ePRO collection and assessment process: caregivers, central raters, and pediatric gastroenterologists. The study's results point to inter-rater variability—among caregivers, raters, and clinicians alike—supporting the need for further research into standardized training or the automated scoring of the Bristol Stool Scale.

‍

Putting the Tools into Practice

Two recent studies from ObvioHealth show the effectiveness of AI-assisted capture and assessment of unstructured data to augment ePRO:

‍

Infant/Toddler Stool Study

Objective

A baby nutrition client sought to create a database of stool images that could be used for internal software development purposes.

Methodology

ObvioHealth recruited 100 mothers of toddlers virtually. We developed the protocol, eCRF, prescreening, and eICF. Mothers captured images of their babies’ stools using their smartphones. They were trained to grade them using the Brussels scale. Images were stratified into seven stool consistency types and validated by two expert scorers.

Results

Enrollment target was reached in 12 weeks. Over the course of the study, 2,366 images were collected, and the virtual study design facilitated strong engagement, resulting in a 95% retention rate. Expert scorers identified some biases in mothers’ scores, highlighting a tendency to over-report those within the “normal” range.

‍

Infant Crying/Fussing Study

Objective

ObvioHealth was asked to conduct an observational study of infant crying and fussing.

Methodology

Caregivers recorded their infants’ crying and fussing, using the ObvioHealth app to indicate their perception of each event through daily questionnaires and e-diaries. A device inserted into an infant onesie also recorded infant crying. Mandatory training and ongoing communication with subjects minimized user-interface issues and improved data integrity.

Results

Reported crying and fussing frequency and duration was 2- to 4-times higher than the device audio-recorded events. The results provided insights into human bias.

‍

Implications for Clinical Practice

The benefits of augmented ePRO extend well beyond clinical trials. Cataloguing a broader range of unstructured data allows care teams to develop more precise baseline measurements for medical conditions. Clinicians can then leverage these enriched data sets in clinical practice to make more accurate diagnoses.
‍
For instance, technology improvements can support early intervention in biliary atresia, a life-threatening congenital liver disorder. Clinicians can educate young providers, and possibly even caregivers, to check for changes in stool color that could signal this or other disorders in newborn children.
‍
Augmented ePRO is also advancing the treatment of asthma by delivering more nuanced data on respiratory symptoms. Researchers can use digital instruments to measure children's usage of inhalers and record their breathing sounds while using inhalers, providing a comprehensive set of structured and unstructured data for expert assessment.

‍

Conclusion

Pediatric clinical trials have traditionally posed some of the industry’s most complex challenges. The emergence of digital instruments and electronic patient-reported outcomes has mitigated some of these difficulties but not fully resolved the subjectivity inherent in caregiver relationships. Augmented ePRO represents the next level of patient-reported outcomes—leveraging precision instrumentation and the wonders of artificial intelligence to deliver better data and streamline the experience at every touchpoint.

‍