Hostname: page-component-78c5997874-j824f Total loading time: 0 Render date: 2024-11-02T06:51:45.782Z Has data issue: false hasContentIssue false

Source Data Verification (SDV) quality in clinical research: A scoping review

Published online by Cambridge University Press:  21 May 2024

Muayad Hamidi*
Affiliation:
University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Eric L. Eisenstein
Affiliation:
Duke University, Durham, NC, USA
Maryam Y. Garza
Affiliation:
University of Arkansas for Medical Sciences, Little Rock, AR, USA
Kayla Joan Torres Morales
Affiliation:
University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Erika M. Edwards
Affiliation:
University of Vermont, Vermont Oxford Network, Burlington, VT, USA
Mitra Rocca
Affiliation:
University of Arkansas for Medical Sciences, Little Rock, AR, USA
Amy Cramer
Affiliation:
Janssen R&D LLC, Raritan, NJ, USA
Gurparkash Singh
Affiliation:
Janssen R&D LLC, Raritan, NJ, USA
Kimberly A. Stephenson-Miles
Affiliation:
ICON PLC employee on an assignment to Janssen R&D LLC, Dublin, Ireland
Mahanaz Syed
Affiliation:
University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Zhan Wang
Affiliation:
University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Holly Lanham
Affiliation:
University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Rhonda Facile
Affiliation:
Clinical Data Interchange Standards Consortium (CDISC), Austin, TX, USA
Justine M. Pierson
Affiliation:
OpenClinica, Needham Heights, MA, USA
Cal Collins
Affiliation:
OpenClinica, Needham Heights, MA, USA
Henry Wei
Affiliation:
Regeneron, Tarrytown, NY, USA
Meredith Zozus
Affiliation:
University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
*
Corresponding author: M. Hamidi; Email: hamidim@uthscsa.edu
Rights & Permissions [Opens in a new window]

Abstract

Introduction:

The value of Source Data Verification (SDV) has been a common theme in the applied Clinical Translational Science literature. Yet, few published assessments of SDV quality exist even though they are needed to design risk-based and reduced monitoring schemes. This review was conducted to identify reports of SDV quality, with a specific focus on accuracy.

Methods:

A scoping review was conducted of the SDV and clinical trial monitoring literature to identify articles addressing SDV quality. Articles were systematically screened and summarized in terms of research design, SDV context, and reported measures.

Results:

The review found significant heterogeneity in underlying SDV methods, domains of SDV quality measured, the outcomes assessed, and the levels at which they were reported. This variability precluded comparison or pooling of results across the articles. No absolute measures of SDV accuracy were identified.

Conclusions:

A definitive and comprehensive characterization of SDV process accuracy was not found. Reducing the SDV without understanding the risk of critical findings going undetected, i.e., SDV sensitivity, is counter to recommendations in Good Clinical Practice and the principles of Quality by Design. Reference estimates (or methods to obtain estimates) of SDV accuracy are needed to confidently design risk-based, reduced SDV processes for clinical studies.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of Association for Clinical and Translational Science

Introduction

Clinical trial complexity continues to rise and increases the effort required at clinical investigational sites [Reference Getz and Campo1Reference Getz, Wenger, Campo, Seguine and Kaitin5]. This has contributed to clinical trial operational inefficiency being considered one of the major impediments to Clinical and Translational Research [Reference Austin6]. The rate of clinical trial cost increase – driven by study complexity and operational inefficiency – is greater than inflation for other segments of the economy [Reference DiMasi, Grabowski and Hansen7Reference Scannell, Blanckley, Boldon and Warrington9]. An estimated 46% percent of on-site monitoring time has been attributed to Source Data Verification (SDV) [Reference Breslauer10]. Estimates for the portion of clinical trial costs attributable to SDV range from 25% to 40%, implicating SDV as a major cost driver [Reference Breslauer10Reference Funning, Grahnén, Eriksson and Kettis-Linblad13]. Reducing the amount of manual SDV in clinical trials would create an opportunity to increase operational efficiency and lower clinical trial costs.

The comparison of study data to the medical record (or other sources) to verify that the medical record data are accurately reflected in the study data is called SDV (definitions Table 1). Since the earliest reported use in the year 1746 [Reference Gibbs14], SDV has been used as a tool to find and fix errors from medical record abstraction (MRA), the process associated with the largest error rate in data processing [Reference Zozus, Pieper and Johnson15].

Table 1. Key definitions relevant to source document verification

Extensive and often 100%, SDV was historically considered necessary to ensure study data quality and has served as the foundation upon which trialists claimed data accuracy and authenticity [Reference Manasco and Bhatt23,Reference Nielsen, Hyder and Deng24]. Cognitively similar to MRA, SDV is a manual inspection process performed by humans, and the error rate is likely similarly high. Finding the twenty-one differences between Figure 1a, b illustrates challenges with SDV as a mechanism to identify data errors.

Figure 1. Visual inspection exercise. © 2020 Highlights for Children, Inc. All rights reserved. Permission to reproduce and distribute this page is granted by Highlights for Children.

Decades ago, the field of HRA demonstrated that errors in the manual inspection process had a significant and often overlooked impact on the average outgoing quality [Reference Drury and Fox25]. SDV has long been used as a tool to assure data quality. However, the quality of this tool itself was rarely examined. Ironically, maintaining a current calibration record for all devices utilized in a clinical trial is required, while no comparable requirement exists for the more fallible processes of manual abstraction from medical record sources and manual inspection by SDV. Though at reduced levels through risk-based approaches, SDV is still being used as a major strategy to identify errors and ensure high-quality data [Reference Houston, Probst and Martin26Reference Ward28]. This thinking must change. As a fallible manual process, SDV was never capable of assuring high-quality data from a fallible source [Reference Drury and Fox25,Reference Megaw29]. SDV provided a convenient answer, “all data were verified against the source,” but the assumption that often followed, “therefore the data are correct,” was flawed. This leaves us with the uncomfortable difficulty of needing to verify the accuracy of data in the absence of a true gold standard, a conundrum that likely perpetuated reliance on manual SDV.

Reports of SDV error and omission started to emerge with studies evaluating various risk-based monitoring approaches[Reference Tantsyura, Dunn and Waters30,Reference Andersen, Byrjalsen and Bihlet31], and along with them, acknowledgment that manual SDV processes are not capable of producing error-free data. In this context, the value of the traditional 100% SDV has increasingly been questioned [Reference Olsen, Bihlet, Kalakou and Andersen11,12,Reference de Vries32Reference Tantsyura, Dunn, Fendt, Kim, Waters and Mitchel34]. In a recent national study, respondents characterized the “amount of money, time and resources spent on certain monitoring methods” such as extensive on-site SDV, as being wasteful for commercial trials [Reference Houston, Yu, Martin and Probst35]. Federally funded and investigator-initiated studies traditionally employed more limited monitoring approaches due to budget constraints [Reference Brosteanu, Houben and Ihrig36Reference Chakravarthy, Cotter, DiMasi, Milne and Wendel39]. With the shift toward risk-based approaches articulated in the United States Food and Drug Administration’s (FDA’s) guidance on risk-based monitoring in 2013 [19] and the 2018 revision of the Good Clinical Practice (GCP) guidelines [40], industry and academic monitoring practices are converging with risk-based approaches now strongly encouraged by prominent industry groups [18,41,42] and regulators [19,21,43]. Prior to the COVID-19 pandemic, however, industry adoption of risk-based monitoring practices, such as off-site remote monitoring, reduced Source Document Review (SDR), reduced SDV, and centralized monitoring (definitions in Table 1), remained low, 25%, 16%, 20%, and 16% respectively [Reference Stansbury, Barnes and Adams44].

Only moderate evidence supports the comparability of traditional (100%) and reduced SDV [Reference Klatte, Pauli-Magnus and Love27]. European Medicines Agency (EMA) has reached beyond the status quo and called for demonstration that alternate monitoring methods are non-inferior to traditional methods [45]. GCP guidelines and the FDA, through Quality by Design (QbD) principles [16,41], have communicated general heightened expectations for quality planning and the design or selection of study processes with the capability to deliver the data accuracy required to support planned study analyses. However, computational models for the a priori design of capable reduced SDV processes are not common, and the inputs needed for the computational models, such as error rates for SDV processes, have not been widely reported. The resulting uncertainty in process capability is a likely contributor to the slow adoption of reduced SDV, along with a lack of comparative evidence, lingering concerns of feasibility, the effort required to change existing organizational processes, and methodological questions regarding reduced SDV. Filling these knowledge gaps will provide the means to design and optimize SDV for clinical studies. We conducted this review to minimize this knowledge gap.

Objective

The goals of this scoping review were (1) to identify published reports of SDV quality and (2) to characterize and summarize the evidence regarding the quality of the SDV performed in clinical studies.

Methods

The Preferred Reporting Items for Systematic Reviews and Meta-analysis extension for Scoping Reviews (PRISMA-ScR) [Reference Tricco, Lillie and Zarin46] methodology was adopted (Supplementary Material 1 - PRISMA-ScR-Fillable-Checklist). The protocol components, as outlined in the PRISMA-Scr, were standardized a priori and used by authors, though the protocol was not registered in a review protocol registry such as Open Science Framework.

Eligibility criteria

Studies were included if (1) the main focus was the quality of the SDV process, i.e., problems with or ways to achieve the quality of the SDV process, (2) the full-text article was available, and (3) the article was written in the English language. Studies were excluded if they (1) were not focused on SDV quality, (2) were not full-text peer-reviewed articles, e.g., abstracts, posters, regulations, and policies, or (3) were not in English. All types of study designs were included; however, policy documents, guidelines, and regulations were excluded.

Information sources

The following databases were searched: MEDLINE, CINAHL, PsycINFO, Embase, Web of Science, Scopus, Cochrane, The Association of Clinical Research Professionals, The Society of Clinical Research Associates, Applied Clinical Trials (ACT), and Google Scholar were used as a safety net to confirm that our search process didn’t miss any relevant papers. The last search was executed in October 2022. In addition, the list of similar articles is suggested by the NLM website next to each search result.

Search

The following PubMed query was used: (“SDV”[Title/Abstract] OR “clinical trial monitoring”[Title/Abstract]) AND (“clin res”[Journal] OR “clinical research”[Title/Abstract] OR “clinical trial”[Title/Abstract] OR “clinical study”[Title/Abstract]) with the syntax modified as needed to run on the other databases.

Selection of sources of evidence the database searches were performed by three authors (MM, KTM, MNZ), with the initial screening of titles and abstracts performed by two authors (MM, MNZ). The articles that passed the initial screening were retrieved for full-text review. Two independent people reviewed the full text of the retrieved articles and attempted consensus on disagreements. Disagreements for which consensus could not be achieved were adjudicated by a majority vote of authors on weekly adjudication calls.

Data extraction process

A spreadsheet (Supplementary Material 2 – Information abstracted from articles) was created to capture the data needed to be extracted. All authors independently participated in abstracting a predefined list of relevant Information from each included article.

Data items

Information abstracted from the included articles is listed in (Table 2), which can be grouped into article metadata, information about research design and context, and information about SDV accuracy.

Table 2. Information abstracted from articles

CER = Clinical Evaluation Report; FDA = The Food and Drug Administration; HSR = Health Services Research; NCT Number = National Clinical Trial number; NIH = National Institutes of Health; PCT = Pragmatic Clinical Trial; RCT = Randomized Controlled Trial; SDV = Source Data Verification.

Critical appraisal of individual sources of evidence

As a scoping review, we sought to characterize all available evidence regarding SDV quality. Since we anticipated a wide variety of evidence, we critically apprized the strength of evidence according to the research design and context of the study.

Synthesis of results

We grouped the studies by study design and context. Examples include prospective versus retrospective approach and experimental versus quasi-experimental, pre-experimental, or descriptive designs. Use of comparators and control, and whether the study measured SDV discrepancies or SDV errors were also used to categorize studies as well as aspects of the research context such as the number of studies, sites, or participants included in the quality assessment, the therapeutic area in which the SDV quality assessment was conducted, and whether the study was industry-funded.

Results

Selection of sources of evidence

The search of bibliographic databases yielded 683 records, and an additional 72 were identified from searching citations (paper references). (Fig. 2). From the total of 755 records identified, there were 207 duplicate records removed, 360 excluded per the Title and Abstract screening process, and 6 unretrievable. In total, 182 articles (110 plus 72 in Fig. 2) were retrieved and underwent full-text review. The full-text review eligibility assessment resulted in the exclusion of 154 articles (91 plus 63 in Fig. 2) and the inclusion of 28 articles. Sixteen of the included articles reported quantitative results.[Reference Mealer, Kittelson and Thompson17,18,Reference Andersen, Byrjalsen and Bihlet31,Reference Mitchel, Cho and Gittleman33,Reference Brosteanu, Schwarz and Houben37,Reference Kim, Kim and Hong38,Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47Reference Yamada, Chiu and Takata56] Twelve articles reported non-quantitative studies [Reference Manasco and Bhatt23,Reference Houston, Probst and Martin26Reference Ward28,Reference de Vries32,Reference Seachrist57Reference Korieth63] (Fig. 2).

Figure 2. Flow diagram [Reference Page, McKenzie and Bossuyt64]. ACRP = The Association of Clinical Research Professionals; ACT = Applied Clinical Trials; EMBASE = Excerpta Medica database; PsycINFO = American Psychological Association PsycInfo database; SOCRA = The Society of Clinical Research Associates; WoS = Web of Science database.

Characteristics of sources of evidence

Quantitative studies

Seven of the included quantitative studies prospectively collected data after assigning the patient, site, or study to some monitoring strategy [Reference Mealer, Kittelson and Thompson17,Reference Brosteanu, Schwarz and Houben37,Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47,Reference Tudur Smith, Stocken and Dunn48,Reference Fougerou-Leurent, Laviolle and Tual52,Reference Wyman Engen, Huppler Hullsiek and Belloso54,Reference Kondo, Kamiyoshihara and Fujisawa55]. Nine analyzed existing data [18,Reference Andersen, Byrjalsen and Bihlet31,Reference Mitchel, Cho and Gittleman33,Reference Kim, Kim and Hong38,Reference Sheetz, Wilson and Benedict49Reference Embleton-Thirsk, Deane and Townsend51,Reference Giganti, Shepherd and Caro-Vega53,Reference Yamada, Chiu and Takata56] (references detailed in Table 3). Ten studies were classified as using pre-experimental designs, two were classified as quasi-experimental, and four were classified as experimental studies (references reported in Table 3; study details reported in Supplementary Material 3).

Table 3. Research design summary for the included quantitative studies

Non-quantitative evidence

A total of 12 non-quantitative articles were included, nine opinions [Reference Manasco and Bhatt23,Reference de Vries32,Reference Seachrist57Reference Korieth63], and three reviews [Reference Houston, Probst and Martin26Reference Ward28]

Opinion: Two of the nine opinion papers reported a clinical trial quality event that served as a major inflection point in thinking and approach [Reference Seachrist57,Reference Cohen58]. These articles reported National Institutes of Health (NIH) tightening the clinical trials monitoring requirements in response to the finding of fraud in the National Surgical Adjuvant Breast and Bowel Project study group [Reference Seachrist57,Reference Cohen58]. At the time, clinical trial monitoring practices, including SDV, varied considerably across multicenter studies funded by the NIH [Reference Cohen58]. Six opinion articles [Reference Manasco and Bhatt23,Reference de Vries32,Reference Tantsyura, Dunn, Fendt, Kim, Waters and Mitchel34,Reference Schuyl and Engel59Reference Duda, Wehbe and Gadd61] provided guidance on different aspects of SDV or practice recommendations to improve the SDV process, multiple of them with the purpose of prompting change. One provided insight into the cost of SDV [Reference Korieth63].

Reviews: three literature reviews were included [Reference Houston, Probst and Martin26Reference Ward28] one review assessed SDV empirically [Reference Klatte, Pauli-Magnus and Love27] (2021 Klatte) adopted the Cochrane methodology which included only prospective empirical studies. The two remaining included reviews [Reference Houston, Probst and Martin26,Reference Ward28] focused on SDV practices. Houston et al. (2018) reported a wide range of SDV methods existing in practice with no best practice evident [Reference Houston, Probst and Martin26]. Similarly, Ward’s review documented the absence of methodological guidelines for SDV, the wide variety in reported practice, and the lack of empirical evidence regarding the impact of SDV [Reference Ward28], which stood out as the earliest identified graduate research presented as a thesis in a master’s degree program.

Discussion

Critical appraisal within sources of evidence

In comparison to the Cochrane review, which included only prospective empirical studies, this review includes a much broader range of evidence from retrospective empirical studies, other reviews, and opinion papers. Though we did not explicitly assign a strength of evidence or risk of bias rating, we fully acknowledge that the retrospective, descriptive, and qualitative papers provide inherently weaker evidence due to threats to internal validity and risk of bias. For example, with one exception, there were no measures of SDV quality reported prior to broadened interest in RBM. However, the two-dimensional framework for evaluating SDV quality (Table 3) would not have emerged without considering the outcomes measured in the retrospective studies. As a scoping review, these sources were included to provide a comprehensive compilation of the available literature and findings across the spectrum of research designs while at the same time recognizing the difference in evidence strength.

The recent systematic literature reviews for Good Clinical Data Management Practices also found scant Cochrane-strength evidence for practice recommendations [Reference Hills, Bartlett, Leconte and Zozus65Reference Pestronk, Johnson and Muthanna69]. In these reviews, randomized, controlled experiments of operational processes were few and far between. It may be that this standard is not a feasible expectation for evidence-supporting practice in clinical research operations. On the other hand, it still seems ironic that the methods from which strong evidence is generated are not themselves held to that same standard. Without such a standard, some may interpret the literature regarding SDV accuracy as sufficient to recommend against SDV, while others looking at the same literature will refrain from such recommendations.

Results of individual sources of evidence and synthesis of results

The included quantitative studies exhibited significant heterogeneity along multiple dimensions. We identified four reported aspects of SDV quality (rows in Table 4): (1) source data availability and access, (2) data quality, (3) study process fidelity, and (4) SDV process fidelity. Furthermore, reports for the SDV quality domains naturally fell into three main categories (columns in Table 4): (1) SDV quality measures, such as rates of findings, missing data, or process problems, (2) the impact of SDV on decreasing future problems, and (3) the impact of SDV on study results. One of the included articles assessed SDV quality on all three levels [Reference Giganti, Shepherd and Caro-Vega53]. Three studies reported a measure of source data availability and access [Reference Mealer, Kittelson and Thompson17,Reference Embleton-Thirsk, Deane and Townsend51,Reference Yamada, Chiu and Takata56] (Table 4). The data quality aspect (domain) of SDV quality (second row in Table 4) was reported as data discrepancies and errors. Six articles reported counts or rates of data discrepancies [Reference Mealer, Kittelson and Thompson17,18,Reference Andersen, Byrjalsen and Bihlet31,Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47,Reference Tudur Smith, Stocken and Dunn48,Reference Fougerou-Leurent, Laviolle and Tual52], while seven reported measures of errors [Reference Mitchel, Cho and Gittleman33,Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47,Reference Sheetz, Wilson and Benedict49,Reference Embleton-Thirsk, Deane and Townsend51,Reference Giganti, Shepherd and Caro-Vega53,Reference Kondo, Kamiyoshihara and Fujisawa55,Reference Yamada, Chiu and Takata56]. An error is a discrepancy with the truth, i.e., an incorrect data value. A discrepancy, on the other hand, is a difference between two data values where either or both could be in error. Discrepancies are often reported when the correct value cannot be determined or was not determined. In the included articles, some reported database changes were made after investigating and resolving discrepancies; we counted these as reports of errors.

Table 4. Aspects of source data verificaiton (SDV) quality and the level at which they were reported

* These assessments were done relative to another method rather than as a quantification of absolute errors. i.e., the studies have not measured SDV actual accuracy, instead, surrogates for accuracy were measured, such as a number of data discrepancies or errors missed relative to some other method.

Includes delivery of the intervention, endpoint Assessment, regulations, guidance, and other requirements to which the protocol must comply.

The third and most frequently reported aspect of SDV quality was the ability to detect or propensity to miss study process errors; we refer to this as study process fidelity (Table 4). The included articles varied greatly concerning the study processes for which SDV quality was reported. Eight articles reported SDV quality with respect to the detection of safety issues [Reference Brosteanu, Schwarz and Houben37,Reference Kim, Kim and Hong38,Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47Reference Sheetz, Wilson and Benedict49,Reference Embleton-Thirsk, Deane and Townsend51,Reference Wyman Engen, Huppler Hullsiek and Belloso54,Reference Kondo, Kamiyoshihara and Fujisawa55]. One reported that SDV improved safety process fidelity over time [Reference Brosteanu, Schwarz and Houben37]. And four reported the impact of SDV on safety-related study results [Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47,Reference Tudur Smith, Stocken and Dunn48,Reference Embleton-Thirsk, Deane and Townsend51,Reference Wyman Engen, Huppler Hullsiek and Belloso54]. Eligibility, informed consent, and protocol adherence were tied as the second most frequently reported aspects of study process fidelity (Table 4). These aspects encompass the EMA (2017) integrated inspection report categories used to summarize and evaluate the potential implications of major or critical findings, “the impact on the integrity of the trial data, the rights, wellbeing, and safety of the subjects, the compliance of the trial with GCP (including ethical principles) …” [70]. Within each SDV quality aspect, the included articles varied in whether and, if so, how the impact of SDV was evaluated. All sixteen included quantitative articles reported counts or rates of items detected or missed by SDV (Table 4, Fig. 3). Eleven [Reference Mealer, Kittelson and Thompson17,18,Reference Andersen, Byrjalsen and Bihlet31,Reference Mitchel, Cho and Gittleman33,Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47Reference Sheetz, Wilson and Benedict49,Reference Fougerou-Leurent, Laviolle and Tual52,Reference Giganti, Shepherd and Caro-Vega53,Reference Kondo, Kamiyoshihara and Fujisawa55,Reference Yamada, Chiu and Takata56] of the sixteen quantitative articles reported measures of data discrepancies or errors. Two articles [Reference Brosteanu, Schwarz and Houben37,Reference Giganti, Shepherd and Caro-Vega53] evaluated the impact of SDV on quality improvement over time within a study, whereas nine [Reference Andersen, Byrjalsen and Bihlet31,Reference Mitchel, Cho and Gittleman33,Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47,Reference Tudur Smith, Stocken and Dunn48,Reference Embleton-Thirsk, Deane and Townsend51Reference Wyman Engen, Huppler Hullsiek and Belloso54,Reference Yamada, Chiu and Takata56] evaluated the impact of SDV findings on study results (Table 4, Fig. 3). However, five [Reference Andersen, Byrjalsen and Bihlet31,Reference Mitchel, Cho and Gittleman33,Reference Fougerou-Leurent, Laviolle and Tual52,Reference Wyman Engen, Huppler Hullsiek and Belloso54,Reference Yamada, Chiu and Takata56] of the articles reporting the impact of SDV findings on study results did so qualitatively, for example, stating that the errors occurred evenly across treatment groups, that the errors occurred in non-critical variables, or that the frequency or extent of the errors was too small to have impacted the analysis. The remaining four [Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47,Reference Tudur Smith, Stocken and Dunn48,Reference Embleton-Thirsk, Deane and Townsend51,Reference Giganti, Shepherd and Caro-Vega53] articles reporting the impact of SDV findings on study results did so by comparing analyses before and after error correction.

Figure 3. Heterogeneity in SDV* quality assessment. Each 3x4 grid in the figure represents SDV quality assessments reported in one included, quantitative article. The SDV methods compared are listed at the bottom of each grid, with NR signifying not reported, extensive signifying high amounts of data values undergone up to 100% SDV, and mixed signifying a combination of two or more SDV methods. *Source Data Verification (SDV): the comparison of study data to their original recording to ensure that, “the reported trial data are accurate, complete, and verifiable from source documents [16].”

In addition to variability in the aspects of the SDV quality measured and the level at which they were reported (Table 4), the included articles exhibited just as much variability in the context in which they were measured. For example, seven assessments were done in industry clinical trials versus nine in investigator-initiated studies (Fig. 3). The included articles were heterogeneous with respect to the SDV process for which the quality was measured. The most common SDV process variants assessed included remote, targeted, and varied extents of reduced SDV (Fig. 3, Supplementary Material 3). All reported SDV quality assessments were relative, for example, comparing the number of SDV findings missed by one method that was subsequently detected by another, or reporting the portion of SDV findings that resulted in database changes. The reported assessments differed in how discrepancies or errors were identified, i.e., which two SDV processes were compared to identify items missed by one of the methods but detected by the other. The assessments differed in whether these discrepancies were verified. Multiple included studies reported the number of data discrepancies detected by SDV, while others reported those missed by SDV. One study [Reference Brosteanu, Schwarz and Houben37] comprehensively reported the number of discrepancies detected and missed by SDV but did so for only one parameter, informed consent (Supplementary Material 3). Further limiting the quantitative synthesis, the unit of analysis was inconsistent across included studies and included counts of individual findings, rates of findings per patient, rates of findings per site, or proportions of patients or sites with one or more findings. Additionally, some studies reported only major findings, while others reported all findings. These differences are detailed in Supplementary Material 3. Although multiple studies quantified one or more aspects of SDV accuracy, no article reported a comprehensive measure of SDV accuracy. No study reported the absolute rate of errors, i.e., detected against the gold standard of truth. One study [Reference Fougerou-Leurent, Laviolle and Tual52] declared 100% SDV to be the gold standard but did not compute accuracy measures (sensitivity and specificity) against it.

The search strategy broadly encompassed articles focused on SDV as well as those focused on clinical study monitoring; relevant reports of SDV quality were found in both types of articles. For example, a study comparing 100% SDV versus reduced SDV that quantified items missed by 100% SDV provided an assessment of SDV quality. Similarly, a study comparing triggered versus non-triggered on-site visits that quantified items missed by SDV provided an assessment of SDV quality.

The majority of included articles reported SDV quality in the context of comparing RBM (including targeted, remote, or reduced SDV) to traditional monitoring approaches usually characterized by more extensive SDV. The heterogeneity, such as differences in the amount, timing, or frequency of SDV, in articles included in the quantitative synthesis (Table 4 and Supplementary Material 3) means that the assessments of SDV quality are not comparable; the methodological heterogeneity limited this synthesis to a scoping review.

The evaluative studies identified and included in this review are varied in terms of which domains of SDV quality are reported for SDV (rows in Table 4). Reported SDV quality domains included accessibility of source data, data error rates, GCP or protocol deviations (often called monitoring findings), and audit-identified deviations in the SDV process. Further, these SDV quality domains were reported at multiple levels, including accessibility of source data needed to identify errors, rates of identified errors, effectiveness at preventing future errors, and impact of the identified errors on study outcomes (columns in Table 4). SDV likely has utility on each level, however, reports at the prevention and study results levels were few compared to counts or rates of findings. A comprehensive evaluation of SDV quality would include the rows and columns in Table 4 as well as items both identified and missed by SDV.

SDV accuracy cannot be calculated without a gold standard and enumeration of true positives, false positives, true negatives, and false negatives. One study [Reference Fougerou-Leurent, Laviolle and Tual52] declared 100% on-site SDV as the gold standard; however, the authors acknowledge that errors remain in 100% SDV’d data and did not report accuracy measures of sensitivity and specificity. Thus, we did not find a quantitative report of absolute SDV accuracy (sensitivity and specificity).

Related findings

There is a good number of reviews and surveys. Despite the fact that they didn’t meet the inclusion criteria, they reported related beneficial results, and we opted to present them. Seven relevant reviews [Reference Olsen, Bihlet, Kalakou and Andersen11,Reference Houston, Probst and Martin26,Reference Klatte, Pauli-Magnus and Love27,Reference Hurley, Shiely and Power71Reference Macefield, Beswick, Blazeby and Lane73] were identified by the search (Fig. 4). Three of the identified reviews met the inclusion criteria (bottom row in Fig. 4). The reviews differed in their scope, with the three included reviews specifically addressing SDV and the four excluded reviews focusing more broadly on clinical trial monitoring or other aspects thereof (Fig. 4). Four of the reviews focused on methods; two included only empirical assessment of monitoring or SDV outcomes, and one review, Olsen et al. (2016), included both methods and empirical results (Fig. 4). While the Ward (2013) survey did not meet inclusion criteria, the systematic review portion of this work did and is included (Fig. 4). Only the three reviews, including Ward (2013), specifically addressed SDV were included in this review.

Figure 4. Categorization of review articles. 1Source Data Verification: the comparison of study data to their original recording to ensure that, “the reported trial data are accurate, complete, and verifiable from source documents [16].”

The Cochrane review by Klatte et al. (2021) included eight prospective empirical studies that compared different monitoring strategies [Reference Klatte, Pauli-Magnus and Love27]. Five of them are also included in our review. Overall, the Cochrane review concluded with moderate certainty that “risk-based monitoring is not inferior to extensive on-site monitoring with respect to critical and major monitoring findings in clinical trials” and noted that “more high-quality monitoring studies that measure effects on all outcomes specified in this review are necessary to draw more reliable conclusions.” The Cochrane review did not directly address the accuracy of the SDV process itself with respect to the identification of errant data.

Eight surveys [Reference Funning, Grahnén, Eriksson and Kettis-Linblad13,Reference Ward28,Reference Odette Jochems and Mountain7479] were identified through the search, all but one [Reference Ward28] were excluded due to minimal focus on SDV quality. However, because each survey contained one or more SDV-relevant questions, we have listed them in Supplementary Material 4 for completeness. Across the surveys, perceptions varied widely with respect to the impact of SDV or monitoring on data quality. Collectively, the survey work indicates variability in SDV frequency, amount, and methods similar to that seen in the quantitative, included articles and in the included reviews. Multiple articles reporting survey results called for additional research on SDV methods, the impact of SDV, and more specific guidelines for methods, including the amount of SDV. However, their message is weakened by the limited generalizability of each survey and the significant changes in context and practice over the almost 30-year span over which survey results were reported.

Evidence throughout the literature, from opinion to experimental studies, supports the ability of SDV to identify unreported events. While reports of SDV-identified significant or systematic findings certainly exist [Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy47,Reference Tudur Smith, Stocken and Dunn48,Reference Embleton-Thirsk, Deane and Townsend51,Reference Fougerou-Leurent, Laviolle and Tual52], the included articles reporting the impact of missed-event-type findings on study results indicated no significant impact [Reference Klatte, Pauli-Magnus and Love27,Reference Brosteanu, Schwarz and Houben37,Reference Embleton-Thirsk, Deane and Townsend51]. Similarly, multiple included studies concluded a lack of impact of SDV-identified data errors on study results[Reference Andersen, Byrjalsen and Bihlet31,Reference Tudur Smith, Stocken and Dunn48,Reference Embleton-Thirsk, Deane and Townsend51]. However, others found SDV-identified data discrepancies or errors impactful on one or more study analyses [Reference Mealer, Kittelson and Thompson17,Reference Giganti, Shepherd and Caro-Vega53]. These apparent differences may reflect differences in the SDV quality domains assessed, differences in measurement methods, differences in the SDV processes themselves, or differences in other aspects of data collection and processing. For example, one of the included studies assessed the type, frequency, and impact of data errors on an observational study and concluded that data errors identified through SDV would have otherwise impacted the study results [Reference Giganti, Shepherd and Caro-Vega53]. However, few upstream data quality control measures were in place. Similarly, two articles concluding no impact of SDV-identified data error measured SDV error for query-clean data and data collected on structured site worksheets (rather than abstracted from medical records) – the upstream data quality control measures described would have likely significantly decreased the number of errors remaining to be detected by SDV. The many-faceted heterogeneity precludes drawing conclusions.

With rising cost pressure, a wide variety of options continue to be explored. The Cochrane review concludes, albeit on what the authors deem to be moderate to low-quality evidence, that there is likely no difference between the different monitoring (and SDV) strategies tested in the five comparisons assessed by the review. We posit that this could be due to the likely high error rate of SDV itself, i.e., less of an error-prone inspection may not yield markedly worse overall quality. One of the most convincing studies comparing the outgoing error rate from RBM to traditional monitoring concluded the same, noninferiority [Reference Andersen, von Sehested, Byrjalsen, Popik, B. and Bihlet80].

Remote access to Electronic Health Records (EHRs) for monitoring and the ability to extract data directly from them greatly impact study monitoring and SDV processes. The justification for pursuing such EHR-to-eCRF data collection includes increasing data quality by decreasing manual medical record abstraction and decreasing the data collection burden at sites and Sponsors. In this case, SDV can be performed by computationally confirming that study data match the EHR source or may obviated altogether as mentioned in the FDA eSource guidance [81]. Eliminating manual SDV for EHR-to-eCRF data would likely reduce burden and cost. Given advances in technology and permissible regulatory guidance, available innovation will likely be applied to SDV, such as (1) establishing traceability back to the source that can be computationally traversed to demonstrate that the final data exactly reflects the medical record source and (2) defining a certified copy in the context of data extracted from EHRs such that sponsors and regulators are guaranteed that the final data are a replica of the source. These cannot be accomplished with manual SDV.

Limitations

Though performed systematically, our search could have missed an important article. The database searches were conducted over a 5-month period which may result in differences (though minimal) of returned articles. The methodological variation observed in the included papers was huge, making it difficult to draw conclusions from the body of the literature. For example, no two reported instances of reduced SDV were exactly alike. Similarly, the data processing and quality control applied to data prior to SDV were often not completely described. Significant differences were observed in available descriptions of upstream data processing, such as performing SDV against the medical record versus against structured site source worksheets. The contributions of SDV versus SDR (defined in Table 1) often could not be distinguished from one another. SDV includes reading the source record to ensure that there were no omissions in the study data which includes some amount of SDR. In fact, multiple included articles remarked that SDV was conducted by reading the full source. For this reason, findings attributed to SDV may have also been found or may be equally findable through SDR. Additionally, the variability in the terminology used in the literature could have easily led to inadvertent misclassification of studies with respect to the SDV process measured, the measures used, and the methods by which they were obtained.

Conclusions

The review exposes significant heterogeneity in the SDV processes measured, the measures used, and the methods by which they were obtained. Though multiple studies quantified one or more aspects of SDV quality, we did not find an article that reported an assessment of SDV quality covering the full set of domains and levels identified in the included articles. Accuracy is not among the reported measures of SDV quality. Either the heterogeneity or the absence of accuracy measures alone is sufficient to preclude reporting, much less comparing, measurements of SDV quality found in the literature.

Due to the likely context sensitivity of SDV, QbD should be applied to ensure that new SDV approaches will not adversely impact human safety and research results. Additional research is needed to develop methods of designing study processes capable of delivering the necessary outgoing quality. Estimates of process and inspection accuracy are needed to support prospective process design.

The variability and error associated with SDV as a manual process suggest opportunities for improvement through advances such as automation and decision support.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/cts.2024.551.

Author contributions

All authors participated in collecting, analysing, and interpreting data and results. In addition to participating and commenting on the manuscript drafted by M. Hamidi, E. and M. Zozus, all authors approved the final version for publication.

Funding statement

This work was partially supported by an Innovation in Regulatory Science Award from the Burroughs Wellcome Fund and the Clinical and Translational Science Award from the National Center for Advancing Translational Science, a component of the National Institutes of Health, to the University of Texas Health Science Center at San Antonio.

Competing interests

None.

References

Getz, KA, Campo, RA. New benchmarks characterizing growth in protocol design complexity. Ther Innov Regul Sci. 2017;52 (1):2228.CrossRefGoogle ScholarPubMed
Sung, NS, Crowley, WF Jr., Genel, M, et al. Central challenges facing the national clinical research enterprise. JAMA. 2003;289 (10):12781287.CrossRefGoogle ScholarPubMed
Eisenstein, EL, Lemons, PW ll, Tardiff, BE, Schulman, KA, Jolly, MK, Califf, RM. Reducing the costs of phase III cardiovascular clinical trials. Am Heart J. 2005;149 (3):482488.CrossRefGoogle ScholarPubMed
Eisenstein, EL, Collins, R, Cracknell, BS, et al. Sensible approaches for reducing clinical trial costs. Clin Trials. 2008;5 (1):7584.CrossRefGoogle ScholarPubMed
Getz, KA, Wenger, J, Campo, RA, Seguine, ES, Kaitin, KI. Assessing the impact of protocol design changes on clinical trial performance. Am J Ther. 2008;15 (5):450457.CrossRefGoogle ScholarPubMed
Austin, CP. Opportunities and challenges in translational science. Clin Transl Sci. 2021;14 (5):16291647. doi: 10.1111/cts.13055.CrossRefGoogle ScholarPubMed
DiMasi, JA, Grabowski, HG, Hansen, RW. Innovation in the pharmaceutical industry: new estimates of R&D costs. J Health Econ. 2016;47:2033.CrossRefGoogle ScholarPubMed
Moore, TJ, Zhang, H, Anderson, G, Alexander, GC. Estimated costs of pivotal trials for novel therapeutic agents approved by the US food and drug administration, 2015-2016. JAMA Intern Med. 2018;1 (11):14511457. doi: 10.1001/jamainternmed.2018.3931.CrossRefGoogle Scholar
Scannell, JW, Blanckley, A, Boldon, H, Warrington, B. Diagnosing the decline in pharmaceutical R&D efficiency. Nat Rev Drug Discov. 2012;1 (3):191200. doi: 10.1038/nrd3681.CrossRefGoogle Scholar
Breslauer, C. Could source document verification become a risk in a fixed-unit price environment? Monitor. 2006;(Dec): 4347.Google Scholar
Olsen, R, Bihlet, AR, Kalakou, F, Andersen, JR. The impact of clinical trial monitoring approaches on data integrity and cost--a review of current literature. Eur J Clin Pharmacol. 2016;72 (4):399412. doi: 10.1007/s00228-015-2004-y.CrossRefGoogle ScholarPubMed
Institute of Medicine. Assuring data quality and validity in clinical trials for regulatory decision making: workshop report, Roundtable on Research and Development of Drugs, Biologics, and Medical Devices, Division of Health Sciences Policy. National Academy Press; 1999:88.Google Scholar
Funning, S, Grahnén, A, Eriksson, K, Kettis-Linblad, Å. Quality assurance within the scope of good clinical practice (GCP) - what is the cost of GCP-related activities? A survey within the Swedish association of the pharmaceutical industry (LIF)’s members. Qual Assur J. 2009;12 (1):37. doi: 10.1002/qaj.433.CrossRefGoogle Scholar
Gibbs, D. For debate: 250th anniversary of source document verification. BMJ. 1996;313 (7060):798798. doi: 10.1136/bmj.313.7060.798.CrossRefGoogle Scholar
Zozus, MN, Pieper, C, Johnson, CM, et al. Factors affecting accuracy of data abstracted from medical records. PLoS One. 2015;10 (10):e0138649.CrossRefGoogle ScholarPubMed
E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1); International Council for Harmonisation; Guidance for Industry; Availability. Vol. 83. 2018:8882. 0097-6326.Google Scholar
Mealer, M, Kittelson, J, Thompson, BT, et al. Remote source document verification in two national clinical trials networks: a pilot study. PLoS One. 2013;8 (12):e81890. doi: 10.1371/journal.pone.0081890.CrossRefGoogle ScholarPubMed
Trancelerate BioPharma Inc. Reveals collaborative methodology for risk-based monitoring of clinical trials. Biotech Business Week. 2013:111.Google Scholar
U.S. Food and Drug Administration. Guidance for Industry Oversight of Clinical Investigations — A Risk-Based Approach to Monitoring, vol. 78. Maryland: FDA, 2013:48173. 0097-6326.Google Scholar
Adachi, K, Shirase, M, Kimura, Y, Kuboki, Y, Yoshino, T. What and how will the risk-based approach to monitoring change? Survey of RBM in medical institutions. J Soc Clin Data Manag. 2021. doi: 10.47912/jscdm.18.CrossRefGoogle Scholar
European Medicines Agency (EMA). Reflection Paper on Risk Based Quality Management in Clinical Trials. London, UK: European Medicines Agency, 2013.Google Scholar
U.S. Food and Drug Administration. Guidance for Industry Q8(R2) Pharmaceutical Development. Maryland: FDA, 2009.Google Scholar
Manasco, P, Bhatt, DL. Evaluating the evaluators - developing evidence of quality oversight effectiveness for clinical trial monitoring: source data verification, source data review, statistical monitoring, key risk indicators, and direct measure of high risk errors. Contemp Clin Trials. 2022;117: 106764. doi: 10.1016/j.cct.2022.106764.CrossRefGoogle ScholarPubMed
Nielsen, E, Hyder, D, Deng, C. A data-driven approach to risk-based source data verification. Ther Innov Regul Sci. 2014;48 (2):173180. doi: 10.1177/2168479013496245.CrossRefGoogle ScholarPubMed
Drury, CG, Fox, JG. Human Reliability in Quality Control. London, UK: Taylor & Francis Ltd, 1975:315.Google Scholar
Houston, L, Probst, Y, Martin, A. Assessing data quality and the variability of source data verification auditing methods in clinical research settings. J Biomed Inform. 2018;83:2532. doi: 10.1016/j.jbi.2018.05.010.CrossRefGoogle ScholarPubMed
Klatte, K, Pauli-Magnus, C, Love, SB, et al. Monitoring strategies for clinical intervention studies. Cochrane Database Syst Rev. 2021;12 (12):MR000051.Google ScholarPubMed
Ward, R. Examining Methods and practices of source data Verification in Canadian critical care randomized controlled trials. Thesis submitted to the Faculty of Graduate and Postdoctoral Studies in Partial fulfillment of the requirements for the Master of Science degree in Epidemiology. Ottawa, Canada: University of Ottawa, 2013. https://ruor.uottawa.ca/bitstream/10393/23974/1/Ward_Roxanne_2013_thesis.pdf.Google Scholar
Megaw, ED. Review of : “human reliability in quality control” edited by C. G. DRURY and J. G. Fox. (London: Taylor & Francis Ltd., 1975.) [Pp. xii+316.] £7 00. Appl Ergonom. 1976;19 (5):649650. doi: 10.1080/00140137608931579.CrossRefGoogle Scholar
Tantsyura, V, Dunn, IM, Waters, J, et al. Extended risk-based monitoring model, on-demand query-driven source data verification, and their economic impact on clinical trial operations. Ther Innov Regul Sci. 2016;50 (1):115122. doi: 10.1177/2168479015596020.CrossRefGoogle ScholarPubMed
Andersen, JR, Byrjalsen, I, Bihlet, A, et al. Impact of source data verification on data quality in clinical trials: an empirical post hoc analysis of three phase 3 randomized clinical trials. Br J Clin Pharmacol Apr. 2014;79 (4): 660668. doi: 10.1111/bcp.12531.CrossRefGoogle Scholar
de Vries, BJ. SDV in good clinical trial practice. Good Clin Pract J. 1996;3 (1):1517.Google Scholar
Mitchel, JT, Cho, T, Gittleman, DA, et al. Time to change the clinical trial monitoring paradigm. Appl Clin Trials. 2014;23:1–7.Google Scholar
Tantsyura, V, Dunn, IM, Fendt, K, Kim, YJ, Waters, J, Mitchel, J. Risk-based monitoring: a closer statistical look at source document verification, queries, study size effects, and data quality. Ther Innov Regul Sci. 2015;49 (6):903910. doi: 10.1177/2168479015586001.CrossRefGoogle Scholar
Houston, L, Yu, P, Martin, A, Probst, Y. Clinical researchers’ lived experiences with data quality monitoring in clinical trials: a qualitative study. BMC Med Res Methodol. 2021;21 (1):187. doi: 10.1186/s12874-021-01385-9.CrossRefGoogle ScholarPubMed
Brosteanu, O, Houben, P, Ihrig, K, et al. Risk analysis and risk adapted on-site monitoring in noncommercial clinical trials. Clin Trials. 2009; 6 (6):585596. doi: 10.1177/1740774509347398.CrossRefGoogle ScholarPubMed
Brosteanu, O, Schwarz, G, Houben, P, et al. Risk-adapted monitoring is not inferior to extensive on-site monitoring: results of the ADAMON cluster-randomised study. Clin Trials. 2017;14 (6):584596. doi: 10.1177/1740774517724165.CrossRefGoogle Scholar
Kim, S, Kim, Y, Hong, Y, et al. Feasibility of a hybrid risk-adapted monitoring system in investigator-sponsored trials in cancer. Ther Innov Regul Sci. 2021;55 (1):180189. doi: 10.1007/s43441-020-00204-5.CrossRefGoogle ScholarPubMed
Chakravarthy, R, Cotter, K, DiMasi, J, Milne, CP, Wendel, N. Public- and private-sector contributions to the research and development of the most transformational drugs in the Past 25 Years: from theory to therapy. Ther Innov Regul Sci. 2016;50 (6):759768. doi: 10.1177/2168479016648730.CrossRefGoogle Scholar
International Council for Harmonization (ICH). E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R2) Guidance for Industry. 2018. https://www.fda.gov/downloads/Drugs/Guidances/UCM464506.pdf.Google Scholar
Clinical Trials Transformation Initiative (CITTI). Quality By Design (QBD) Toolkit. https://ctti-clinicaltrials.org/our-work/quality/qbd-quality-by-design-toolkit/. Accessed September 15, 2022.Google Scholar
eClinical Forum. Risk-based approaches - best practices for ensuring clinical trial data quality. 2013. https://eclinicalforum.org/downloads/risk-based-approaches-best-practices-for-ensuring-clinical-data-quality. Accessed September 15, 2022.Google Scholar
UK DoHSC. Medicines & healthcare products regulatory agency (MHRA). Risk-adapted approach to clinical trials and risk assessments. In: UK DoHSC, eds. GOV.UK. London, UK: Department of Health and Social Care, 2022.Google Scholar
Stansbury, N, Barnes, B, Adams, A, et al. Risk-based monitoring in clinical trials: increased adoption throughout 2020. Ther Innov Regul Sci. 2022;56 (3):415422. doi: 10.1007/s43441-022-00387-z.CrossRefGoogle ScholarPubMed
Procedure for reporting of GCP inspections requested by the Committee for Medicinal Products for Human Use (CHMP) dated 28th March 2017, EMA/INS/GCP/158549/2016 Rev. 1, good clinical practice inspectors working group (GCP IWG).Google Scholar
Tricco, AC, Lillie, E, Zarin, W, et al. PRISMA extension for scoping reviews (PRISMA-scR): checklist and explanation. Ann Int Med. 2018;169 (7): 467473. doi: 10.7326/M18-0850.CrossRefGoogle ScholarPubMed
Maruszewski, B, Lacour-Gayet, F, Monro, JL, Keogh, BE, Tobota, Z, Kansy, A. An attempt at data verification in the EACTS congenital database. Eur J Cardiothorac Surg. 2005;28 (3):400404. doi: 10.1016/j.ejcts.2005.03.051.CrossRefGoogle ScholarPubMed
Tudur Smith, C, Stocken, DD, Dunn, J, et al. The value of source data verification in a cancer clinical trial. PLoS One. 2012;7 (12):e51623. doi: 10.1371/journal.pone.0051623.CrossRefGoogle Scholar
Sheetz, N, Wilson, B, Benedict, J, et al. Evaluating source data verification as a quality control measure in clinical trials. Ther Innov Regul Sci. 2014;48 (6):671680. doi: 10.1177/2168479014554400.CrossRefGoogle ScholarPubMed
Agrafiotis, DK, Lobanov, VS, Farnum, MA, et al. Risk-based monitoring of clinical trials: an integrative approach. Clin Ther. 2018;40 (7):12041212. doi: 10.1016/j.clinthera.2018.04.020.CrossRefGoogle ScholarPubMed
Embleton-Thirsk, A, Deane, E, Townsend, S, et al. Impact of retrospective data verification to prepare the ICON6 trial for use in a marketing authorization application. Clin Trials. 2019;16 (5):502511. doi: 10.1177/1740774519862528.CrossRefGoogle Scholar
Fougerou-Leurent, C, Laviolle, B, Tual, C, et al. Impact of a targeted monitoring on data-quality and data-management workload of randomized controlled trials: a prospective comparative study. Br J Clin Pharmacol. 2019;85 (12):27842792. doi: 10.1111/bcp.14108.CrossRefGoogle ScholarPubMed
Giganti, MJ, Shepherd, BE, Caro-Vega, Y, et al. The impact of data quality and source data verification on epidemiologic inference: a practical application using HIV observational data. BMC Public Health. 2019;19 (1):1748. doi: 10.1186/s12889-019-8105-2.CrossRefGoogle ScholarPubMed
Wyman Engen, N, Huppler Hullsiek, K, Belloso, WH, et al. A randomized evaluation of on-site monitoring nested in a multinational randomized trial. Clin Trials. 2020;17 (1):314. doi: 10.1177/1740774519881616.CrossRefGoogle Scholar
Kondo, H, Kamiyoshihara, T, Fujisawa, K, et al. Evaluation of data errors and monitoring activities in a trial in Japan using a risk-based approach including central monitoring and site risk assessment. Ther Innov Regul Sci. 2021;55 (4):841849. doi: 10.1007/s43441-021-00286-9.CrossRefGoogle Scholar
Yamada, O, Chiu, SW, Takata, M, et al. Clinical trial monitoring effectiveness: remote risk-based monitoring versus on-site monitoring with 100% source data verification. Clin Trials. 2021;18 (2):158167. doi: 10.1177/1740774520971254.CrossRefGoogle ScholarPubMed
Seachrist, L. NIH tightens clinical trials monitoring. Science. 1994; 264 (5158):499499. doi: 10.1126/science.8160006.CrossRefGoogle ScholarPubMed
Cohen, J. Clinical trial monitoring: hit or miss? Science. 1994;264 (5165): 15341537. doi: 10.1126/science.8202707.CrossRefGoogle ScholarPubMed
Schuyl, ML, Engel, T. A review of the source document verification process in clinical trials. Drug Inform Assoc J. 1999;33 (3):789–797.Google Scholar
Lorstad, M. Data quality of the clinical trial process - costly regulatory compliance at the expense of scientific proficiency. Qual Assur J. 2004; 8:177–182. doi: 10.1002/qaj.288.CrossRefGoogle Scholar
Duda, SN, Wehbe, FH, Gadd, CS. Desiderata for a computer-assisted audit tool for clinical data source verification audits. Stud Health Technol Inform. 2010;160 (Pt 2):894898.Google ScholarPubMed
Tantsyura, V. Risk-based source data verification approaches: prons and cons. Drug Inform Assoc J. 2010;8615 (2010):745756.CrossRefGoogle Scholar
Korieth, K. The high cost and questionable impact of 100% SDV. Central Watch Mon. 2011;18 (2):15–17.Google Scholar
Page, MJ, McKenzie, JE, Bossuyt, PM, et al. The PRISMA 2020 Statement: an updated guideline for reporting systematic reviews. Plos Med. 2021;18 (3):e1003583. doi: 10.1371/JOURNAL.PMED.1003583.CrossRefGoogle ScholarPubMed
Hills, K, Bartlett, T, Leconte, I, Zozus, MN. CRF completion guidelines (CCGs). Data Basic Soc Clin Data Manag. 2020;26 (1)):3355.Google Scholar
Lebedys, E, Famatiga-Fay, C, Bhatkar, P, Johnson, D, Viswanathan, G, Zozus, MN. Good clinical data management practices data management plan (DMP) chapter. Soc Clin Data Manag. 2020;26:74100.Google Scholar
Eade, D, Pestronk, M, Russo, R, et al. Web-based electronic data capture (EDC) study implementation and start-up. J Soc Clin Data Manag. 2021;1 (1):1–24.Google Scholar
Montano, O, Johnson, D, Muthanna, M, et al. Electronic data capture (EDC) – study conduct, maintenance and closeout. J Soc Clin Data Manag. 2021;1 (1):1–22.Google Scholar
Pestronk, M, Johnson, D, Muthanna, M, et al. Electronic data capture (EDC) selecting an EDC system. J Soc Clin Data Manag. 2021;1 (1):1–19.Google Scholar
European Medicines Agency (EMA). Procedure for Reporting of GCP Inspections Requested by the Committee for Medicinal Products for Human Use (CHMP). 2017.Google Scholar
Hurley, C, Shiely, F, Power, J, et al. Risk based monitoring (RBM) tools for clinical trials: a systematic review. Contemp Clin Trials. 2016;51:1527. doi: 10.1016/j.cct.2016.09.003.CrossRefGoogle ScholarPubMed
Bakobaki, J, Joffe, N, Burdett, S, Tierney, J, Meredith, S, Stenning, S. A systematic search for reports of site monitoring technique comparisons in clinical trials. Clin Trials. 2012;9 (6):777780. doi: 10.1177/1740774512458993.CrossRefGoogle ScholarPubMed
Macefield, RC, Beswick, AD, Blazeby, JM, Lane, JA. A systematic review of on-site monitoring methods for health-care randomised controlled trials. Clin Trials Feb. 2013;10 (1):104124. doi: 10.1177/1740774512467405.CrossRefGoogle ScholarPubMed
Odette Jochems, JJ, Mountain, NJ. David R hutchinson source data verification in the Netherlands, Belgium and the UK: results of a survey to establish the current pharmaceutical industry approach. Survey Eur J Clin Res. 1993;4:4548.Google Scholar
Hurley, C, Sinnott, C, Clarke, M, et al. Perceived barriers and facilitators to risk based monitoring in academic-led clinical trials: a mixed methods study. Trials. 2017;18 (1):423. doi: 10.1186/s13063-017-2148-4.CrossRefGoogle ScholarPubMed
Houston, L, Probst, Y, Yu, P, Martin, A. Exploring data quality management within clinical trials. Appl Clin Inform. 2018;9 (1):7281. doi: 10.1055/s-0037-1621702.Google ScholarPubMed
Morrison, BW, Cochran, CJ, White, JG, et al. Monitoring the quality of conduct of clinical trials: a survey of current practices. Clin Trials. 2011;8 (3):342349. doi: 10.1177/1740774511402703.CrossRefGoogle ScholarPubMed
Hamrell, MR, Mostek, K, Goldsmith, L. Monitoring of clinical trials - are remote activities helpful in controlling quality? Clin Res. 2016;30(5):18–21.Google Scholar
C Kunzl, B Breuer, Y Rollinger, M Sigmund, V Kunert. RBM-An update of experiences among European CRAs Applied Clinical Trials. 2017. https://www.appliedclinicaltrialsonline.com/view/rbm-update-experiences-among-european-cras. Accessed October 19, 2017.Google Scholar
Andersen, JR, von Sehested, C, Byrjalsen, I, Popik, S, B., FA, Bihlet, AR. Impact of monitoring approaches on data quality in clinical trials. Br J Clin Pharmacol. 2022;89 (6):111. doi: 10.1111/bcp.15615.Google Scholar
U.S. Food and Drug Administration. Guidance for Industry Electronic Source Data in Clinical Investigations. Maryland: FDA, 2013.Google Scholar
Figure 0

Table 1. Key definitions relevant to source document verification

Figure 1

Figure 1. Visual inspection exercise. © 2020 Highlights for Children, Inc. All rights reserved. Permission to reproduce and distribute this page is granted by Highlights for Children.

Figure 2

Table 2. Information abstracted from articles

Figure 3

Figure 2. Flow diagram [64]. ACRP = The Association of Clinical Research Professionals; ACT = Applied Clinical Trials; EMBASE = Excerpta Medica database; PsycINFO = American Psychological Association PsycInfo database; SOCRA = The Society of Clinical Research Associates; WoS = Web of Science database.

Figure 4

Table 3. Research design summary for the included quantitative studies

Figure 5

Table 4. Aspects of source data verificaiton (SDV) quality and the level at which they were reported

Figure 6

Figure 3. Heterogeneity in SDV* quality assessment. Each 3x4 grid in the figure represents SDV quality assessments reported in one included, quantitative article. The SDV methods compared are listed at the bottom of each grid, with NR signifying not reported, extensive signifying high amounts of data values undergone up to 100% SDV, and mixed signifying a combination of two or more SDV methods. *Source Data Verification (SDV): the comparison of study data to their original recording to ensure that, “the reported trial data are accurate, complete, and verifiable from source documents [16].”

Figure 7

Figure 4. Categorization of review articles. 1Source Data Verification: the comparison of study data to their original recording to ensure that, “the reported trial data are accurate, complete, and verifiable from source documents [16].”

Supplementary material: File

Hamidi et al. supplementary material 1

Hamidi et al. supplementary material
Download Hamidi et al. supplementary material 1(File)
File 86.4 KB
Supplementary material: File

Hamidi et al. supplementary material 2

Hamidi et al. supplementary material
Download Hamidi et al. supplementary material 2(File)
File 37 KB
Supplementary material: File

Hamidi et al. supplementary material 3

Hamidi et al. supplementary material
Download Hamidi et al. supplementary material 3(File)
File 14.7 MB
Supplementary material: File

Hamidi et al. supplementary material 4

Hamidi et al. supplementary material
Download Hamidi et al. supplementary material 4(File)
File 16.5 KB