Introduction
There are growing demands for accountability and a recognized need for constant quality improvement (QI) in primary care practices (Government of Ontario, 2012). Performance measurement is often used to identify and address strengths and weaknesses through data collection and the provision of feedback (Simpson and Birdshell, Reference Simpson and Birdshell2006; WHO, 2008). However traditional methods of data collection, particularly those measuring patient-reported outcomes, can be laborious and costly and many primary care practices lack the time and resources to implement them (Stroebel et al., Reference Stroebel, McDaniel, Crabtree, Miller, Nutting and Stange2005; Parker et al., Reference Parker, de Pillis, Altschuler, Rubenstein and Meredith2007; Ivers et al., Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012; Creswell, Reference Creswell2013). Thus for tools to be applicable to primary care settings they must cater to practice capabilities and concerns and also be efficient, minimally disruptive, and cost effective (Stroebel et al., Reference Stroebel, McDaniel, Crabtree, Miller, Nutting and Stange2005; Parker et al., Reference Parker, de Pillis, Altschuler, Rubenstein and Meredith2007; Government of Ontario, 2012; Ivers et al., Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012). In looking for such methods, we conducted a scoping review of the literature on automated performance measurement (APM) through the use of interactive voice response technology (IVR), in order to assess lessons for its potential use in primary care health services research. For this review APM is defined, using elements from two sources, as: the use of telephones to administer computer-driven surveys, utilizing computerized or pre-recorded voices and touch tone response or speech recognition, to gather performance data (Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007; Shaw and Verma, Reference Shaw and Verma2007). We specifically sought to identify the benefits and challenges in using an IVR approach (that includes any type of IVR system using a telephone only) to APM to determine feasibility for primary care practices’ quality improvement efforts and practice-based research studies.
Methods
In order to broadly explore this technology, we adopted a scoping review method outlined by Arksey and O’Malley (Reference Arksey and O’Malley2005) and Levac et al. (Reference Levac, Colquhoun and O’Brien2010). Our goal was to identify relevant lessons from published literature on the potential of an emerging technology, automated interactive voice response surveys, for primary care health services research and practice-based quality improvement. We sought all relevant literature regardless of study design, to record the approach used, and main findings, both broad and specific. We searched the MEDLINE database from the year 2000 to 2013 using keywords related to interactive voice response, performance measurement, and primary care. A list of the search terms and combined searches may be found in Appendix. International articles reporting the use of IVR for any reason in any health care practice setting were included if they were research studies or literature reviews. Articles were excluded if they were not published in English, contained a non-health care-related research question, the abstract was not available through Medline, or their interactive voice response systems were not conducted through the telephone.
M.F. ran the search and screened the titles and abstracts for meeting the inclusion criteria. Included articles at this stage underwent full text review by M.F. Their references were also reviewed for relevant sources. Articles were assessed for relevance to use of IVR for health care performance measurement such as applicable methods or population, as only a limited number addressed performance measurement specifically. Articles discussing interactive voice response survey technology in other health care contexts outside of primary care were also included to broaden the search. Included articles underwent in depth review and data extraction, the rest were excluded following initial review because they were not relevant to IVR and automated performance measurement.
Studies using IVR for a broad range of functions such as health intervention, follow-up calls, disease management, health education, and health databases were analysed alongside performance measurement articles due to a lack of data specifically on performance measurement. We extracted data from included studies on study goals, population/context, method of IVR implementation, type of IVR administration (call in/call out), survey questions, response rates, number of follow-up calls required, data quality, cost information, and factors reported as facilitating or hindering integration into a clinical practice. Descriptive data from each article was analysed by M.F., S.J., and W.H. to identify lessons for the use of interactive voice response technology for performance measurement in primary care. We identified common findings across studies, and paid attention to disconfirming results and areas requiring further investigation. To promote the rigour of the study, our analysis team was multidisciplinary including two clinician primary care health services researchers and one non clinician health systems researcher. Further we presented our early analysis findings to several health services researchers and an information technology expert in the field of IVR incorporating feedback into our ongoing assessment of relevance to primary care. We synthesized the main findings relevant for primary care, based on recurrence between studies and/or the impact or potential impact for performance measurement. These are presented under categories in the result section.
Results
The initial search retrieved 1987 articles, 123 of which were selected for full text review. 10 additional sources were added after searching the references of included articles. In all, 33 articles were included for in depth review and data extraction. A summarized figure of the search results is shown in Figure 1.
Figure 1 Scoping review
A total of 33 articles were included in our search and addressed interactive voice response technology use in health care but very few specifically addressed primary care and performance measurement. Five studies utilized IVR for collection of patient satisfaction data while 14 studies used IVR alongside patient services such as monitoring and education. Multiple study types were reviewed, including 14 randomized controlled studies, other experimental designs, observational studies, and systematic reviews. Five studies were literature reviews of IVR or related technologies. The studies used diverse methods, sample populations, implementations of IVR, and outcomes measures. In all, 27 studies had sample populations ranging from 36 to 58 440, with a total of 117 692 participants across all studies reviewed. An additional study included three practices as their sample population and the remainder of studies were literature reviews. All studies were set in the United States except for one that was conducted in Canada by Reid et al. (Reference Reid, Pipe, Quinlan and Oda2007).
This section presents the results most pertinent to the topic of interactive voice response for automated performance measurement. The following tables present brief summaries of each source grouped by study focus. Table 1 summarizes the methods and relevant findings of studies specifically evaluating IVR survey methods. Table 2 summarizes the findings of studies which combined IVR surveys with another intervention. The articles detailing technological specifications or implementation issues are summarized in Table 3. The studies investigating mode effect are described in Table 4, and Table 5 presents the literature reviews on this topic. Highly relevant or commonly reported aspects of IVR surveys are presented below.
Table 1 IVR methods and findings from included studies relevant to potential use for primary care patient-reported outcome measures
IVR=interactive voice response.
Table 2 IVR methods and relevant findings from included studies which paired IVR with another intervention
IVR=interactive voice response.
Table 3 Studies detailing IVR technological specifications and/or implementation issues
IVR=interactive voice response; EMR=electronic medical record.
Table 4 Studies investigating mode effects
IVR=interactive voice response.
Table 5 Literature reviews on IVR
IVR=interactive voice response.
IVR features
Several sources discussed the multiple features that interactive voice response technology provides. IVR can reach multiple simultaneous participants, across a wide geographic spread, in multiple languages and accents, and most people own and know how to use a phone (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Brodey et al., Reference Brodey, Rosen, Brodey, Sheetz and Unutzer2005; Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007; Estabrooks et al., Reference Estabrooks, Shoup, Gattshall, Dandamudi, Shetterly and Xu2009; Oake et al., Reference Oake, Jennings, van Walraven and Forster2009; Dalal et al., Reference Dalal, Demuro-Mercon, Lewis, Nelson, Gilligan and McLeod2010; Lundy and Coons, Reference Lundy and Coons2012; Skolarus et al., Reference Skolarus, Holmes-Rovner, Hawley, Dunn, Barr, Willard, Wei, Piette and An2012). IVR may use computerized voice or pre-recorded messages, can ask multiple choice and open-ended questions, and can randomize participants to receive different messages or survey questions (Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007; Welker, Reference Welker2007; Willig et al., Reference Willig, Krawitz, Panjamapirom, Ray, Nevin, English, Cohen and Berner2013). IVR allows skipping and branching of irrelevant questions to save participant time and can call-out to patients during specific time intervals or allow patients to call-in at their own convenience (Lee et al., Reference Lee, Friedman, Cukor and Ahern2003; Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007; Shaw and Verma, Reference Shaw and Verma2007; Naylor et al., Reference Naylor, Keefe, Brigidi, Naud and Helzer2008; Shea et al., Reference Shea, Guerra, Weiner, Aguirre, Ravenell and Asch2008). Finally IVR technology can be integrated with databases and enter information directly into tables and graphs (Aharonovich et al., Reference Aharonovich, Greenstein, O’Leary, Johnston, Seol and Hasin2012).
Data quality
IVR produced data quality comparable to other modes in multiple studies (Weiler et al., Reference Weiler, Christ, Woodworth, Weiler and Weiler2004; Shea et al., Reference Shea, Guerra, Weiner, Aguirre, Ravenell and Asch2008; Dalal et al., Reference Dalal, Demuro-Mercon, Lewis, Nelson, Gilligan and McLeod2010; Skolarus et al., Reference Skolarus, Holmes-Rovner, Hawley, Dunn, Barr, Willard, Wei, Piette and An2012; Houser et al., Reference Houser, Ray, Maisiak, Panjamapirom, Willing, Schiff, English, Nevin and Berner2013). Mode effects due to the telephone delivery method and automated nature of IVR were experienced in some studies, as respondents were found more likely to choose the extreme categories (Rodriguez et al., Reference Rodriguez, von Glahn, Rogers, Chang, Fanjiang and Safran2006; Dillman et al., 2009). Some participants became frustrated by the survey pace causing them to ‘satisfice’, selecting the first available answer to move on to the next question, which biased results (Rodriguez et al., Reference Rodriguez, von Glahn, Rogers, Chang, Fanjiang and Safran2006). In contrast, recency effects may result when the participant remembers the last answer choice read out, due to it being heard most recently, and is more prone to pick it as a result (Dillman et al., 2009).
IVR was used effectively to reach diverse populations and hard to reach patient samples such as the visually impaired, illiterate, lower education, and non-English speaking patients. However, studies noted that those with hearing impairment may have difficulty participating in a phone survey and some populations may not trust reporting information to a machine (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Brodey et al., Reference Brodey, Rosen, Brodey, Sheetz and Unutzer2005; Goldman et al., Reference Goldman, Sanchez-Hernandez, Ross-Degnan, Piette, Trinacty and Simon2008; Naylor et al., Reference Naylor, Keefe, Brigidi, Naud and Helzer2008; Dalal et al., Reference Dalal, Demuro-Mercon, Lewis, Nelson, Gilligan and McLeod2010; Rose et al., Reference Rose, MacLean, Skelly, Badger, Ferraro and Helzer2010a; Reference Rose, Skelly, Badger, Maclean, Malgeri and Helzer2010b; Graham et al., Reference Graham, Tomcavage, Salek, Sciandra, Davis and Stewart2012; Skolarus et al., Reference Skolarus, Holmes-Rovner, Hawley, Dunn, Barr, Willard, Wei, Piette and An2012).
Sources suggested that to achieve good data quality installers need to carefully consider variables of speech recognition such as sensitivity of speech, risks of words being misrecognized, poor audio quality such as echoes, poor telephone service, and that phones encourage the use of shorter rating scales and simplified wording (Welker, Reference Welker2007; Dillman et al., 2009; Skolarus et al., Reference Skolarus, Holmes-Rovner, Hawley, Dunn, Barr, Willard, Wei, Piette and An2012; Willig et al., Reference Willig, Krawitz, Panjamapirom, Ray, Nevin, English, Cohen and Berner2013).
Costs
Multiple sources cited IVR technology as a feasible tool for practices and though there is a substantial initial expense, it is balanced against low incremental costs and long term savings (Janda et al., Reference Janda, Janda and Tedford2001; Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007; Piette et al., Reference Piette, Kerr, Richardson and Heisler2008; Oake et al., Reference Oake, Jennings, van Walraven and Forster2009; Rose et al., Reference Rose, MacLean, Skelly, Badger, Ferraro and Helzer2010a; Houser et al., Reference Houser, Ray, Maisiak, Panjamapirom, Willing, Schiff, English, Nevin and Berner2013; Willig et al., Reference Willig, Krawitz, Panjamapirom, Ray, Nevin, English, Cohen and Berner2013). Initial expenses included paying for: the system, programming, script creation, technical support staff, and hardware for data storage (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Ariza et al., Reference Ariza, Binns and Christoffel2004; Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007). Janda et al. (Reference Janda, Janda and Tedford2001) listed two programs that required programming knowledge to use, EASE (2800$) and Artisoft (1400$). Another package by VOS ($3300) did not require programming knowledge. Furthermore an outsourced company could be used for 40$ per respondent (Janda et al., Reference Janda, Janda and Tedford2001); Graham et al. (Reference Graham, Tomcavage, Salek, Sciandra, Davis and Stewart2012) reported that IVR calls were 25$ per patient. Shaw and Verma (Reference Shaw and Verma2007) found cost savings of 100$ per IVR call compared with a live interviewer. However, Rodriguez et al. (Reference Rodriguez, von Glahn, Rogers, Chang, Fanjiang and Safran2006) reported that a mailed survey was cheaper than IVR in their study and IVR costs increased when low response rates required crossover mailing. Lower response rates in some studies created a need to supplement IVR with other formats to improve response rates, which can incur additional expenses (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Rodriguez et al., Reference Rodriguez, von Glahn, Rogers, Chang, Fanjiang and Safran2006; Shaw and Verma, Reference Shaw and Verma2007; Dillman et al., Reference Dillman, Phelps, Tortora, Swift, Kohrell, Berck and Messer2009).
Some studies used the technology for service delivery or clinical care including for monitoring health outcomes, reminders, interventions and disease management (Reid et al., Reference Reid, Pipe, Quinlan and Oda2007; Goldman et al., Reference Goldman, Sanchez-Hernandez, Ross-Degnan, Piette, Trinacty and Simon2008; Naylor et al., Reference Naylor, Keefe, Brigidi, Naud and Helzer2008; Sikorskii et al., Reference Sikorskii, Given, Given, Jeon and You2009; Bender et al., Reference Bender, Apter, Bogen, Dickinson, Fisher, Wamboldt and Westfall2010; Kempe et al., Reference Kempe, Shetterly, France and Levin2012; Greaney et al., Reference Greaney, Puleo, Sprunck-Harrild, Bennett, Cunningham, Gillman, Coeling and Emmons2012; Houser et al., Reference Houser, Ray, Maisiak, Panjamapirom, Willing, Schiff, English, Nevin and Berner2013). The cost was part of health care delivery, often eliminating the need for a member of the health care team to carry out the task.
Response rate
Low response rates (around 30%) were observed when studies focused solely on survey administration (Isenberg et al., Reference Isenberg, Davis, Adams, Isenberg and Isenberg2001; Tourangeau et al., Reference Tourangeau, Couper and Steiger2003; Rodriguez et al., Reference Rodriguez, von Glahn, Rogers, Chang, Fanjiang and Safran2006; Shea et al., Reference Shea, Guerra, Weiner, Aguirre, Ravenell and Asch2008; Dillman et al., Reference Dillman, Phelps, Tortora, Swift, Kohrell, Berck and Messer2009). Aside from two sources, much higher response rates were usually observed and sustained over longer periods when IVR questions were paired with interventions (Table 2). For example, a 79% response rate was observed in an intervention to reduce non-injection drug use (Aharonovich et al., Reference Aharonovich, Greenstein, O’Leary, Johnston, Seol and Hasin2012) and there was a 74% response rate at 12 months in an IVR study that aided in a weight loss intervention (Estabrooks et al., Reference Estabrooks, Shoup, Gattshall, Dandamudi, Shetterly and Xu2009).
Many studies sought survey recruitment and patient consent using a different format such as mail, a live phone call, or were recruited during a visit to the practice and expected to call into the system (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Shea et al., Reference Shea, Guerra, Weiner, Aguirre, Ravenell and Asch2008; Aharonovich et al., Reference Aharonovich, Greenstein, O’Leary, Johnston, Seol and Hasin2012). Patient thoughts on such approaches throughout the process indicated annoyance with having to call into the IVR system and noted that they would rather receive calls (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Rodriguez et al., Reference Rodriguez, von Glahn, Rogers, Chang, Fanjiang and Safran2006; Aharonovich et al., Reference Aharonovich, Greenstein, O’Leary, Johnston, Seol and Hasin2012); it is possible to combine both call in and call out methods to further accessibility (Estabrooks et al., Reference Estabrooks, Shoup, Gattshall, Dandamudi, Shetterly and Xu2009).
Several studies reported strategies to improve response rates including: brief calls, call-in lines for additional time, pre-recorded voices, money incentives, calling cards, draw prizes, medical information, instructions, reminders, useful feedback, physician endorsements, indicators of practice improvement, and use of caller ID and statements to distinguish from telemarketing calls (Isenberg et al., Reference Isenberg, Davis, Adams, Isenberg and Isenberg2001; Lee et al., Reference Lee, Friedman, Cukor and Ahern2003; Brodey et al., Reference Brodey, Rosen, Brodey, Sheetz and Unutzer2005; Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007; Shaw and Verma, Reference Shaw and Verma2007; Welker, Reference Welker2007; Goldman et al., Reference Goldman, Sanchez-Hernandez, Ross-Degnan, Piette, Trinacty and Simon2008; Aharonovich et al., Reference Aharonovich, Greenstein, O’Leary, Johnston, Seol and Hasin2012; Skolarus, 2012; Willig et al., Reference Willig, Krawitz, Panjamapirom, Ray, Nevin, English, Cohen and Berner2013).
Ease of integration of IVR with clinical practice
Once installed, IVR caused little disruption to normal practice functioning in several circumstances (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Bender et al., 2010; Rose et al., Reference Rose, Skelly, Badger, Maclean, Malgeri and Helzer2010b; Willig et al., Reference Willig, Krawitz, Panjamapirom, Ray, Nevin, English, Cohen and Berner2013). Studies also noted that the integration of IVR into a database allowed for timely, accurate information, and decreased staff hours (Lee et al., Reference Lee, Friedman, Cukor and Ahern2003; Shaw and Verma, Reference Shaw and Verma2007; Goldman et al., Reference Goldman, Sanchez-Hernandez, Ross-Degnan, Piette, Trinacty and Simon2008; Naylor et al., Reference Naylor, Keefe, Brigidi, Naud and Helzer2008; Rose et al., Reference Rose, MacLean, Skelly, Badger, Ferraro and Helzer2010a; Graham et al., Reference Graham, Tomcavage, Salek, Sciandra, Davis and Stewart2012; Skolarus et al., Reference Skolarus, Holmes-Rovner, Hawley, Dunn, Barr, Willard, Wei, Piette and An2012; Aharonovich et al., 2012).
Discussion
The review of studies reporting on interactive voice response technology found some important considerations for practices and research teams considering IVR as a tool to better understand the patient experience in primary care. Several features of IVR survey capacity are particularly noteworthy as they might address limitations in our current approaches to collecting data on patient-reported outcome measures. The capacity to reach diverse populations through IVR technology addresses a significant limitation of the common practice-based waiting-room administration of surveys which reach only those who attend the practice, potentially creating biased samples (Green et al., Reference Green, Hogg, Savage, Johnston, Russell, Jaakkimainen, Glazier, Barnsley and Birtwhistle2012). IVR surveys offer the flexibility to programme for a random sample from a defined roster of patients, perhaps the entire list, or a sub-set of highest interest for the study, and use different languages. As the telephone is almost universal, IVR surveys could facilitate access to a more representative sample of the group being studied. This is especially important for those patients who are traditionally more difficult to reach with paper-based waiting room surveys or even online surveys such as the visually impaired, illiterate, lower education, and non-English speaking patients (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Brodey et al., Reference Brodey, Rosen, Brodey, Sheetz and Unutzer2005; Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007; Estabrooks et al., Reference Estabrooks, Shoup, Gattshall, Dandamudi, Shetterly and Xu2009; Oake et al., Reference Oake, Jennings, van Walraven and Forster2009; Dalal et al., Reference Dalal, Demuro-Mercon, Lewis, Nelson, Gilligan and McLeod2010; Lundy and Coons, Reference Lundy and Coons2012; Skolarus et al., Reference Skolarus, Holmes-Rovner, Hawley, Dunn, Barr, Willard, Wei, Piette and An2012).
A major barrier to practices participating in research or collecting data on patient reported outcomes of care is the perceived lack of time and concern of added burden on already stretched practices (Fitzpatrick, Reference Fitzpatrick2009; Johnston et al., Reference Johnston, Liddy, Hogg, Donskov, Russell and Gyorfi-Dyke2010). Once installed IVR could have a limited burden, if any at all, on a practice and if integrated with an electronic medical record (EMR) efficiently create outreach to patients without involving practice staff (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Shaw and Verma, Reference Shaw and Verma2007; Bender et al., 2010; Rose et al., Reference Rose, Skelly, Badger, Maclean, Malgeri and Helzer2010b; Aharonovich et al., 2012; Willig et al., Reference Willig, Krawitz, Panjamapirom, Ray, Nevin, English, Cohen and Berner2013). It could be programmed to generate and provide data at meaningful intervals to match study or practice needs.
The range in response rates and the relatively high upfront costs of investing in IVR technology require attention to maximize response rates. A significant feature of IVR for primary care is the ability to have the technology multi-task (Isenberg et al., Reference Isenberg, Davis, Adams, Isenberg and Isenberg2001; Janda et al., Reference Janda, Janda and Tedford2001; Tourangeau et al., Reference Tourangeau, Couper and Steiger2003; Rodriguez et al., Reference Rodriguez, von Glahn, Rogers, Chang, Fanjiang and Safran2006; Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007; Shea et al., Reference Shea, Guerra, Weiner, Aguirre, Ravenell and Asch2008; Dillman et al., Reference Dillman, Phelps, Tortora, Swift, Kohrell, Berck and Messer2009; Estabrooks et al., Reference Estabrooks, Shoup, Gattshall, Dandamudi, Shetterly and Xu2009; Aharonovich et al., Reference Aharonovich, Greenstein, O’Leary, Johnston, Seol and Hasin2012). The capacity to customise IVR and integrate it with EMR (Lee et al., Reference Lee, Friedman, Cukor and Ahern2003; Shaw and Verma, Reference Shaw and Verma2007; Goldman et al., Reference Goldman, Sanchez-Hernandez, Ross-Degnan, Piette, Trinacty and Simon2008; Naylor et al., Reference Naylor, Keefe, Brigidi, Naud and Helzer2008; Rose et al., Reference Rose, MacLean, Skelly, Badger, Ferraro and Helzer2010a; Graham et al., Reference Graham, Tomcavage, Salek, Sciandra, Davis and Stewart2012; Skolarus et al., Reference Skolarus, Holmes-Rovner, Hawley, Dunn, Barr, Willard, Wei, Piette and An2012; Aharonovich et al., 2012; Willig et al., Reference Willig, Krawitz, Panjamapirom, Ray, Nevin, English, Cohen and Berner2013) opens possibilities for combining patient services such as appointment reminders or information on clinical services with patient survey questions. Services such as appointment reminders are often welcome by patients and might help overcome reluctance to complete general survey questions (Dillman et al., Reference Dillman, Phelps, Tortora, Swift, Kohrell, Berck and Messer2009). Additionally, several practices could share one IVR system, which would help diminish the burden of cost on an individual practice (Rose et al., Reference Rose, MacLean, Skelly, Badger, Ferraro and Helzer2010a).
Several studies suggested patients prefer a simpler process of a direct call-out to them rather than a call-in option (Stuart et al., Reference Stuart, Laraia, Ornstein and Nietert2003; Rodriguez et al., Reference Rodriguez, von Glahn, Rogers, Chang, Fanjiang and Safran2006; Aharonovich et al., Reference Aharonovich, Greenstein, O’Leary, Johnston, Seol and Hasin2012). However, using a simple call-out survey will raise challenges in securing patient consent when IVR is used for research purposes and there have been variable approaches by institutional ethics review boards on whether prior consent is required (Nelson et al., Reference Nelson, Garcia, Brown, Mangione, Louis, Keeler and Cretin2002). The potential harm to patients from the responses they might give would need to be very low to make a short consent process possible via automated recorded message ethically acceptable. Securing patient informed consent beforehand, such as during a practice visit, or through a mail-out to potential participants requesting permission to contact them by phone could significantly decrease participation rates or limit the sample to those who attend a practice (Nelson et al., Reference Nelson, Garcia, Brown, Mangione, Louis, Keeler and Cretin2002; Angus et al., Reference Angus, Entwistle, Emslie, Walker and Andrew2003; Krousel-Wood et al., Reference Krousel-Wood, Muntner, Jannu, Hyre and Breault2006). However the use of IVR for practice-based quality improvement patient surveys might not require the same degree of information as patient informed consent for research purposes.
An automated IVR phone survey is a different mode from the ones used for many previously validated surveys (Weiler et al., Reference Weiler, Christ, Woodworth, Weiler and Weiler2004; Sikorskii et al., Reference Sikorskii, Given, Given, Jeon and You2009; Dalal et al., Reference Dalal, Demuro-Mercon, Lewis, Nelson, Gilligan and McLeod2010). The potential for mode effects such as satisficing and recency require attention such as changing the order of questions asked (Rodriguez et al., Reference Rodriguez, von Glahn, Rogers, Chang, Fanjiang and Safran2006; Dillman et al., Reference Dillman, Phelps, Tortora, Swift, Kohrell, Berck and Messer2009). Survey instruments used in other modes would have to be validated for IVR (Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007). Designing surveys which take advantage of the strengths of IVR would maximise the value of this approach. Shorter surveys spread over a larger population, customized by patient input on language, would be best suited for IVR (Weiler et al., Reference Weiler, Christ, Woodworth, Weiler and Weiler2004; Abu-Hasaballah et al., Reference Abu-Hasaballah, James and Aseltine2007; Goldman et al., Reference Goldman, Sanchez-Hernandez, Ross-Degnan, Piette, Trinacty and Simon2008; Naylor et al., Reference Naylor, Keefe, Brigidi, Naud and Helzer2008; Shea et al., Reference Shea, Guerra, Weiner, Aguirre, Ravenell and Asch2008; Dalal et al., Reference Dalal, Demuro-Mercon, Lewis, Nelson, Gilligan and McLeod2010; Rose et al., Reference Rose, Skelly, Badger, Maclean, Malgeri and Helzer2010b; Greaney et al., Reference Greaney, Puleo, Sprunck-Harrild, Bennett, Cunningham, Gillman, Coeling and Emmons2012; Willig et al., Reference Willig, Krawitz, Panjamapirom, Ray, Nevin, English, Cohen and Berner2013). Survey questions benefitting from specific follow up timing would also be suited for IVR (Houser et al., Reference Houser, Ray, Maisiak, Panjamapirom, Willing, Schiff, English, Nevin and Berner2013; Willig et al., Reference Willig, Krawitz, Panjamapirom, Ray, Nevin, English, Cohen and Berner2013).
The various studies highlight diverse strategies to improve response rates, which could be a potential weakness of IVR used for research or quality improvement purposes. Using the strengths of flexibility to optimise timing for patients and the capacity to integrate the IVR survey with patient services are attractive ways to enhance the functionality of IVR and possibly increase response rates (Reid et al., Reference Reid, Pipe, Quinlan and Oda2007; Goldman et al., Reference Goldman, Sanchez-Hernandez, Ross-Degnan, Piette, Trinacty and Simon2008; Naylor et al., Reference Naylor, Keefe, Brigidi, Naud and Helzer2008; Sikorskii et al., Reference Sikorskii, Given, Given, Jeon and You2009; Bender et al., Reference Bender, Apter, Bogen, Dickinson, Fisher, Wamboldt and Westfall2010; Kempe et al., Reference Kempe, Shetterly, France and Levin2012; Greaney et al., Reference Greaney, Puleo, Sprunck-Harrild, Bennett, Cunningham, Gillman, Coeling and Emmons2012; Houser et al., Reference Houser, Ray, Maisiak, Panjamapirom, Willing, Schiff, English, Nevin and Berner2013). Further exploration of response rates and reach capacity with IVR surveys linked to patient services would significantly advance the understanding of this technology to improve primary health care data collection capacity.
Conclusion
As primary care reform continues, the need for understanding the patient experience and improving data collection capacity on patient-reported outcome measures, especially from traditionally difficult to reach populations, is essential. This review of interactive voice response technology, for potential use in automated performance measurement of the patient experience in primary care practices, suggests it may be a feasible alternative to traditional patient data collection methods, which should be further explored. The lessons from the use of IVR in many other health care sectors can guide implementation efforts within the primary care sector to maximise the potential of this approach.
Acknowledgments
Thank you to Dr. Matthew Hogel and Megan James as well as the Elizabeth Bruyère research team for additional support and guidance throughout the creation of this manuscript.
Authors’ Contributions
This scoping review was conducted by Michael Falconi under the guidance of Dr. William Hogg and Dr. Sharon Johnston. Michael Falconi wrote the first draft and Dr. William Hogg and Dr. Sharon Johnston contributed to the writing and critically reviewed the manuscript. All authors approve the final manuscript.
Financial Support
This work was supported by the TRANSFORM CBPHC Team Grant (S.W., grant number 11R26796).
Conflicts of Interest
None.
Ethical Standards
No ethical clearance was required for this scoping review.
Appendix
Appendix A: Scoping searches
