Goals and Rationales for Public and Patient Involvement

Our review of HTA Web sites identified only a small number (n = 5) of organizations that articulated a clear set of goals or rationales for involving the public or patients in their activities (19–23). They highlight 3 main goals for PPI: (i) improving the relevance of assessments (19;21;22); (ii) strengthening the research and complementing the expertise of healthcare professionals and researchers (21–23); or (iii) enhancing procedures, that is, the openness and inclusiveness of the decision process (20).

The published literature offers a more conceptually rich discussion of the goals for PPI. Earlier work, which first applied theories of public participation to HTA, is especially illuminating. From a democratic perspective, PPI may be a means for promoting accountable health systems that respond to public and patients’ values and expectations, and taxpayers’ financial stakes in publicly financed health systems (Reference Wynne24–Reference Bovens, Ferlie, Lynn and Pollitt26). Contexts of scarce resources and rapid technological change present policymakers with complex and contentious funding decisions (Reference Chinitz27;Reference Coulter and Ellins28). These contribute to the emergence of PPI mechanisms as a political imperative for more informed, transparent, accountable, and legitimate decisions about health technologies (Reference Abelson, Giacomini, Lehoux and Gauvin3;Reference Gauvin, Abelson, Giacomini, Eyles and Lavis18).

The value of PPI has also gained traction as a way of promoting a more comprehensive approach to HTA rather than relying solely on clinical and cost-effectiveness evidence. The political and ethical backdrops of the health technology policy landscape have precipitated a more comprehensive approach to determine which technologies a publicly funded health plan can justify morally, economically, and functionally. In recent years, researchers and HTA agencies have given more serious consideration to incorporating social values and ethics into HTAs (Reference Facey, Boivin and Gracia29;Reference DeJean, Giacomini, Schwartz and Miller30), to greater patient-focused HTAs that incorporate the patients’ values, needs, preferences, and lived experiences (Reference Bridges and Jones4;Reference Facey, Boivin and Gracia29;Reference Lehoux31), and to involving a broader range of stakeholders including patients and service users as well as the broader public in conducting HTAs (Reference Abelson, Giacomini, Lehoux and Gauvin3;Reference Bridges and Jones4;Reference DeJean, Giacomini, Schwartz and Miller30).

PPI in HTA may also achieve instrumental goals, to make better quality decisions across all the stages of an HTA (Reference Abelson, Giacomini, Lehoux and Gauvin3). For example, HTA agencies may seek the most meaningful ways to gather public and patient input to improve the prioritizing of requests, the scoping of the assessment topic, the development of recommendations, or the dissemination of findings (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis18;Reference Facey, Boivin and Gracia29). Lastly, PPI may pursue developmental goals, such as increasing the public's understanding of health technologies and HTA, as well as strengthening the public's and patients’ competence and capacity to contribute to health technology-policy issues (Reference Abelson, Giacomini, Lehoux and Gauvin3;Reference Gauvin, Abelson, Giacomini, Eyles and Lavis18).

More recent published discussion echoes and builds on these goals. Generally, making the work of HTA agencies relevant to the public is cited as necessary to gain public support for its funding (Reference Menon and Stafinski6;Reference Drummond, Tarricone and Torbica32). Menon and Stafinski (Reference Menon and Stafinski6) advocate patient involvement in every step of the HTA process, to ensure that the assessment adopts a broader health condition perspective, rather than the narrower technology perspective characteristic of more traditional HTAs. Gagnon et al. (Reference Gagnon, Desmartis and Lepage-Savary5;Reference Gagnon, Desmartis and Gagnon33) cite three key rationales for PPI. First, PPI provides context for the research, which improves the usefulness of assessments for decision makers. In turn, this improves the appropriateness and applicability of the recommendations that arise from HTAs. Second, PPI might contribute to better acceptability, adoption, and implementation of recommendations. Finally, PPI focuses on the primary beneficiaries of the decisions.

Public and Patient Involvement Approaches

Our Web site scan identified thirteen of the fifty-three HTA agencies (25 percent) with documented approaches to PPI (20;22;23;34–44), roughly consistent with proportions documented by previous surveys (Reference Hailey, Werko and Bakri2;Reference Whitty7). It is worth noting that decisions to undertake PPI activities within agencies that support other agencies responsible for HTA decision making may be constrained by the scope and funding received for their work; however, we were unable to assess this objectively. The most commonly used PPI approach (n = 8 agencies) is to have patients and/or members of the public directly participate on committees of the agency (19;23;34;37;41;42;44); very few responding agencies indicated that they had engaged the public through more participatory approaches (Reference Whitty7;Reference Hailey and Nordwall17).

Conversely, the results of a self-reported survey of HTA agencies identified a much stronger emphasis on communication and consultation mechanisms (Reference Whitty7). The most commonly used mechanisms reported in the survey include communicating through public meetings, or consulting through the use of documents or focus groups (Reference Whitty7). Many of the agencies facilitated communication by preparing plain-language versions of their reports to increase accessibility of the assessment (Reference Hailey and Nordwall17). Patient organizations also increasingly participate in disseminating HTA findings (Reference Hailey and Nordwall17).

Those HTA agencies that use PPI approaches do so across all stages of the HTA process. Most (n = 7; 54 percent) formally solicit “public comment” toward the end of the HTA process (11;19;23;41;42;45–47). Public reviews of draft documents, surveys, or face-to-face discussions occur most typically at the appraisal and draft recommendation stage (Reference Whitty7). At the earliest topic selection stage, six agencies (46 percent) invite the general public to submit assessment requests (22;34;36;37;41;42). The HTA literature notes a trend among HTA agencies toward increased involvement of patients and the public in the early stages of HTA (e.g., topic selection, prioritization, and scoping stages), as well as the value of PPI in the evidence analysis process to capture patient values and experiences (Reference Hailey, Werko and Bakri2;Reference Abelson, Bombard, Gauvin, Simeonov and Boesveld14;Reference Oliver, Armes and Gyte48). However, only three agencies (23 percent) currently claim to involve the public or patients in articulating the research questions to guide the HTA process, or in the evidence collection and analysis process (23;34;44).

Evidence of PPI Effectiveness

We found minimal evaluation of the effectiveness of PPI or the impacts of involvement on the HTA process or decision making; both Web sites and literature evince this gap (Reference Menon and Stafinski6;Reference Abelson, Bombard, Gauvin, Simeonov and Boesveld14;Reference Jones and Einsiedel49;Reference Chambers, Wilson and Thompson50). The few efforts document potential instrumental benefits (e.g., including patient preferences and patient-relevant outcomes in HTAs) and developmental benefits (e.g., raising public awareness and understanding) (Reference Gagnon, Desmartis and Lepage-Savary5;Reference Moran and Davidson51). A small number of ethnographic evaluations document the impacts, and facilitators or barriers, of a particular type of public involvement mechanism, for example, face-to-face citizen deliberations (Reference Abelson, Bombard, Gauvin, Simeonov and Boesveld14;Reference Jones and Einsiedel49;Reference Davies, Wetherell and Barnett52).

PPI Feasibility Considerations

HTA agency Web sites do not address feasibility considerations related to PPI, but the published literature emphasizes organizational and recruitment challenges (Reference Menon and Stafinski6;Reference Drummond, Tarricone and Torbica32;Reference Gauvin, Abelson, Giacomini, Eyles and Lavis53). Cultural challenges include tensions between the traditional focus within HTA organizations on clinical and economic evidence, and pressures to incorporate patient and social values input. Public involvement initiatives must combat beliefs that patient views are an anecdotal and biased source of evidence, and perceptions that patients and publics are unable to contribute meaningfully to the process (Reference Menon and Stafinski6;Reference Drummond, Tarricone and Torbica32;Reference Gauvin, Abelson, Giacomini, Eyles and Lavis53). Some papers also noted perceptions that engaging patients and the public would politicize what should be a solely evidence-informed process (Reference Menon and Stafinski6).

Key organizational challenges include the time, financial resources, and expertise required to support high-quality PPI. Ensuring capacity requires organizational commitment. Recruitment challenges involve struggles to obtain “representative” input, and concerns that PPI will allow narrow interests to trump fairness considerations (Reference Oliver, Armes and Gyte48). Enablers to meaningful PPI include a supportive organizational culture, appropriate supports for patient/public committee members and those interacting with them, dedicated time devoted to “patient perspectives” on meeting agendas, and strategic use of new and existing networks of patient organizations for external consultations (Reference Menon and Stafinski6;Reference Facey, Boivin and Gracia29).

Framework for Public and Patient Involvement in Ontario's HTA Process

Informed by the committee deliberations and evidentiary sources discussed above, our PPI framework is organized around four actionable elements detailed in the following sections.

Element 1: Articulate the Principles and Goals for Public and Patient Involvement in HTA

The committee agreed to six principles that should guide HQO's approach to public and patient involvement: (i) purposeful; (ii) pragmatic; (iii) fair and equitable; (iv) proportional; (v) evidence-informed; and (vi) transparent (Supplementary Table 1). Many of these principles are common to the values identified in HTAi's Values and Quality Standards for Patient Involvement in HTA, which resulted from an 18-month Delphi consensus process with input from 150 respondents in thirty-nine countries worldwide (54).

Table 1. Building a common language for public and patient involvement in HTA

HQO, Health Quality Ontario; OHTAC, Ontario Health Technology Advisory Committee.

Linked to these principles is a set of operational goals specific to the HQO-OHTAC process, and which provide the foundation for the PPI framework:

1. 1. PPI will be strengthened and supported: (i) to improve the quality of the outputs that arise from each stage of the HQO-OHTAC process; (ii) to create more informed, transparent, accountable and legitimate processes for deliberating about health technologies; (iii) to promote a more robust and comprehensive science of HTA that investigates social values and ethics, as well as patients’ problems, lived experiences, outcomes, and preferences; (iv) to increase public and patient understanding of health technologies and HTA, and to strengthen the public's and patients’ competence and capacity to contribute to various stages of the health technology assessment process.

2. 2. PPI efforts will be informed by evidence (where possible), best practice in the absence of evidence, and sound principles.

3. 3. Evaluation is to be embedded into all aspects of HQO's PPI activities given the lack of a strong evidence base in this area and given HQO-OHTAC's emphasis in this area.

4. 4. PPI efforts will support and, in turn, be supported by HQO's decision-making framework, which includes a specific emphasis on the incorporation of social values and ethics into the evidence-review process.

Element 2: Establish a Common Language to Support Public and Patient Involvement Efforts

Conceptual and empirical studies have noted divergent views within the HTA community, about what “public and patient involvement” means (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis18). This ambiguity can lead to conflicting goals and visions for PPI in HTA (Reference Hailey, Werko and Bakri2;Reference Whitty7;Reference Hailey and Nordwall17;Reference Gauvin, Abelson, Giacomini, Eyles and Lavis18), and more practically, conflicting views about who should be engaged, who they represent, what role they should play at different stages of the HTA process, and using what types of involvement mechanisms (Reference Menon and Stafinski6;Reference Whitty7;Reference Gauvin, Abelson, Giacomini, Eyles and Lavis18). Our framework emphasizes the importance of establishing a common language to effectively support efforts to involve the public and patients in the HTA process. Informed by the literature synthesis, we defined the “who” of PPI with three categories of relevant perspectives: (i) public (or publics or citizens); (ii) patients, family members, and caregivers; and (iii) stakeholders (Supplementary Figure 1).

The “what” of PPI was established by adopting Rowe and Frewer's (Reference Rowe and Frewer55) typology of involvement, which includes three commonly cited levels of involvement: communication, consultation, and participation (Supplementary Figure 2). Each type reflects a different set of relationships between the initiator of the involvement activity and the participants. Taken together, these mechanisms are often portrayed as a spectrum of involvement with increasing opportunities for participants to be empowered through each stage of the PPI process. These conceptualizations are linked to illustrate the early but critical step in any PPI strategy where key decisions are made about which publics to involve and how, guided by explicit goals (Table 1).

Figure 2. Rationales and Goals for Patient and Public Involvement by HTA Stage. *Newly proposed stage to provide early opportunity for PPI. †This stage was added following the tabling of the subcommittee's report in July 2014. ‡Not addressed by the subcommittee. HQO, Health Quality Ontario; HTA, Health Technology Assessment; OHTAC, Ontario Health Technology Advisory Committee.

Element 3: Describe a Flexible Menu of Public and Patient Involvement Approaches

A key message from the Web site scan and literature synthesis is that there is no one-size-fits-all approach to PPI. Rather, the choice of method should be considered in the context of each HTA stage and matched to the motivation for incorporating societal and/or patient perspectives into the process, and the relevant societal and/or patient values at stake. We have undertaken this mapping process in two stages. In the first stage (Figure 2), we have listed the potential goals for PPI that might be pursued in each HTA stage. In the second stage (Figure 3), we have provided a menu of PPI mechanisms and tools that could be used in each of these stages depending on the goal and the societal or patient perspectives sought.

Figure 3. Who to Involve and How by HTA Stage. *Indicates link to Subcommittee report recommendation. HQO, Health Quality Ontario; HTA, Health Technology Assessment; OHTAC, Ontario Health Technology Advisory Committee.

The mapping process focuses on enhancing the transparency, legitimacy, and overall quality of at the front end of the HQO-OHTAC process, especially in the topic selection and scoping stages. This emphasis reflects the Subcommittee's priorities to facilitate increased awareness and understanding among interested patients, broader publics, and stakeholder groups, and about how topics are selected for review and appraisal, a stage of the HTA process that is opaque in most HTA organizations.

Seen as equally important was the need to increase the relevance and responsiveness of OHTAC's work by undertaking consultation and participation activities to prospectively identify potential review topics and emerging issues of concern to patients and priority populations, and to ensure that the framing of the questions guiding the evidence review process reflect the values of relevant health technology users. These areas of emphasis were reinforced by our synthesis of the evidence, which revealed increased attention to, and sophistication of, the methods used to involve patients and citizens in these early HTA stages (e.g., patient members of expert panels, on-line and face-to-face consultations with individual patients, and/or patient groups, and the synthesis of primary qualitative research studies about patients’ views and experiences).

The committee also sought to strengthen and diversify the communication and consultation activities already present in the later stages of the HQO-OHTAC process, again, to enhance the awareness of OHTAC's work among relevant populations, and to encourage broader input on its draft recommendations.

Element 4. Measurement and Evaluation

Given the weak evidence base for PPI in the HTA field, efforts in this regard should initially focus on basic formative evaluation metrics to determine whether the intended goals of the PPI activities are being achieved, including basic process and impact measures such as numbers and types of consultations, how different types of PPI input are being used in the various stages of the HTA process, and the resources required to carry out relevant activities. These will provide valuable information to inform mid-course adjustments to approaches. Over time, more robust evaluation metrics could be developed, allowing the possibility to compare different PPI approaches using trial and quasi-experimental designs consistent with several recently published high-quality public involvement evaluation studies in related fields (Reference Carman, Heeringa and Heil56;Reference Boivin, Lehoux, Lacombe, Burgers and Grol57).