Skip to main content Accessibility help
×
Home

Information:

  • Access
  • Cited by 1

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Exploratory factor analysis and reliability of the Primary Health Care Engagement (PHCE) Scale in rural and remote nurses: findings from a national survey
        Available formats
        ×

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        Exploratory factor analysis and reliability of the Primary Health Care Engagement (PHCE) Scale in rural and remote nurses: findings from a national survey
        Available formats
        ×

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        Exploratory factor analysis and reliability of the Primary Health Care Engagement (PHCE) Scale in rural and remote nurses: findings from a national survey
        Available formats
        ×
Export citation

Abstract

Aim

The study purpose was to provide evidence of validity for the Primary Health Care Engagement (PHCE) Scale, based on exploratory factor analysis and reliability findings from a large national survey of regulated nurses residing and working in rural and remote Canadian communities.

Background

There are currently no published provider-level instruments to adequately assess delivery of community-based primary health care, relevant to ongoing primary health care (PHC) reform strategies across Canada and elsewhere. The PHCE Scale reflects a contemporary approach that emphasizes community-oriented and community-based elements of PHC delivery.

Methods

Data from the pan-Canadian Nursing Practice in Rural and Remote Canada II (RRNII) survey were used to conduct an exploratory factor analysis and evaluate the internal consistency reliability of the final PHCE Scale.

Findings

The RRNII survey sample included 1587 registered nurses, nurse practitioners, licensed practical nurses, and registered psychiatric nurses residing and working in rural and remote Canada. Exploratory factor analysis identified an eight-factor structure across 28 items overall, and good internal consistency reliability was indicated by an α estimate of 0.89 for the final scale. The final 28-item PHCE Scale includes three of four elements in a contemporary approach to PHC (accessibility/availability, community participation, and intersectoral team) and most community-oriented/based elements of PHC (interdisciplinary collaboration, person-centred, continuity, population orientation, and quality improvement). We recommend additional psychometric testing in a range of health care providers and settings, as the PHCE Scale shows promise as a tool for health care planners and researchers to test interventions and track progress in primary health care reform.

Introduction

There is no single common definition of primary health care (PHC) in the contemporary literature. Further, the term is often not explicitly defined and is sometimes used synonymously with primary care (Henderson et al., 2014; Canadian Nurses Association, 2015; White, 2015). However, PHC is commonly agreed to broadly address the social determinants of health and offer essential education and health care (eg, health promotion; disease prevention, treatment and management) to communities based on principles of accessibility, equity, community participation, and intersectoral collaboration (Henderson et al., 2014; White, 2015; Baum et al., 2016).

As a component of PHC, primary care focuses on clinical health care services and medical care provided after first contact between individuals and health care providers. Part of the confusion in terminology stems from advocacy efforts to advance the PHC philosophy, resulting in bifurcation of the term ‘primary health care’. PHC that is ‘professional’ (Levesque et al., 2011), ‘profession-centred’ (White, 2015), or ‘selective’ has come to represent a primary care approach (Baum et al., 2016), while PHC that is ‘community-based’, ‘societal’, and ‘comprehensive’ signals a contemporary PHC approach founded on the original vision set out in the Alma Ata Declaration on Primary Health Care (World Health Organization, 1978).

In developing the Primary Health Care Engagement (PHCE) Scale for health care providers, we sought to closely align the instrument with a contemporary PHC approach that would be relevant to ongoing PHC reform strategies across Canada (eg, Alberta Health, 2014; Saskatchewan Ministry of Health, 2016) and elsewhere (Liu et al., 2011; Thomas et al., 2014; Galdikiene et al., 2016). Although other instruments are available to measure primary care delivery performance based on perceptions of health care providers (eg, Schoen et al., 2006; Dahrouge et al., 2009), community-based PHC performance has not been well addressed at the provider level. Aside from the PHCE Scale, the Canadian Institute for Health Information (CIHI) PHC Provider Survey (Johnston and Burge, 2013) is the only other provider-level instrument that specifically targets community-based PHC performance, to our knowledge. Developed to complement the CIHI patient experience and organizational surveys, the CIHI provider survey addresses the PHC principle of intersectoral collaboration but not the principles of accessibility, equity, and community participation. Although these latter attributes may be more appropriately measured at the patient level, it is also worthwhile to gauge the degree of workplace involvement in PHC delivery according to front-line health care providers, given their central role in the actualization of PHC principles.

We were also keen that the new PHCE instrument apply to rural and remote (hereafter rural/remote) communities where effective PHC delivery is highly valued in light of persistent difficulties meeting health and social needs (Thomas et al., 2014; Ward et al., 2014). Rural/remote health care providers may work alone or in small multidisciplinary teams with members who are not co-located on a regular basis (MacLeod et al., 2004), and thus it is important to make the best use of scarce health human resources in these settings (Ford, 2016). Community participation in health service planning is also a policy expectation in rural communities, although meaningful participation can be difficult to operationalize and achieve (Kenny et al., 2015). For these reasons, we drew on community-oriented (Levesque et al., 2011) and community-based (CIHR, 2015) conceptualizations of PHC in the development of the PHCE Scale. Community-oriented PHC is understood as including ‘… a wide range of professionals who deliver a broad spectrum of health and social services’ (Levesque et al., 2011: 23). Further, community-based PHC is person-centred, population-centred, coordinated, and integrated care delivered by multiple health care providers ‘in a range of community settings’ that covers the continuum from primary prevention to palliative care (CIHR, 2015).

The 40-item PHCE Scale was developed in a three-phase process reported previously (Kosteniuk et al., 2016); 10 dimensions of community-oriented PHC most relevant to community-based and rural/remote PHC were endorsed by our 16-member research team and 19-member advisory board, collectively representing the 13 Canadian provinces and territories. The 40-item PHCE Scale consisted of 10 subscales: accessibility/availability, equity, community participation, intersectoral team, patient-centred care, continuity, population orientation, interdisciplinary collaboration, comprehensiveness, and quality improvement. Based on initial psychometric testing in a pilot survey of 89 nurses in current practice in Canada with nursing experience in rural/remote communities, the 40-item PHCE Scale (3–5 items per subscale) demonstrated good reliability (α=0.91). Although Cronbach’s α coefficients below 0.70 indicated evidence of low reliability for three subscales (equity, comprehensiveness, and patient-centred care), these were nonetheless retained in the final 40-item version given their theoretical importance to community-oriented PHC delivery in rural/remote communities.

The purpose of the present study was to provide validity evidence for the PHCE Scale, by evaluating the scale’s internal structure based on factor analysis and reliability findings from a large national survey of rural/remote regulated nurses in Canada. Recent years have seen major health care reforms targeting PHC in Canada (Strumpf et al., 2012) and other countries (Liu et al., 2011; Thomas et al., 2014; Galdikiene et al., 2016). We anticipate the PHCE Scale will be useful to researchers and health care planners in assessing the level of PHC delivery in health care settings, identifying the degree to which PHC elements are being enacted and the elements that may require intervention, and tracking progress in PHC reform.

Methods

Design

This study uses data from the pan-Canadian study ‘Nursing Practice in Rural and Remote Canada II (RRNII)’, which included a cross-sectional survey with a target sample of 10 072 regulated nurses [registered nurses (RNs), nurse practitioners (NPs), licensed practical nurses, and registered psychiatric nurses] residing in rural/remote Canada. Further details of the RRNII survey are reported elsewhere (MacLeod et al., 2017).

Participants

The RRNII survey population included a stratified systematic sample of regulated nurses in every Canadian province, as well as all regulated nurses working in the Territories and all rural and remote NPs. Eligible nurses practiced in a rural or remote community at the time of the survey, or had been on leave for six months or less. For the purposes of eligibility, communities with a core population of less than 10 000 were considered rural (du Plessis et al., 2001), and northern communities in the Territories (Nunavut, Northwest, and Yukon) were considered remote.

Regardless of their primary place of employment, we chose to include all survey participants in the exploratory factor analysis (EFA) for two reasons. First, community-based PHC is delivered ‘in a range of community settings’ outside of clinics and public health settings (CIHR, 2015). Second, institutional settings such as hospitals have a long history of health and wellness promotion in both urban and rural communities (Olden and Hoffman, 2011). Therefore, participants were included whose primary place of employment was a community-based health care site [ie, directly accessible by patients and outside of an inpatient setting (Gibson et al., 2015)], hospital/rehabilitation or convalescent centre/integrated facility, nursing home/long-term care facility, or other setting (Table 1).

Table 1 Characteristics of Nursing Practice in Rural and Remote Canada II survey respondents included and excluded from analysis

a Calculated by the χ 2 test unless otherwise noted.

b Mann–Whitney U test.

c Defined as primary health care settings that are directly accessible by patients and outside of an inpatient setting (Gibson et al., 2015), includes mental health centre/crisis centre, community health centre, home care agency, public health department/unit, private nursing/self-employed, occupational health, physician’s office/family practice unit or team, nurse practitioner led clinic, and multidisciplinary primary health care clinic.

d Includes hospital, rehabilitation/convalescent centre, and integrated facility (ie, acute and long-term care).

e Includes educational institution, professional association/government, and other.

A total of 3822 participants completed the RRNII survey. Of 10 072 sampled, 450 were ineligible (eg, incorrect addresses, duplicate registrations, retired) and 9622 were eligible, resulting in a response rate of 40% (3822/9622). Of these participants, 1587 completed all 40 items in the PHCE Scale and were included in the factor analysis; 2235 participants completed fewer than 40 items and were excluded to avoid artificially high correlations that may result from imputing missing values. Compared with excluded participants, those included were more highly educated and held higher registration status, and were more likely to work in smaller communities under 10 000 population (Table 1).

Data collection

RRNII survey data were collected from April 2014 to August 2015, with surveys distributed by the RRNII’s research centre at the University of Northern British Columbia as well as by provincial and territorial nursing associations. Participant recruitment relied on the Dillman method (Dillman et al., 2014) to improve the response rate (eg, participants were offered a chance to win an iPad as an incentive). Online and paper survey packages were provided in English and French; non-respondents received a total of four contacts that included a first survey package, first reminder, second reminder, and final survey package.

Measures

Development of the original 40-item PHCE Scale consisted of a literature review and expert consultation, content evaluation and item revision, and testing in a small pilot survey of 89 nurses (Kosteniuk et al., 2016). Items developed for the original PHCE Scale drew from multiple indicators of quality and performance [World Health Organization, 1986; Flocke, 1997; Shi et al., 2001; Canadian Institute for Health Information (CIHI), 2006; Davis et al., 2007; Bloch et al., 2011; Levesque et al., 2011; Wong et al., 2011; Saskatchewan Ministry of Health, 2012], including nine items that were minimally adapted with permission from a study by Dahrouge et al. (2009).

The 27-page RRNII survey employed in the present study consisted of five sections that focused on participants’ individual characteristics, work community, workplace, nursing practice, and aspects of their psychosocial health. The 40-item PHCE Scale included three to five items in each of 10 subscales, with items rated on a five-point scale as 1 (strongly disagree), 2 (disagree), 3 (neutral), 4 (agree), 5 (disagree), or 97 (not applicable). ‘Not applicable’ responses were coded as ‘missing’. Participants were instructed to respond in relation to the primary workplace where they spent most of their work time in the past 12 months, and to the community represented by the catchment area of their primary workplace. After reverse scoring negatively worded items, higher scores for each subscale and the overall scale suggest perceptions of greater workplace engagement in primary health care delivery.

Across the 40 PHCE items, the proportion of ‘not applicable’ responses (1.9–49.2%) was substantially higher than the proportion of true missing responses (3.1–7.7%). Specifically, between 20.1 and 49.2% of participants chose the ‘not applicable’ option in each of eight items across three subscales (accessibility/availability, population orientation, and intersectoral team). Participants whose primary place of employment was a community-based health care setting were more likely to provide a valid response to all of the scale items and also significantly more likely than their counterparts in other settings (ie, hospital, nursing home/long-term care facility, and other) to be included in the present analysis (57.1 versus 35.2%; P<0.001). However, those outside of community-based settings accounted for the majority of included participants (59.9%), demonstrating the wide applicability of the PHCE Scale across health care settings.

Statistical analyses

Initial principal component analysis and descriptive statistics were performed with IBM SPSS Statistics 23 and EFA was conducted with SAS 9.3.

The sample of 1587 participants in the present study exceeded the recommended 5:1 participant-to-item ratio and 200 participant minimum (Howard, 2016). The large sample size took into account the small number of items per factor (<5) in the 40-item PHCE scale (Gaskin and Happell, 2014) as well as potentially weak item communalities (<0.40) (Costello and Osborne, 2005), which would indicate that the factor model accounted for a small amount of variance in those items (Tabachnick and Fidell, 2013). Missing values were not imputed, as imputation with estimated values may result in overfit data and artificially high correlations (Tabachnick and Fidell, 2013). Instead, listwise deletion was employed to delete every case with one or more missing items.

The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity were first calculated with the 1587 participants to assess whether factor analysis was warranted (Howard, 2016). A KMO value of 0.80 or above, on a range of 0–1, indicates that the items share a common factor and factor analysis is supported (Kellar and Kelvin, 2013).

An initial principal component analysis to calculate eigenvalues was conducted, and the number of factors to extract was identified with the assistance of web-based parallel analysis (Patil, 2008). Parallel analysis compares eigenvalues calculated from the actual data to eigenvalues generated on the basis of random data matrices matched to the actual data on sample size and number of factor analysis items (Patil et al., 2008). The number of factors to extract is equal to the number of eigenvalues in the actual data that are greater than the corresponding eigenvalues in the random data.

A first EFA was then conducted using unweighted least squares with polychoric correlations, a method recommended when extracting factors from ordinal data (Basto and Pereira, 2012; Gaskin and Happell, 2014). We selected promax oblique rotation given that oblique rotation methods allow the derived factors to be correlated (Costello and Osborne, 2005), consistent with our assumption of correlations between factors in the present study.

We compared several factor solutions to identify the ‘cleanest factor structure’ in terms of data and theoretical fit (Costello and Osborne, 2005: 3). Items were initially assessed for retention on the basis of the 0.40–0.30–0.20 rule: primary factor loadings ⩾0.40, secondary factor loadings <0.30, and ⩾0.20 difference between primary and secondary loadings (Howard, 2016). Any remaining items with low communalities between 0.20 and 0.40 (Gaskin and Happell, 2014) indicated that the factor solution accounted for a low proportion of variance in these items (Tabachnick and Fidell, 2013). These items were evaluated for their theoretical contribution to the overall model, and considered for removal (Costello and Osborne, 2005). Furthermore, factors that consisted of only two items were considered for removal on the basis of their correlation with each other as well as with other items in the factor model (Tabachnick and Fidell, 2013).

A second EFA was conducted with the items remaining after the first EFA. We report the mean item scores and standardized Cronbach’s α coefficients for the final PHCE subscales and overall scale, using a cut-off point of 0.70 as evidence of adequate internal consistency reliability (Nunnally and Bernstein, 1994).

Results

Sample characteristics

As shown in Table 1, the majority of included participants were female, RNs, staff nurses, had attained a diploma as their highest level of education, were primarily employed outside of community-based health care settings, and were employed in communities with less than 10 000 population. Included participants were 47.0 years of age on average (SD=11.6) and had been registered to practice in Canada an average of 21.0 years (SD=13.0).

Exploratory factor analysis

Based on principal component analysis with data from 1587 participants, the KMO measure (0.91) and significant Bartlett’s test of sphericity [χ 2 (780)=23 859.80, P=0.000] pointed to a strong association among the scale items, which supported the use of factor analysis. Parallel analysis suggested an eight-factor solution, with eigenvalues generated by principal component analysis exceeding random values (in brackets) for the first eight factors: 9.68 (1.34), 2.65 (1.31), 2.27 (1.27), 1.60 (1.25), 1.49 (1.23), 1.47 (1.21), 1.38 (1.19), 1.21 (1.17), 1.10 (1.16).

In the first EFA (n=1587), we produced solutions of 5, 6, 7, 8, 9, and 10 factors for comparison. An eight-factor solution was optimal in terms of data fit as well as conceptual fit with the majority of the original subscales in the 40-item PHCE scale (Table 2). From the original 40 items, a total of 12 items were removed based on the 0.40–0.30–0.20 rule (Howard, 2016). First, 11 items with primary factor loadings <0.40 were removed, as well as one item (C3) with a low factor loading of 0.41 and a low communality of 0.19 (Table 3). Second, the secondary factor loadings of all 40 items were <0.30 and therefore no items were removed on the basis of this criterion. Third, of the 12 items removed, 10 also demonstrated a difference of <0.20 between their primary and secondary factor loadings.

Table 2 Factor loadings of items included in the eight-factor solution based on first exploratory factor analysis

Bold font indicates primary factor loadings.

Table 3 Factor loadings of items removed from the 8-factor solution based on first exploratory factor analysis

Bold font indicates primary factor loadings.

With respect to the remaining 28 items with factor loadings ranging from 0.42 to 0.89, the majority demonstrated communalities in the recommended range (0.42–0.79). However, five items had communalities below 0.40, namely G1 and G4 (factor 3), F1 and F4 (factor 4), and C4 (factor 8). We made the decision to retain these five items given their theoretical contributions to the final scale.

The eight-factor solution (Table 2) included five factors (1, 2, 3, 5, and 7) that retained all of the items from five original dimensions (community participation, patient-centred care, intersectoral team, accessibility/availability, and interdisciplinary collaboration). Factor 4 included items that loaded from two separate original dimensions (population orientation and equity). Given that the population orientation items (D3 and D5) had higher loadings than the equity items (F1 and F4), and the equity items measured barriers between the community population and the participant’s workplace, we preserved the label ‘population orientation’ for Factor 4 and dropped the label ‘equity’. Factors 6 and 8 consisted of a reduced but sufficient number of items supporting two original dimensions (quality improvement and continuity) and were therefore retained. Although Factor 6 consisted of only two items, it was retained because these items were highly correlated with one another (>0.70) and uncorrelated or moderately correlated (<0.40) with other items. The factor loadings of items from the last original dimension (comprehensiveness) were <0.40, and as a result this factor was not included in the eight-factor solution. Therefore, the two original dimensions that were not included in the final eight-factor model were equity and comprehensiveness.

Based on the second EFA performed with the final 28 items, the total variance explained by the final eight-factor solution was 14.8%, with the proportion of variance explained by each factor 2.4% (community participation), 2.4% (patient-centred care), 2.1% (intersectoral team), 2.1% (accessibility/availability), 1.6% (interdisciplinary collaboration), 1.5% (quality improvement), 1.5% (population orientation), and 1.2% (continuity). The eight factors demonstrated moderate correlation (>0.30) between one another with the exception of: patient-centred care and quality improvement (0.27), continuity with all other factors except patient-centred care, and accessibility/availability with all other factors (Table 4).

Table 4 Factor correlation matrix of eight-factor solution based on second exploratory factor analysis

Reliability and summary statistics

Internal consistency reliability was estimated by the Cronbach’s α coefficient, which was 0.89 for the final 28-item PHCE Scale (Table 5). Six of the final eight subscales demonstrated α estimates ranging from 0.77 to 0.88, which exceeded the recommended standard of 0.70 (Nunnally and Bernstein, 1994). The α estimates for two of the final eight subscales fell below 0.70, specifically continuity (α=0.61) and population orientation (α=0.61).

Table 5 Summary statistics of final 28-item Primary Health Care Engagement (PHCE) scale and subscales

Scale scoring: 1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree.

The mean item scores for the final 28-item PHCE Scale and eight subscales demonstrated low (3.1–4.0) to high agreement (4.1–5.0) among rural/remote nurses on workplace engagement in key dimensions of primary health care (Table 5). Mean scores ranged from a low of 3.1 (SD=0.7) on intersectoral team indicating a low level of agreement that collaboration exists between the primary workplace and community sectors, to a high of 4.2 (SD=0.6) on patient-centred care indicating high agreement that respect, safety, and confidentiality toward patients is present in the primary workplace. Overall, the mean PHCE Scale score of 3.6 (SD=0.5) indicated low agreement that the primary workplaces of rural/remote nurses were engaged in primary health care delivery.

Discussion

The present study describes an evaluation to provide validity evidence for an instrument to measure workplace engagement in primary health care delivery. Evidence for the construct validity of conclusions that may be drawn from an instrument is generally derived from five sources, including the scale’s content and internal structure (Cook and Beckman, 2006). In an earlier study, we provided content evidence to demonstrate that the PHCE Scale represents community-based/oriented primary health care (Kosteniuk et al., 2016). Findings from the present study provide evidence of the internal structure of the PHCE Scale, based on an eight-factor structure identified in EFA and good internal consistency reliability (α=0.89).

Other provider-level instruments are available to assess health care sites on performance indicators of primary care (Schoen et al., 2006; Dahrouge et al., 2009) and PHC delivery (Johnston and Burge, 2013). However, the 28-item PHCE Scale differs from previous provider-level instruments in that it includes most of the principles common to contemporary community-based PHC approaches (eg, accessibility, community participation, and intersectoral collaboration) (Henderson et al., 2014; White, 2015; Baum et al., 2016).

Based on the results of EFA, the original 40-item PHCE Scale across 10 subscales was reduced to 28 items across eight factors. Five of the final eight factors retained all of the items proposed in the original subscales (community participation, patient-centred care, intersectoral team, accessibility/availability, and interdisciplinary collaboration). The factor loadings of these items were well above the recommended cut-off, and reliability estimates of the five factors were satisfactory (α>0.70). Two of the final eight factors (continuity and quality improvement) lost items from the original subscales, however, the remaining items were sufficient in number and demonstrated adequate loadings for the factors to be retained. The last of the eight factors consisted of items loading from two separate original subscales (population orientation and equity), and therefore the label that was consistent with the highest loading items was retained (population orientation). Lower than expected reliability estimates (α<0.70) were demonstrated by two of the final factors (continuity and population orientation), which may be due to low correspondence between items or heterogeneity of concepts within the factors (Tavakol and Dennick, 2011).

The overall 28-item PHCE score provides a broad measure of workplace engagement in primary health care delivery, while the eight separate 2–4 item subscales allow measurement of particular aspects of PHC engagement. Higher scores on the PHCE Scale indicate a higher level of workplace participation in PHC delivery. The final 28-item scale directly represents three of four principles common to a contemporary approach to PHC (Henderson et al., 2014; White, 2015; Baum et al., 2016) based on the original Alma Ata Declaration on Primary Health Care (World Health Organization, 1978): accessibility (accessibility/availability), community participation, and intersectoral collaboration (intersectoral team). The fourth element (equity) is indirectly represented in the population orientation subscale, indicating possible overlap in these two concepts or in the design of the items representing these two concepts. The remaining dimensions in the final PHCE Scale directly represent most elements of community-oriented (Levesque et al., 2011) and community-based PHC models (CIHR, 2015): a wide range of providers (interdisciplinary collaboration), person-centredness (patient-centred care), coordination and integration (continuity), population-centredness (population orientation), and quality improvement. An important element missing from the final PHCE Scale is comprehensiveness, the delivery ‘of a broad spectrum of health and social services’ (Levesque et al., 2011: 23) considered essential to community-based/oriented PHC. Although it is possible that as a conceptual category, comprehensiveness may not be an essential dimension of PHCE, it is more likely that the items did not adequately capture the complexity of this concept in the context of diverse rural/remote patient populations.

Some study limitations must be acknowledged. First, the proportion of participants that responded ‘not applicable’ was greater than 20% and as high as 49% for eight of the original 40 items across three subscales. We chose not to impute these ‘not applicable’ responses and other missing values, since including these values may have resulted in artificially high correlations (Tabachnick and Fidell, 2013). Six of these items remained in the final 28-item scale (A2, A3, and A4 in accessibility/availability; G1, G2, and G3 in intersectoral team). The high proportion of ‘not applicable’ responses to the accessibility/availability items indicates that some participants may view questions regarding workplaces that ‘open’ and ‘close’ as irrelevant to their practice context. These items may be more relevant to community-based settings that offer regular office hours and appointment times than to institutional settings that offer 24/7 care. The high proportion of ‘not applicable’ responses to the intersectoral team items may reflect a ‘tyranny of the acute’ organizational culture that places a low emphasis on population health issues in some workplaces, as well as challenges to working intersectorally without formal supporting structures in place (CIHI, 2014). However, accessibility/availability and intersectoral collaboration are principles of a contemporary approach to PHC and therefore it is important and appropriate to retain these items in the final 28-item PHCE Scale.

A second limitation is that we applied a stringent listwise deletion criterion to exclude cases with at least one missing item, resulting in a large number of excluded cases (n=2235). Participants employed in community-based health care settings were more likely to provide a valid response to all of the scale items and to be included in the analysis than their counterparts in institutional settings (eg, hospital, long-term care). However, the majority of included participants were employed outside of community-based health care sites, indicating that the overall PHCE Scale may be applicable across a range of health care settings. We recommend the use of a case mean imputation strategy (El-Masri and Fox-Wasylyshyn, 2005) when employing the PHCE Scale for purposes other than psychometric testing, so as to obtain the maximum number of valid responses (see the PHCE Scale in the Appendix for specific imputation guidelines). Third, the study sample included only regulated nurses working in rural/remote Canadian communities, and as such this limits the generalizability of our findings in terms of professional discipline and geography. Future research should evaluate the instrument outside of these populations. Finally, the comprehensive 27-page RRNII survey provided an opportunity to administer the original 40-item PHCE Scale to regulated nurses working across rural/remote Canada. However, the purpose of the survey was to gather data on the overall nature of rural/remote nursing practice across five wide-ranging content domains such as work community and nursing practice (MacLeod et al., 2017) and as such we were unable to include other measures that may be expected to correlate with the PHCE Scale. Therefore, future research should focus on assessing the relations between the 28-item PHCE Scale and other variables, to provide further evidence of the construct validity of conclusions drawn from the scores (Cook and Beckman, 2006).

Conclusion

The most important finding of the present study is that the results supported an eight-factor structure of the PHCE Scale with 28 items in total, offering a comprehensive measure of workplace engagement in primary health care that is short and easy to administer. The scale is intended to be administered by researchers and health care planners to any health care provider likely to be involved in PHC delivery.

The 28-item PHCE Scale reflects the high priority placed on community participation in rural/remote PHC programs and services (Preston et al., 2010), and the importance of community participation in PHC more broadly as emphasized in recent position papers (Henderson et al., 2014; Canadian Nurses Association, 2015; White, 2015; Baum et al., 2016) and PHC reform strategy reports (Alberta Health, 2014; Saskatchewan Ministry of Health, 2016). Although the scale may be more relevant to community-based than institutional health care settings such as hospitals and nursing homes, findings from the present study indicate that it is relevant to rural/remote providers from both settings. Further psychometric testing with a diversity of health care providers and settings is recommended.

Acknowledgements

The article stems from the study: ‘Nursing Practice in Rural and Remote Canada II’, led by Martha MacLeod, Norma Stewart and Judith Kulig (http:// ruralnursing.unbc.ca). The authors are grateful to the nurses who responded to this survey and to Nadine Mix, Leana Garraway, and Larine Sluggett for their assistance with data analysis.

Financial Support

The full study was funded by the Canadian Institutes of Health Research (MOP 130260).

Conflicts of Interest

None.

Ethical Standards

The ethics committees of the research team approved the RRNII survey: University of Northern British Columbia Research Ethics Board (E2013.0320.037.02), University of Saskatchewan (Behavioural Research Ethics Board Certificate of Approval), University of Lethbridge (Certificate of Human Participant research), Aurora College (Scientific Research License), University of Montreal (Hospital Maisonneuve-Rosemont), and Dalhousie University (Health Science Research Ethics Board letter of approval).

References

Alberta Health 2014: Alberta’s primary health care strategy. Retrieved 19 January 2017 from http://www.health.alberta.ca/documents/Primary-Health-Care-Strategy-2014.pdf.
Basto, M. and Pereira, J.-M. 2012: An SPSS R-menu for ordinal factor analysis. Journal of Statistical Software 46, 129.
Baum, F., Freeman, T., Sanders, D., Labonte, R., Lawless, A. and Javanparast, S. 2016: Comprehensive primary health care under neo-liberalism in Australia. Social Science and Medicine 168, 4352.
Bloch, G., Rozmovits, L. and Giambrone, B. 2011: Barriers to primary care responsiveness to poverty as a risk factor for health. BMC Family Practice 12, 62.
Canadian Institute for Health Information (CIHI) 2006. Pan-Canadian Health Care Indicators. Volume 1 Ottawa: CIHI, 1109.
Canadian Institute for Health Information (CIHI) 2014: Exploring a population health approach in health system planning and decision-making. Retrieved 14 April 2017 from https://secure.cihi.ca/free_products/CIHI_Bridiging_Final_EN_web.pdf.
Canadian Institutes of Health Research (CIHR) 2015: Community-based primary health care. Retrieved 19 January 2017 from http://www.cihr-irsc.gc.ca/e/43626.html.
Canadian Nurses Association 2015: Primary health care (Position Statement). Retrieved from 19 January 2017 from https://cna-aiic.ca/~/media/cna/page-content/pdf-en/primary-health-care-position-statement.pdf?la=en.
Cook, D. and Beckman, T. 2006: Current concepts in validity and reliability for psychometric instruments: theory and application. The American Journal of Medicine 119, e7e16.
Costello, A. and Osborne, J. 2005: Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation 10, 19.
Dahrouge, S., Hogg, W., Russell, G., Geneau, R., Kristjansson, E., Muldoon, L. and Johnston, S. 2009: The comparison of models of primary care in Ontario (COMP-PC) study: methodology of a multi-faceted cross-sectional practice-based study. Open Medicine 3, 149164.
Davis, K., Schoen, C., Schoenbaum, S., Doty, M., Holmgren, A., Kriss, J. and Shea, K. 2007: Mirror, mirror on the wall: an international update on the comparative performance of American Health Care. The Commonwealth Fund publication number 1027. Retrieved 19 January 2017 from commonwealthfund.org.
Dillman, D., Smyth, J.D. and Christian, L.M. 2014. Internet, phone, mail, and mixed-mode surveys: the tailored design method, 4th edition. Hoboken, NJ: John Wiley & Sons, Inc.
du Plessis, V., Beshiri, R., Bollman, R.D. and Clemenson, H. 2001: Definitions of rural. Rural and Small Town Canada Analysis Bulletin. Retrieved 19 January 2017 from http://www.statcan.gc.ca/pub/21-006-x/21-006-x2001003-eng.pdf.
El-Masri, M.M. and Fox-Wasylyshyn, S.M. 2005: Missing data: an introductory conceptual overview for the novice researcher. Canadian Journal of Nursing Research 37, 156171.
Flocke, S. 1997: Measuring attributes of primary care: development of a new instrument. Journal of Family Practice 45, 6474.
Ford, D. 2016: Four persistent rural healthcare challenges. Healthcare Management Forum 29, 243246.
Galdikiene, N., Asikainen, P., Rostila, I., Geen, P., Balciunas, S. and Suominen, T. 2016: Organizational social context in primary health care. Nordic Journal of Nursing Research 26, 103111.
Gaskin, C. and Happell, B. 2014: On exploratory factor analysis: a review of recent evidence, an assessment of current practice, and recommendations for future use. International Journal of Nursing Studies 51, 511521.
Gibson, O., Lisy, K., Davy, C., Aromataris, E., Kite, E., Lockwood, C., Ritano, D., McBride, K. and Brown, A. 2015: Enablers and barriers to the implementation of primary health care interventions for Indigenous people with chronic diseases: a systematic review. Implementation Science 10, 71.
Henderson, J., Koehne, K., Verrall, C., Gebbie, K. and Fuller, J. 2014: How is primary health care conceptualized in nursing in Australia? A review of the literature. Health and Social Care in the Community 22, 337351.
Howard, M. 2016: A review of exploratory factor analysis decisions and overview of current practices: what are we doing and how can we improve? International Journal of Human-Computer Interaction 31, 5162.
Johnston, S. and Burge, F. 2013: Measuring provider experiences in primary health care: report on the development of a PHC Provider Survey for the Canadian Institute for Health Information. Ottawa: Bruyere Research Institute. Retrieved 19 January 2017 from http://www.cphcrin-rcrissp.ca.
Kellar, S. and Kelvin, E. 2013. Munro’s statistical methods for health care research, 6th edition. Philadelphia, PA: Wolters Kluwer, Lippincott Williams & Wilkins.
Kenny, A., Farmer, J., Dickson-Swift, V. and Hyett, N. 2015: Community participation for rural health: a review of challenges. Health Expectations 18, 19061917.
Kosteniuk, J., Wilson, E., Penz, K., MacLeod, M., Stewart, N., Kulig, J., Karunanayake, C. and Kilpatrick, K. 2016: Development and psychometric evaluation of the Primary Health Care Engagement (PHCE) Scale: a pilot survey of rural and remote nurses. Primary Health Care Research and Development 17, 7286.
Levesque, J.-F., Haggerty, J., Burge, F., Beaulieu, I., Gass, D., Pineault, R. and Santor, D. 2011: Canadian experts’ views on the importance of attributes within professional and community-oriented primary healthcare models. Healthcare Policy 7, 2130.
Liu, Q., Wang, B., Kong, Y. and Cheng, K.K. 2011: China’s primary health care reform. Lancet 377, 20642066.
MacLeod, M., Kulig, J., Stewart, N. and Pitblado, R. 2004: Nursing practice in rural and remote Canada: final report to Canadian Health Services Research Foundation. Retrieved 27 January 2017 from http://www.unbc.ca/sites/default/files/sections/rural-nursing/en/17-rrnfinalreport.pdf.
MacLeod, M., Stewart, N., Kulig, J., Anguish, P., Andrews, M.E., Banner, D., Garraway, L., Hanlon, N., Karunanayake, C., Kilpatrick, K., Koren, I., Kosteniuk, J., Martin-Misener, R., Mix, N., Moffitt, P., Olynick, J., Penz, K., Sluggett, L., Van Pelt, L., Wilson, E. and Zimmer, L. 2017: Nurses who work in rural and remote communities in Canada: a national survey. Human Resources for Health15, 34.
Nunnally, J. and Bernstein, I. 1994. Psychometric Theory, 3rd edition. New York: McGraw-Hill, Inc.
Olden, P. and Hoffman, K. 2011: Hospitals’ health promotion services in their communities: findings from a literature review. Health Care Management Review 36, 104113.
Patil, B., Singh, S., Mishra, S. and Donavan, D. 2008: Efficient theory development and factor retention criteria: abandon the ‘eigenvalue greater than one’ criterion. Journal of Business Research 61, 162170.
Preston, R., Waugh, H., Larkins, S. and Taylor, J. 2010: Community participation in rural primary health care: intervention or approach? Australian Journal of Primary Health 16, 416.
Saskatchewan Ministry of Health 2012: Saskatchewan Framework for Primary Health Care Report. Regina, Saskatchewan. Retrieved 19 January 2017 from www.health.gov.sk.ca/primary-health-care.
Saskatchewan Ministry of Health 2016: Saskatchewan Advisory Panel on health system structure report: optimizing and integrating patient-centred care. Retrieved 19 January 2017 from http://s3.documentcloud.org/documents/3251960/Saskatchewan-Advisory-Panel-on-Health-System.pdf.
Schoen, C., Osborn, R. and Huynh, P. 2006: On the front lines of care: primary care doctors’ office systems, experiences, and views in seven countries. Health Affairs 25, w555w571.
Shi, L., Starfield, B. and Xu, J. 2001: Validating the adult primary care assessment tool. Journal of Family Practice 50, 1171.
Strumpf, E., Levesque, J.-F., Coyle, N., Hutchinson, B., Barnes, M. and Wedel, R. 2012: Innovative and diverse strategies toward primary health care reform: lessons learned from the Canadian experience. Journal of the American Board of Family Medicine 25, S27S33.
Tabachnick, B. and Fidell, L. 2013. Using multivariate statistics, 6th edition. Boston, MA: Pearson Education.
Tavakol, M. and Dennick, R. 2011: Making sense of Cronbach’s alpha. International Journal of Medical Education 2, 5355.
Thomas, S., Wakerman, J. and Humphreys, J. 2014: What core primary health care services should be available to Australians living in rural and remote communities? BMC Family Practice 15, 143.
Ward, B.M., Buykx, P.F., Tham, R., Kinsman, L. and Humphreys, J.S. 2014: Investing in longitudinal studies of primary healthcare: what can we learn about service performance, sustainability and quality? Rural and Remote Health 14, 3059.
White, F. 2015: Primary health care and public health: foundations of universal health systems. Medical Principles and Practice 24, 103116.
World Health Organization 1978: Declaration of Alma-Ata: Report of the International Conference on Primary Health Care Alma-Ata, USSR. Retrieved 11 July 2017 from http://apps.who.int/iris/bitstream/10665/39228/1/9241800011.pdf.
World Health Organization 1986: Ottawa charter for health promotion. Ottawa. Retrieved January 19, 2017 from http://www.phac-aspc.gc.ca/ph-sp/docs/charter-chartre/pdf/charter.pdf.
Wong, S., Browne, A., Varcoe, C., Lavoi, J., Smye, V., Godwin, O., Littlejohn, D. and Tu, D. 2011: Enhancing measurement of primary health care indicators using an equity lens: an ethnographic study. International Journal for Equity in Health 10, 38.

Appendix

Table A1 28-item Primary Heath Care Engagement (PHCE) Scale

Scoring: 1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree (including a ‘not applicable’ response option is not recommended).

Reverse items: 3, 18, 25, 26, 27, 28.

Subscales: quality improvement (1, 2); community participation (3, 4, 5, 6); patient-centred care (7, 8, 9, 10); accessibility-availability (11, 12, 13, 14); intersectoral team (15, 16, 17, 18); interdisciplinary collaboration (19, 20, 21); continuity (22, 23, 24); population orientation (25, 26, 27, 28).

Case mean imputation guidelines: in the four-item subscales, the case mean may be imputed where 25% or less of items is missing (ie, 1 item) (El-Masri and Fox-Wasylyshyn, 2005); if a participant’s subscale is missing two or more items, then that participant’s subscale should be discarded. Case mean imputation should not be performed in the two-item and three-item subscales; if a participant is missing one or more item, then that participant’s subscale should be discarded.