Skip to main content Accessibility help
×
×
Home

Information:

  • Access
  • Cited by 1
  • Cited by
    This article has been cited by the following publications. This list is generated based on data provided by CrossRef.

    Verma, Amol A. Klich, John Thurston, Adam Scantlebury, Jordan Kiss, Alex Seddon, Gayle and Sinha, Samir K. 2018. Paramedic-Initiated Home Care Referrals and Use of Home Care and Emergency Medical Services. Prehospital Emergency Care, Vol. 22, Issue. 3, p. 379.

    ×

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Paramedics assessing Elders at Risk for Independence Loss (PERIL): Derivation, Reliability and Comparative Effectiveness of a Clinical Prediction Rule
        Available formats
        ×

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        Paramedics assessing Elders at Risk for Independence Loss (PERIL): Derivation, Reliability and Comparative Effectiveness of a Clinical Prediction Rule
        Available formats
        ×

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        Paramedics assessing Elders at Risk for Independence Loss (PERIL): Derivation, Reliability and Comparative Effectiveness of a Clinical Prediction Rule
        Available formats
        ×
Export citation

Abstract

Objectives

We conducted a program of research to derive and test the reliability of a clinical prediction rule to identify high-risk older adults using paramedics’ observations.

Methods

We developed the Paramedics assessing Elders at Risk of Independence Loss (PERIL) checklist of 43 yes or no questions, including the Identifying Seniors at Risk (ISAR) tool items. We trained 1,185 paramedics from three Ontario services to use this checklist, and assessed inter-observer reliability in a convenience sample. The primary outcome, return to the ED, hospitalization, or death within one month was assessed using provincial databases. We derived a prediction rule using multivariable logistic regression.

Results

We enrolled 1,065 subjects, of which 764 (71.7%) had complete data. Inter-observer reliability was good or excellent for 40/43 questions. We derived a four-item rule: 1) “Problems in the home contributing to adverse outcomes?” (OR 1.43); 2) “Called 911 in the last 30 days?” (OR 1.72); 3) male (OR 1.38) and 4) lacks social support (OR 1.4). The PERIL rule performed better than a proxy measure of clinical judgment (AUC 0.62 vs. 0.56, p=0.02) and adherence was better for PERIL than for ISAR.

Conclusions

The four-item PERIL rule has good inter-observer reliability and adherence, and had advantages compared to a proxy measure of clinical judgment. The ISAR is an acceptable alternative, but adherence may be lower. If future research validates the PERIL rule, it could be used by emergency physicians and paramedic services to target preventative interventions for seniors identified as high-risk.

Introduction

Many older adults are at an increased risk for preventable events such as falls, delirium, and functional decline. 1 One in five seniors lack the support they need to function daily. 2 Gurley et al. chillingly captured the potential severity of the problem: those found helpless or dead accounted for 3.7% of calls to emergency medical services (EMS). 3 This population-based study showed that the rate of being “found down” increased dramatically with age, from 3/1,000 person years in the 60–64 age group, to 27/1,000 person years among patients 85 and older. 4 More than 35% of those found down had been incapacitated for over 12 hours; those found down after 72 hours had a mortality of 67% during their hospitalization. 3 Clearly, there are isolated vulnerable older people living in the community with unmet needs who could benefit from a safety net. 2

Due to social isolation and a lack of support and timely access to primary care, such frail older adults frequently depend on EMS to function as their safety net. 4 This may explain why, despite their increased disease burden, the proportion of non-urgent calls increases with age and is twice as common among patients 65 years and older compared to younger adults. 5 Up to one-third of ambulance calls among older adults are for assistance with mobility (e.g., being unable to rise from the toilet or after a non-injurious fall). 4 Older adults are more likely to call 911 on multiple occasions and to decline transportation to hospital after calling 911, compared to younger adults. 6 Repeat 911 calls account for 18% to 40% of ambulance calls among people 65 years of age or older. 7 Thus, an older adult calling 911 for a seemingly minor problem may be a “sentinel” event, signaling a diminished ability to continue functioning independently. 8 However, interventions to prevent adverse outcomes such as falls, 9 hospitalization, 10 and functional decline 10 have not been shown to be either effective or economical when applied broadly to all older adults. 10 - 12 Interventions must be targeted at high-risk individuals to be effective. 13 , 14 The Identifying Seniors at Risk (ISAR) tool was developed to identify high-risk older patients in the ED, but has never been validated for use by paramedics. 15

Given their frequent encounters in the homes of older adults, and the substantial proportion of older EMS patients who are not transported to ED, paramedics may be uniquely positioned to identify those at risk for adverse outcomes. 16 This potential has increasingly been recognized by the creation of “community paramedicine” programs, 17 - 20 but paramedics have historically received little specialized geriatric training to help them identify older adults at risk for adverse outcomes. 12 Thus we sought to develop the “Paramedics assessing Elders at Risk for Independence Loss” (PERIL) clinical prediction rule to permit paramedics to identify high-risk older adults.

Our specific objectives were: 1) to test the reliability of 43 candidate questions for a clinical prediction rule; 2) to derive the PERIL clinical prediction rule, a checklist for use by paramedics to assess whether an older adult is at high risk for repeat ED visits, hospitalization or death in the month following an EMS encounter; 3) to compare our prediction rule to a proxy measure of paramedic global assessment of risk; and 4) to compare our prediction rule to the existing ISAR prediction tool when employed by paramedics in the prehospital setting.

Methods

Study design

This program of research included the development of potential variables, reliability testing, and the derivation of the PERIL clinical prediction rule following methodological standards for decision rules. 21 A pool of 43 candidate predictor variables was selected based on three published systematic reviews. 11 , 22 , 23 Our aim was to ensure that the rule was “clinically sensible,” i.e., all required data elements would be available to paramedics and feasible to collect during the EMS encounter. 26 Development of the data collection form was guided by an expert panel composed of a clinical epidemiology and decision rule expert, two geriatricians, and a paramedic with research experience, as well as paramedic focus groups (see Figure 1). All items were framed as simple yes or no questions. One question was included to act as a proxy measure of paramedics’ global assessment of risk for adverse outcomes: “Given the current home situation, are there any problems that would prevent this client from being safely discharged home from the ED, or contribute to recurrent EMS/Emergency use?” We recognized that when completing our proxy variable of clinical judgment, paramedics in the study would have the benefit of the training they received and the completion of the 42 other variables at the same time. This meant that our proxy measure of clinical judgment would be a “conservative” comparison standard for the performance of the PERIL rule relative to truly unaided clinical judgment by untrained paramedics. We also included all questions from the Identifying Seniors at Risk (ISAR), and trained paramedics in how to use the ISAR questions. 24

Figure 1 PERIL (Paramedics assessing Elders at Risk for Independence Loss) data collection form.

We then developed a curriculum to educate paramedics about the study rationale, objectives, and use of the PERIL data form. 25 We used existing one-hour EMS continuing educational sessions to train 1,185 paramedics, including advanced and basic care providers. Paramedics received a didactic lecture about the presentation of common geriatric syndromes and signs of reduced function evident in the home setting. Paramedics then viewed a video-taped simulated encounter between a paramedic crew and an older patient, and recorded any problems they observed. Finally, we conducted a prospective observational cohort study to derive the PERIL clinical prediction rule. We assessed inter-rater reliability for all candidate predictor variables in a convenience sub-sample when two paramedics were able to complete the PERIL data for the same patient. 25

Study participants

Eligible patients of three Canadian EMS systems, including Toronto, Ottawa, and Renfrew County, Ontario, were 65 years of age and older and assessed in their home by a paramedic responding to a 911 call. Recruitment occurred from January 1, 2005 to March 15, 2008. We excluded patients who were critically ill, as determined by the Canadian Triage Assessment Scale 26 (CTAS category 1); who lived in a nursing home, as determined by a list of such institutions for each community; or who were attended to in a public place rather than in the their home, because our hypothesis was that paramedics could observe clues of declining function in the patient’s home.

Methods of measurement

Baseline measures

Baseline demographic measures included patients’ age and sex. Information abstracted from the paramedic Ambulance Call Record (ACR) included the reason for the 911 call, chronic medical conditions, regular prescription medications, vital signs, and whether the patient was transported to an ED or not.

PERIL data-collection form

The standardized PERIL checklist comprised of 38 dichotomous variables (see Figure 1) that paramedics observed when attending to an older adult in the patient’s home. The five ISAR items were based on asking the patient questions instead of direct observations, for a total of 43 items (See Figure 1).

Outcome measures

Our primary composite measure of adverse outcomes was based on previous studies looking at adverse outcomes in older ED patients. 24 , 27 , 28 It included any repeat visit to the ED, hospitalization, or death within 30 days of the initial EMS encounter. We excluded adverse outcomes that were related to the initial EMS call, i.e., occurred on the same day as the initial EMS call with no ED discharge occurring prior to the adverse event. Patients’ data were linked to provincial administrative databases, including the National Ambulatory Care Reporting System, the Discharge Abstract Database, and the Registered Persons Database. We used deterministic matching for 79% of the cohort where Ontario Health Insurance Plan (OHIP) numbers were available and probabilistic matching based on name and date of birth for the rest of the cohort. These datasets were linked using unique, encoded identifiers and analyzed at the Institute for Clinical Evaluative Sciences (ICES).

Statistical analysis

Sample size calculation

We estimated that a sample size of 1,000 older adults would provide sufficient power to examine a minimum of 35 predictive variables 29 and detect odds ratios of 1.6 or greater, assuming a power of 80%, alpha level of 0.05, and adjusting for an r-squared of 0.4 between the independent variables in the model, and a primary outcome rate of 35% observed in our pilot data. 29 , 30

Model derivation

We chose logistic regression modeling over recursive partitioning to derive and maximize the specificity of the PERIL prediction rule, 31 given the limited resources available for frail older adults. A highly sensitive rule with low specificity would identify too many “false positive” individuals who would consume the limited available resources without actually requiring them. We used Harrell’s technique to build logistic regression models. 32 In summary, only clinically relevant predictor variables were chosen and variables were not screened based on univariate association with the primary outcome to protect against type-I error. We refined the list by excluding predictors that lack variables with greater than 95% of observations in a single category, or variables with poor inter-rater agreement (κ<.60). We used correlation matrix analysis to assess for high multicollinearity (VIF > 2.5), and if variables showed high multicollinearity, we chose the most clinical relevant variable, or ran bootstrap simulations for sets of equally clinical relevant variables with high multicollinearity. 33 We then built candidate predictive models using backward stepwise regression and selected the final model based on predictive ability, ease of use, and parsimony (i.e., fewest number of variables). Receiver Operating Curve (ROC) of the final PERIL prediction rule, paramedic clinical judgment, and the ISAR tool were also reported and areas under the curve (AUC) were compared. 34

Ethics

The Sunnybrook and Ottawa Hospital Research Ethics Boards approved the study. As a minimal risk observational study, the requirement for written informed consent was waived.

Results

Characteristics of study subjects

Paramedics screened 1,418 older adults, and 1,065 (75.1%) were eligible. All 1,065 eligible subjects were linked with provincial databases to acquire patient outcomes and 764 (71.7%) had complete data allowing logistic-regression (see Figure 2). There was no difference in adverse outcome rates between patients with and without complete data (269/764 or 35.2%, 95% CI 31.8% to 38.7%, for subjects with complete data, vs. 372/1,065 or 34.9% , 95% CI 32.1% to 37.9%, among subjects missing at least one variable). Their mean age was 80.9 years (interquartile range 75.3 to 86.3 years), 297 (38.9%) were male and 54 (7%) were not transported to hospital (see Table 1).

Figure 2 PERIL derivation study flow chart.

Table 1 Demographic characteristics of eligible subjects in the cohort used to derive the PERIL decision rule

Note: CI=confidence interval; CTAS=Canadian Triage and Acuity Scale.

Primary outcome

A total of 269 individuals experienced 291 adverse outcomes, for a primary outcome rate of 35.2% (269/764). The three components of the primary outcome broke down as 169/764 (22.1%) older adults who were hospitalized, 89/764 (11.6%) who had a subsequent ED visit, and 33/764 who died after the initial EMS encounter (4.8%). All adverse events occurred within 30 days of the initial call to 911. Of the total 291 composite adverse outcomes, 18 older adults experienced two events and two experienced three events.

Selection of variables

Five variables were answered “yes” in less than 5% of patients, and were excluded from analysis: home too cold, chronic renal failure, agitation, delusions, and suspected elder abuse. Given the extremely low frequency and their weak univariate association with our primary outcome, they did not influence the final model in a sensitivity analysis. Two sets of variables showed multicollinearity: a) “previous 911 use” and “previous ED use”; and b) the three variables “lacks help needed to bathe/shower,” “lacks help needed to get to toilet,” and “lacks help needed to clean living quarters.” For the first sets of collinear variables, we selected “previous 911 use,” given our focus on prehospital screening. For the second set of collinear variables, “lacks help needed to bathe/shower” was selected because it showed the strongest association with the outcome in bootstrap simulation analysis in the absence of any clear superiority clinically. Thus a total of 38 of the original 43 variables were entered into the logistic regression model (see Table 2).

Table 2 Inter-observer reliability, adherence and univariate association with adverse outcomes of 38 potentially predictive variables included in the logistic regression analysis

Reliability and adherence

There was “good” (kappa ≥0.6) to “excellent” (kappa ≥0.8) reliability for 40/43 of the questions. 35 Among the 38 questions retained for logistic regression, reliability was “good” to “excellent” for 37/38 questions and “substantial” for one question (history of chronic medical conditions—see Table 2). Thus no questions were excluded on the basis of poor reliability or kappa <0.6. Paramedic adherence in completing individual questions is also listed in Table 2. In addition, rates of missing data among the 38 questions ranged from 0 to 328/1065 (30.8%). The ISAR variables that required paramedics to question the patient directly were missing in 27% of cases or more, whereas variables based on paramedic observations had high adherence rates (see Table 2).

Model derivation

Four significant independent predictors remained after model selection: 1) 911 use in the past 30 days; 2) “Given the current home situation, are there any problems that would prevent this client from being safely discharged home from the ED, or contribute to recurrent EMS/Emergency use?”; 3) overall lack of social support, defined as “Patient has unmet needs for social support that, in your opinion, will contribute to recurrent EMS/Emergency use”; and 4) male sex (see Table 3).

Table 3 The Paramedics assessing Elders at Risk for Independence (PERIL) Rule

Note: ED=Emergency Department; EMS=emergency medical services.

The four-item PERIL prediction rule was significantly associated with the composite adverse primary outcome measure, and the Hosmer-Lemeshow statistic failed to show any lack of fit (χ2=2.51, df=7, p=0.93). The PERIL prediction rule had an AUC of 0.62, compared to the AUC of 0.57 for clinical judgment as captured by the single proxy variable of paramedic overall risk assessment (p=0.02). Since the odds ratios for all variables in the final predictive model were approximately equal, we constructed the PERIL prediction rule, with one point assigned for each positive question resulting in a maximum of four points, with a range of one to four points. (See Table 4). The sensitivity and specificity for all cut-off values are listed in Table 5.

Table 4 Predictive performance of the Paramedics assessing Elders at Risk for Independence (PERIL) clinical prediction rule

Note: CI=confidence interval; ED=emergency department; EMS=emergency medical services.

Table 5 Comparing adverse outcomes, sensitivity and specificity of the Paramedics assessing Elders at Risk for Independence Loss (PERIL) compared to single global risk assessment variable

Note: ED=Emergency Department; EMS=emergency medical services.

Finally, we compared the predictive performance of the ISAR to the four-item PERIL rule in a subsample of 633 subjects with complete ISAR variables. The AUC for the ISAR was similar to the PERIL tool in this subsample (0.57 vs. 0.61 respectively, p=0.28).

Discussion

This large multi-centred observational study demonstrated the feasibility of Canadian paramedics identifying older adults at high risk of future adverse events using a clinically sensible prehospital rule, i.e., one that uses information that can be reasonably collected by paramedics at the time of the 911 call. We found that paramedics could reliably observe data needed for the PERIL rule when attending to older patients; inter-observer reliability was substantial to excellent for all variables considered for the model.

This study adds to the evidence supporting the feasibility of community paramedicine, a model of care whereby paramedics apply their training and skills in “non-traditional,” preventative, community-based roles. 19 - 24 Gerson et al. first studied screening by paramedics for clues of unmet needs in the home environment of older adults. 18 In 121/124 cases where paramedics identified older adults as having unmet needs, this was confirmed during a home-visit by a “gold-standard” geriatric assessor from a community agency. Gerson et al. also reported that 91% of the paramedics supported the screening of vulnerable elders and wished to continue doing so.

Paramedic ability to identify high-risk older adults using unaided clinical judgment had not been studied, so we assessed whether a single proxy variable assessing paramedic global assessment of risk would be sufficient to identify high-risk older adults. Addition of the three other PERIL questions had superior predictive ability compared to our single proxy variable of “unaided” clinical judgement (see Table 5). An additional advantage of the four-question prediction rule, over the single proxy measure of paramedic judgment alone, is that it allows the use of different cut-off levels tailored to the needs of the paramedic service (see Table 5). The specificity achieved with the single proxy variable of clinical judgement was 41.5%, whereas specificity was improved to 83.8% when a cut-off of three was chosen for the PERIL rule (see Table 5). High-volume systems that need to limit the number of referrals made may choose a more specific threshold. In low-volume rural settings, a more sensitive threshold can be chosen. It should also be noted that participating paramedics had undergone 60 minutes of training on assessing the 42 other risk factors for adverse outcomes, and this may have improved the predictive ability of our proxy variable compared to truly “unaided” clinical judgement among untrained paramedics.

McCusker et al. were the first group that followed methodological standards for developing an ED-based clinical prediction rule targeting older adults. The five-item ISAR tool was originally designed to identify ED patients over 70 at high risk for adverse events within six months of the original ED visit. The adverse events observed included functional decline in 54%, death in 35%, and nursing-home admission in 10.8%. The ISAR showed an AUC of 0.71, sensitivity of 72%, and specificity of 58% for this composite outcome. 32 The data from the original study were subsequently used to assess the ability of the ISAR to predict return to the ED within 30 days; the predictive performance for early return to ED was substantially lower (AUC 0.63). 27 Our literature search revealed that our study is the first to examine use of the ISAR by paramedics.

While there was no statistically significant difference in the predictive performance of ISAR compared to the PERIL rule, the completion of the ISAR questions was lower—all five of the ISAR questions were missing in more than 27% of eligible patients. In contrast, paramedics completed the three observation-based PERIL items nearly 100% of the time. Thus, paramedics provided sufficient data to compute a meaningful score based on three items of the PERIL score nearly 100% of the time.

Our composite outcome was chosen based on existing ED screening rules, and was intended to represent adverse outcomes that might be prevented by appropriate preventative management. There is some debate as to whether return ED visits should be considered as an adverse outcome, as McCusker has shown that most ED visits by older people are appropriate. 23 One approach is to look at whether repeat ED visits are for the same condition. However, it is problematic to use administrative databases such as NACRS to capture whether a return visit is for the same problem. For example, a patient with non-cardiac chest pain could have several ICD-10 discharge codes, even if they returned with a similar complaint. However, 169/269 (62.8%) of adverse outcomes in this study were subsequent hospitalization, and only 89/269 (33.1%) were for return to the ED.

This study has several limitations. First, we were unable to collect data on patients paramedics failed to screen due to problems with paramedic system information systems at the time of the study, and thus we have no data on the potential number of patients that were not screened. There was a further loss of 28.3% of eligible subjects due to missing variables. Clearly, our data show that paramedics are more likely to complete observation-based items, as opposed to items that require paramedics to ask patients additional questions that were outside their usual workflow. The only variable in the final PERIL prediction rule that was collected directly from the patient was “previous 911 use,” which was asked of the patients in 71.8% of cases. The recent adoption of electronic patient care records by many paramedic and EMS services will allow for the automated collection of previous 911 use in the future, and improve the accuracy and capture rate of this variable.

The predictive performance of the PERIL prediction rule was only moderate, with an area under the curve of 0.62, and low sensitivity. However the PERIL rule was designed to maximize specificity over sensitivity, to avoid patients falsely identified as being at high risk. We chose this strategy because in most Canadian health care systems, there is more demand for community health care services than supply, and the current standard for referral to such services is unaided clinical judgement. Multiple studies have shown that ED staff have difficulty identifying older adults at high risk for adverse outcomes using clinical judgement alone. 11 , 36 , 37 Our data confirm that paramedics are also challenged to identify high-risk older individuals with their clinical judgement alone, even when aided by training and completion of 42 other variables to assess risk at the same time.

In summary, this study demonstrated the feasibility, high reliability, and adherence of trained paramedics with completing a 43-item assessment of older patients to identify those at high risk for adverse outcomes within one month of an EMS encounter. The PERIL prediction rule has advantages compared to a proxy measure of paramedic clinical judgement, including better predictive performance and the ability to set different thresholds tailored to the needs of the paramedic service. While the ISAR is an acceptable alternative in terms of predictive ability, PERIL is a simpler rule that had better adherence. Further research should validate the PERIL predication rule in an independent population of older patients, validate the four-item version of PERIL, and explore methods to optimize screening of older adults at high risk for adverse outcomes. If validated, future research should explore whether emergency physicians’ and paramedic services’ use of the PERIL prediction rule can be used to target high-risk individuals for referrals to preventative community-based interventions (e.g., primary care, home care, community paramedicine teams, or geriatric outreach teams), and whether such referrals reduce adverse outcomes.

Acknowledgments

JSL, RPV, MJS, LM, IS, DR, AK and GN conceived the study idea, and designed the trial. JSL, RPV, MJS, LC, IS, JT, LM, MN, BHR, SS, DR, AK and GN helped obtain research funding. JSL, LC, MN, BHR, and SS undertook recruitment of participating centers supervised the conduct of trial and data collection. JSL conducted training, executed recruitment of patients, data collection and managed the data, including quality control. MS, GN, IS, RPV and AK provided statistical advice on study design and analyzed the data. MS, LM, DR, AK, and GN provided advice on analysis. GN is the senior author and acted as academic mentor to JSL. JSL drafted the manuscript, and all authors contributed substantially to its revision and provided final approval. JSL takes responsibilities for the paper as a whole.

Competing Interests: The authors acknowledge the Canadian Institutes of Health Research, Institute of Aging, who supported this work. Dr. Rowe is supported as a Tier I Canada Research Chair in Evidence-based Emergency Medicine by the Canadian Institutes of Health Research (CIHR) through the Government of Canada (Ottawa, ON). Dr. Naglie is supported by the George, Margaret and Gary Hunt Family Chair in Geriatric Medicine, University of Toronto. Dr. Morrison holds the Robert and Dorothy Pitts Chair in Acute Care and Emergency Medicine, Li Ka Shing Knowledge Institute, St Michael’s Hospital and the University of Toronto. She also receives salary support from the National Institute of Health, USA for studies related to cardiac arrest and trauma.

References

1. Fried, LP, Ferrucci, L, Darer, J, et al. Untangling the concepts of disability, frailty, and comorbidity: implications for improved targeting and care. J Gerontol A Biol Sci Med Sci 2004;59(3):255-263.
2. Desai, MM, Lentzner, HR, Weeks, JD. Unmet need for personal assistance with activities of daily living among older adults. Gerontologist 2001;41(1):82-88.
3. Gurley, R, Lum, N, Sande, M, et al. Persons found in their homes helpless or dead. N Engl J Med 1996;27(334(26)):1710-1716.
4. Shah, MN, Glushak, C, Karrison, TG, et al. Predictors of emergency medical services utilization by elders. Acad Emerg Med 2003;10(1):52-58.
5. Clark, MJ, FitzGerald, G. Older people’s use of ambulance services: a population based analysis. J Accid Emerg Med 1999;16(2):108-111.
6. Marks, PJ, Daniel, TD, Afolabi, O, et al. Emergency (999) calls to the ambulance service that do not result in the patient being transported to hospital: an epidemiological study. Emerg Med J 2002;19(5):449-452.
7. Weiss, SJ, Ernst, AA, Miller, P, et al. Repeat EMS transports among elderly emergency department patients. Prehosp Emerg Care 2002;6(1):6-10.
8. Bernstein, E. Repeat visits by elder emergency department patients: sentinel events. Acad Emerg Med 1997;4(6):538-539.
9. Tinetti, ME, Baker, DI, McAvay, G, et al. A multifactorial intervention to reduce the risk of falling among elderly people living in the community. N Engl J Med 1994;331(13):821-827.
10. McCusker, J, Jacobs, P, Dendukuri, N, et al. Cost-effectiveness of a brief two-stage emergency department intervention for high-risk elders: results of a quasi-randomized controlled trial. Ann Emerg Med 2003;41(1):45-56.
11. Aminzadeh, F, Dalziel, WB. Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions. Ann Emerg Med 2002;39(3):238-247.
12. Wilber, ST, Gerson, LW. A research agenda for geriatric emergency medicine. Acad Emerg Med 2003;10(3):251-260.
13. Miller, DK, Lewis, LM, Nork, MJ, et al. Controlled trial of a geriatric case-finding and liaison service in an emergency department. J Am Geriatr Soc 1996;44(5):513-520.
14. IICE ed. Challenges and Innovations in the Care of Older Emergency Department Patients. Montreal, Quebec, Canada: International Interdisciplinary Conference on Emergencies; 2005.
15. McCusker, J, Bellavance, F, Cardin, S, et al. Screening for geriatric problems in the emergency department: reliability and validity. Identification of Seniors at Risk (ISAR) Steering Committee. Acad Emerg Med 1998;5(9):883-893.
16. Gerson, LW, Schelble, DT, Wilson, JE. Using paramedics to identify at-risk elderly. Ann Emerg Med 1992;21(6):688-691.
17. Ball, L. Setting the scene for the paramedic in primary care: a review of the literature. Emerg Med J 2005;22(12):896-900.
18. Dixon, S, Mason, S, Knowles, E, et al. Is it cost effective to introduce paramedic practitioners for older people to the ambulance service? Results of a cluster randomised controlled trial. Emerg Med J 2009;26(6):446-451.
19. Mason, SM, Knowles, EL, Colwell, B. A randomized controlled trial to evaluate the effectiveness of community paramedic practitioners managing older people calling emergency medical services with minor acute conditions [Abstract]. Acad Emerg Med 2005;12(Suppl 1):134-135.
20. Weiss, SJ, Chong, R, Ong, M, et al. Emergency medical services screening of elderly falls in the home. Prehosp Emerg Care 2003;7(1):79-84.
21. Stiell, IG. Clinical decision rules in the emergency department. CMAJ 2000;163(11):1465-1466.
22. Grief, CL. Patterns of ED use and perceptions of the elderly regarding their emergency care: a synthesis of recent research. J Emerg Nurs 2003;29(2):122-126.
23. McCusker, J, Karp, I, Cardin, S, et al. Determinants of emergency department visits by older adults: a systematic review. Acad Emerg Med 2003;10(12):1362-1370.
24. McCusker, J, Bellavance, F, Cardin, S, et al. Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc 1999;47(10):1229-1237.
25. Lee, J, Verbeek, R, Morrison, L, et al. PERIL - Paramedics and Elders at Risk of Independence Loss: Agreement between paramedics in both simulated and live patient encounters. Can J Ger 2008;11(1):56 [Abstract].
26. Beveridge, R, Ducharme, J, Janes, L, et al. Reliability of the Canadian emergency department triage and acuity scale: interrater agreement. Ann Emerg Med 1999;34(2):155-159.
27. McCusker, J, Bellavance, F, Cardin, S, et al. Prediction of hospital utilization among elderly patients during the 6 months after an emergency department visit. Ann Emerg Med 2000;36(5):438-445.
28. Meldon, SW, Mion, LC, Palmer, RM, et al. A brief risk-stratification tool to predict repeat emergency department visits and hospitalizations in older patients discharged from the emergency department. Acad Emerg Med 2003;10(3):224-232.
29. Peduzzi, P, Concato, J, Kemper, E, et al. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996;49(12):1373-1379.
30. Hsieh, FY, Bolock, DA, Larsen, MD. A simple method of sample size calculation for linear and logistic regression. Stat Med 1998;17(14):1623-1634.
31. Cook, EF, Goldman, L. Empiric comparison of mulitvariate analytic techniques: Advantages and disadvantages of recursive paritioning analysis. J Chronic Dis 1984;37(9-10):721-731.
32. Harrell, F. Regression Modeling Strategies. New York: Springer-Verlag; 2002.
33. Efron, B, Tibshirani, RJ. An Introduction to the Bootstrap. New York: Chapman & Hall; 1993.
34. Lee, JS, Schwindt, G, Langevin, M, et al. Validation of the triage risk stratification tool to identify older persons at risk for hospital admission and returning to the emergency department. J Am Geriatr Soc 2008;56(11):2112-2117.
35. Landis, JR, Koch, GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-174.
36. McNamara, RM, Rousseau, E, Sanders, AB. Geriatric emergency medicine: a survey of practicing emergency physicians. Ann Emerg Med 1992;21(7):796-801.
37. Singal, BM, Hedges, JR, Rousseau, EW, et al. Geriatric patient emergency visits. Part I: Comparison of visits by geriatric and younger patients. Ann Emerg Med 1992;21(7):802-807.