Skip to main content Accessibility help
×
Home

Methodology of a multi-site reliability study: EPSILON Study 3

  • Aart H. Schene (a1), Maarten Koeter (a1), Bob van Wijngaarden (a1), Helle Charlotte Knudsen (a2), Morven Leese (a3), Mirella Ruggeri (a4), Ian R. White (a5), José Luis Vázquez-Barquero (a6) and EPSILON Study Group...

Abstract

Background

The European Psychiatric Services: Inputs Linked to Outcome Domains and Needs (EPSILON) Study aims to produce standardised versions in five European languages of instruments measuring needs for care, family or caregiving burden, satisfaction with services, quality of life, and sociodemographic and service receipt.

Aims

To describe background, rationale and design of the reliability study, focusing on reliable instruments, reliability testing theory, a general reliability testing procedure and sample size requirements.

Method

A strict protocol was developed, consisting of definitions of the specific reliability measures used, the statistical methods used to assess these reliability coefficients, the development of statistical programmes to make inter-centre reliability comparisons, criteria for good reliability, and a general format for the reliability analysis.

Conclusion

The reliability analyses are based on classical test theory. Reliability measures used are Cronbach's α, Cohen's κ and the intraclass correlation coefficient. Intersite comparisons were extended with a comparison of the standard error of measurement. Criteria for good reliability may need to be adapted for this type of study. The consequences of low reliability, and reliability differing between sites, must be considered before pooling data.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Methodology of a multi-site reliability study
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Methodology of a multi-site reliability study
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Methodology of a multi-site reliability study
      Available formats
      ×

Copyright

Corresponding author

Professor Aart H. Schene, Academic Medical Centre, Rm. A3.254, PO Box 22700, 1100 DE Amsterdam, The Netherlands. Tel: +31 20 566 2088; fax: +31 20 697 1971

Footnotes

Hide All

Declaration of interest

No conflict of interest. Funding detailed in Acknowledgements.

Footnotes

References

Hide All
Armitage, P. & Berry, G. (1994) Statistical Methods in Medical Research (3rd edn). Oxford: Blackwell Scientific.
Becker, T., Knapp, M., Knudsen, H. C., et al (1999) The EPSILON Study of schizophrenia in five European countries: design and methodology for standardising outcome measures and comparing patterns of care and service costs. British Journal of Psychiatry, 175, 514521.
Becker, T., Knapp, M., Knudsen, H. C., et al (2000) Aims, outcome measures, study sites and patient sample. EPSILON Study I. British Journal of Psychiatry, 177 (suppl. 39), s1s7.
Beecham, J. & Knapp, M. (1992) Costing psychiatric interventions. In Measuring Mental Health Needs (eds Thornicroft, G., Brewin, C. R. & Wing, J.), pp. 163183. London: Gaskell.
Chisholm, D., Knapp, M. R. J., Knudsen, H. C., et al (2000) The Client Socio-Demographic and Service Receipt Inventory: development of an instrument for international research. Epsilon Study 5. British Journal of Psychiatry, 177 (suppl. 39), s28s33.
Crocker, L. & Algina, J. (1986) Introduction to Classical and Modern Test Theory. New York: Holt, Rinehart & Winston.
Donner, A. & Bull, S. (1983) Inferences concerning a common intraclass correlation coefficient. Biometrics, 39, 771775.
Donner, A. & Eliasziw, M. (1987) Sample size requirements for reliability studies. Statistics in Medicine, 6, 441448.
Dunn, G. (1989) Design and Analysis of Reliability Studies. London: Edward Arnold.
Feldt, L., Woodruff, D. & Salih, F. (1987) Statistical inference for coefficient alpha. Applied Psychological Measurement, 11, 93103.
Gaite, L., Vázquez-Barquero, J. L., Arriaga Arrizabalaga, A., et al (2000) Quality of life in schizophrenia: development, reliability and internal consistency of the Lancashire Quality of Life Profile – European Version. EPSILON Study 8. British Journal of Psychiatry, 177 (suppl. 39), s49s54.
Knudsen, H. C. & Thornicroft, G. (1996) Mental Health Service Evaluation. Cambridge: Cambridge University Press.
Knudsen, H. C., Vázquez-Barquero, J. L., Welcher, B., et al (2000) Translation and cross-cultural adaptation of outcome measurements for schizophrenia. EPSILON Study 2. British Journal of Psychiatry, 177 (supp. 39), s8s14.
Landis, J. R. & Koch, G. G. (1977) The measurement of agreement for categorical data. Biometrics, 33, 159–74.
McCrone, P., Leese, M., Thornicroft, G., et al (2000) Reliability of the Camberwell Assessment of Need – European Version. EPSILON Study 6. British Journal of Psychiatry, 177 (suppl. 39), s34s40.
Oliver, J. (1991) The social care directive: development of a quality of life profile for use in the community services for the mentally ill. Social Work & Social Sciences Review, 3, 545.
Oliver, J., Huxley, P., Priebe, S. & Kaiser, W. (1997) Measuring the quality of life of severely mentally ill people using the Lancashire Quality of Life Profile. Social Psychiatry and Psychiatric Epidemiology, 32, 7683.
Phelan, M., Slade, M., Thornicroft, G., et al (1995) The Camberwell Assessment of Need: the validity and reliability of an instrument to assess the needs of people with severe mental illness. British Journal of Psychiatry, 167, 589595.
Ruggeri, M. & Dall'Agnola, R. (1993) The development and use of the Verona Expectations for Care Scale (VECS) and the Verona Service Satisfaction Scale (VSSS) for measuring expectations and satisfaction with community-based psychiatric services in patients, relatives and professionals. Psychological Medicine, 23, 511523.
Ruggeri, M., Lasalvia, A., Dall'Agnola, R., et al (2000) Development, internal consistency and reliability of the Verona Service Satisfaction Scale – European Version. EPSILON Study 7. British Journal of Psychiatry, 177 (suppl. 39), s41s48.
Sartorius, N. & Kuyken, W. (1994) Translations of health status instruments. In Quality of Life Assessments: International Perspectives (eds Orley, J. & Kuyken, W.), pp. 1932. Berlin, Heidelberg: Springer.
Schene, A. H. (1994) Report of the First International ENMESH Conference: Mental Health Service Evaluation: Developing Reliable Measures. Amsterdam: Department of Psychiatry, Academic Medical Centre, University of Amsterdam.
Schene, A. H. & van Wijngaarden, B. (1992) The Involvement Evaluation Questionnaire. Amsterdam: Department of Psychiatry, Academic Medical Centre, University of Amsterdam.
Schene, A. H., Koeter, M. & van Wijngaarden, B. (1997) Assessing Needs and Cost-Effectiveness of Care for People Severely Disabled by Schizophrenia in the EU: Reliability Protocol. Amsterdam: Department of Psychiatry, Academic Medical Centre, University of Amsterdam.
Shrout, P. E. (1998) Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7, 301307.
Statacorp (1997) Stata Statistical Software Release 5.0. College Station, TX: Stata Corporation.
Streiner, D. & Norman, G. (1995) Health Measurement Scales: A Practical Guide to their Development and Use. Oxford: Oxford University Press.
Tansella, M., (ed.) (1997) Making Rational Mental Health Services. Epidemiologia e Psichiatria Sociale, Monograph Supplement I. Rome: II Pensiero Scientifico Editore.
van Wijngaarden, B., Schene, A. H., Koeter, M., et al (2000) Caregiving in schizophrenia: development, internal consistency and reliability of the Involvement Evaluation Questionnaire – European Version. EPSILON Study 4. British Journal of Psychiatry, 177 (suppl. 39), s21s27.
Vázquez-Barquero, J. L. & Gaite, L. (1996) Focus Group Interview. Final Protocol. Santander: Clinical and Social Psychiatry Research Unit, University of Cantabria.
World Health Organization (1992) Schedules for Clinical Assessment in Neuropsychiatry (ed.-in-chief Wing, J. K.). Geneva: WHO.
Wouters, L. (1998) Amsterdam Alpha-testing Program ALPHA.EXE. Amsterdam: Academic Medical Centre.

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed

Methodology of a multi-site reliability study: EPSILON Study 3

  • Aart H. Schene (a1), Maarten Koeter (a1), Bob van Wijngaarden (a1), Helle Charlotte Knudsen (a2), Morven Leese (a3), Mirella Ruggeri (a4), Ian R. White (a5), José Luis Vázquez-Barquero (a6) and EPSILON Study Group...
Submit a response

eLetters

No eLetters have been published for this article.

×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *