Hostname: page-component-78c5997874-g7gxr Total loading time: 0 Render date: 2024-10-31T11:39:32.713Z Has data issue: false hasContentIssue false
  • English
  • Français

Interobserver reliability of measures of the Welfare Quality® animal welfare assessment protocol for sows and piglets

Published online by Cambridge University Press:  01 January 2023

L Friedrich ,
J Krieter ,
N Kemper  and
I Czycholl
L Friedrich*
Affiliation:
Institute of Animal Breeding and Husbandry, Christian-Albrechts-University, Olshausenstr 40, D-24098 Kiel, Germany
J Krieter
Affiliation:
Institute of Animal Breeding and Husbandry, Christian-Albrechts-University, Olshausenstr 40, D-24098 Kiel, Germany
N Kemper
Affiliation:
Institute for Animal Hygiene, Animal Welfare and Farm Animal Behaviour, University of Veterinary Medicine Hannover, Foundation, Bischofsholer Damm 15, D-30173 Hannover, Germany
I Czycholl
Affiliation:
Institute of Animal Breeding and Husbandry, Christian-Albrechts-University, Olshausenstr 40, D-24098 Kiel, Germany
*
* Contact for correspondence: lfriedrich@tierzucht.uni-kiel.de
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

The aim of this study was to assess the interobserver reliability of the measures forming the Welfare Quality® animal welfare assessment protocol for sows and piglets. The study was carried out at nine farms in Northern Germany. Two trained observers evaluated identical animals simultaneously but independently in 40 joint farm visits. Interobserver reliability was calculated at individual animal level using Cohen's kappa, weighted kappa and the prevalence-adjusted, bias-adjusted kappa (PABAK) and at farm level using Spearman's rank correlation coefficient (RS), the intraclass correlation coefficient (ICC), smallest detectable change (SDC) and limits of agreement (LoA). While a direct comparison of the adjectives of the qualitative behaviour assessment showed poor interobserver reliability, a Principal Component Analysis detected good interobserver reliability. The assessment of social and exploratory behaviours showed acceptable interobserver reliability, while the assessment of stereotypies displayed good interobserver reliability. The human-animal relationship test showed only poor interobserver reliability at individual animal and farm levels. In most cases, measures of health and physical state assessed in sows and piglets exhibited acceptable or good interobserver reliability. In conclusion, after some measures are revised, particularly those examining the human-animal relationship, the Welfare Quality® protocol for sows and piglets will represent a reliable approach in terms of interobserver reliability to assess the welfare of sows and piglets.

Keywords

animal-basedanimal welfareinterobserver reliabilitypigsowsWelfare Quality®
Type
Research Article
Information
Animal Welfare , Volume 29 , Issue 3 , August 2020 , pp. 323 - 337
Copyright
© 2020 Universities Federation for Animal Welfare

References

AWIN 2015 AWIN welfare assessment protocol for horses. http://dx.doi.org/10.13130/AWIN_horses_2015CrossRefGoogle Scholar
Bland, MJ and Altman, DG 1986 Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 327: 307–310. https://doi.org/10.1016/S0140-6736(86)90837-8CrossRefGoogle Scholar
Blokhuis, HJ 2008 International cooperation in animal welfare. The Welfare Quality® project. Acta Agriculturae Scandinavica, Section A 50(S1): 10. https://doi.org/10.1186/1751-0147-50-S1-S10Google Scholar
Bokkers, EAM, de Vries, M, Antonissen, ICMA and de Boer, IJM 2012 Inter- and intra-observer reliability of experienced and inexperienced observers for the Qualitative Behaviour Assessment in dairy cattle. Animal Welfare 21: 307–318. https://doi.org/10.7120/09627286.21.3.307CrossRefGoogle Scholar
Botreau, R, Veissier, I, Butterworth, A, Bracke, M and Keeling, LJ 2007 Definition of criteria for overall assessment of animal welfare. Animal Welfare 16: 225–228CrossRefGoogle Scholar
Brennan, RL and Prediger, DJ 1981 Coefficient kappa: Some uses, misuses and alternatives. Educational and Psychological Measurement 41: 687–699. https://doi.org/10.1177/001316448104100307CrossRefGoogle Scholar
Brenner, H and Kliebsch, U 1996 Dependence of weighted kappa coefficients on the number of categories. Epidemiology 7: 199–202. https://doi.org/10.1097/00001648-199603000-00016CrossRefGoogle ScholarPubMed
Burn, CC, Pritchard, JC and Whay, HR 2009 Reliability of a welfare assessment for working horses and donkeys in developing countries. Animal Welfare 18: 177–187CrossRefGoogle Scholar
Byrt, T, Bishop, J and Carlin, JB 1993 Bias, prevalence and kappa. Journal of Clinical Epidemiology 46: 423–429. https://doi.org/10.1016/0895-4356(93)90018-VCrossRefGoogle ScholarPubMed
Candland, DK 1993 Feral Children and Clever Children: Reflections on Human Nature. Oxford University Press: New York, USAGoogle Scholar
Cohen, J 1960 A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20: 37–46. https://doi.org/10.1177/001316446002000104CrossRefGoogle Scholar
Cohen, J 1968 Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin 70: 213–220. https://doi.org/10.1037/h0026256CrossRefGoogle ScholarPubMed
Czycholl, I, Grosse Beilage, E, Henning, C and Krieter, J 2017 Reliability of the qualitative behavior assessment as included in the Welfare Quality® assessment protocol for growing pigs. Journal of Animal Science 95: 3445–3454. https://doi.org/10.2527/jas.2017.1525Google ScholarPubMed
Czycholl, I, Kniese, C, Büttner, K, Grosse Beilage, E, Schrader, L and Krieter, J 2016a Interobserver reliability of the Welfare Quality® animal welfare assessment protocol for growing pigs. SpringerPlus 5: 1114. https://doi.org/10.1186/s40064-016-2785-1CrossRefGoogle ScholarPubMed
Czycholl, I, Kniese, C, Büttner, K, Grosse Beilage, E, Schrader, L and Krieter, J 2016b Test-retest reliability of the Welfare Quality® animal welfare assessment protocol for growing pigs. Animal Welfare 25: 447–459. https://doi.org/10.7120/09627286.25.4.447CrossRefGoogle Scholar
Dalmau, A, Geverink, NA, van Nuffel, A, van Steenbergen, L, van Reenen, K, Hautekiet, V, Vermeulen, K, Velarde, A and Tuyttens, FAM 2010 Repeatability of lameness, fear and slipping scores to assess animal welfare upon arrival in pig slaugh-terhouses. Animal 4: 804–809. https://doi.org/10.1017/S1751731110000066CrossRefGoogle Scholar
de Passillé, AM and Rushen, J 2005 Can we measure human-animal interactions in on-farm animal welfare assessment? Some unresolved issues. Applied Animal Behaviour Science 92: 193–209. https://doi.org/10.1016/j.applanim.2005.05.006CrossRefGoogle Scholar
de Vet, HC, Terwee, CB, Knol, DL and Bouter, LM 2006 When to use agreement versus reliability measures. Journal of Clinical Epidemiology 59: 1033–1039. https://doi.org/10.1016/j.jclinepi.2005.10.015CrossRefGoogle ScholarPubMed
Docking, CM, van de Weerd, HA, Day, J and Edwards, SA 2008 The influence of age on the use of potential enrichment objects and synchronisation of behaviour of pigs. Applied Animal Behaviour Science 11: 244–257. https://doi.org/10.1016/j.applanim.2007.05.004CrossRefGoogle Scholar
Eurobarometer 2016 Attitudes of Europeans towards Animal Welfare: Special Eurobarometer Report 442. Eurobarometer: Brussels, BelgiumGoogle Scholar
European Commission (EU) 2007 Council Regulation (EC) No 834/2007 of 28 June 2007 on organic production and labelling of organic products and repealing Regulation (EEC) No 2092/91. EC: Brussels, BelgiumGoogle Scholar
Feinstein, AR and Cicchetti, DV 1990 High agreement but low kappa: I the problems of two paradoxes. Journal of Clinical Epidemiology 43: 543–549. https://doi.org/10.1016/0895-4356(90)90158-LCrossRefGoogle ScholarPubMed
Forkman, B and Keeling, LJ 2009 Welfare Quality® Reports: Assessment of animal welfare measures for sows, piglets and fattening pigs. Cardiff University: Cardiff, UKGoogle Scholar
Friedrich, L, Krieter, J, Kemper, N and Czycholl, I 2019a Test-retest reliability of the Welfare Quality® animal welfare assessment protocol for sows and piglets, Part 1. Assessment of the welfare principle of Appropriate Behaviour’. Animals 9: 398. https://doi.org/10.3390/ani9070398CrossRefGoogle Scholar
Friedrich, L, Krieter, J, Kemper, N and Czycholl, I 2019b Test–retest reliability of the Welfare Quality® Assessment pro-tocol for pigs applied to sows and piglets, Part 2. Assessment of the principles good feeding, good housing, and good health. Journal of Animal Science 97: 1143–1157. https://doi.org/10.1093/jas/skz018CrossRefGoogle Scholar
Friedrich, L, Krieter, J, Kemper, N and Czycholl, I 2019c Frothy saliva: A novel indicator to assess stereotypies in sows? Applied Animal Behaviour Science 222: 104897. https://doi.org/10.1016/j.applanim.2019.104897CrossRefGoogle Scholar
Gauthier, TD 2001 Detecting trends using Spearman’s Rank cor-relation coefficient. Environmental Forensics 2: 359–362. https://doi.org/10.1006/enfo.2001.0061CrossRefGoogle Scholar
German Animal Welfare Act 2006 Tierschutzgesetz in der Fassung der Bekanntmachung vom 18 Mai 2006 (BGBl I S 1206,1313), das zuletzt durch Artikel 141 des Gesetzes vom 29 März 2017(BGBl I S 626) geändert worden ist. https://www.gesetze-im-inter-net.de/tierschg/BJNR012770972.htmlGoogle Scholar
German Order for the Protection of Production Animals used for Farming Purposes and other Animals kept for the Production of Animal Products 2006 Tierschutz-Nutztierhaltungsverordnung in der Fassung der Bekanntmachung vom 22 August 2006 (BGBl I S 2043), die durch Artikel 3 Absatz 2 des Gesetzes vom 30 Juni 2017 (BGBl I S 2147) geändert worden ist. https://www.gesetze-im-internet.de/tierschnutztv/BJNR275800001.htmlGoogle Scholar
Geverink, N, Meuleman, M, van Nuffel, A, van Steenbergen, L, Hautekiet, V and Vermeulen, K 2009 Repeatability of a lameness score measured on farm. In: Forkman B and Keeling L (eds) Welfare Quality® Reports: Assessment of Animal Welfare Measures for Sows, Piglets and Fattening Pigs pp 73–78. Cardiff University Press: Cardiff, UKGoogle Scholar
Gibbons, J, Vasseur, E, Rushen, J and de Passillé, AM 2012 A training programme to ensure high repeatability of injury scoring of dairy cows. Animal Welfare 21: 379–388. https://doi.org/10.7120/09627286.21.3.379CrossRefGoogle Scholar
Held, S, Mendl, M, Laughlin, K and Byrne, RW 2002 Cognition studies with pigs: Livestock cognition and its implication for pro-duction. Journal of Animal Science 80: E10–E17Google Scholar
Hewetson, M, Christley, TM, Hunt, ID and Voute, LC 2006 Investigations of the reliability of the observational gait analysis for the assessment of lameness in horses. Veterinary Record 158: 852–858. https://doi.org/10.1136/vr.158.25.852CrossRefGoogle ScholarPubMed
Hoehler, FK 2000 Bias and prevalence effects on kappa viewed in terms of sensitivity and specificity. Journal of Clinical Epidemiology 53: 499–503. https://doi.org/10.1016/S0895-4356(99)00174-2CrossRefGoogle ScholarPubMed
Knierim, U, Lentfer, T, Staack, M and Wemelsfelder, F 2007 How reliable is a qualitative behavioural assessment of laying hens? KTBL SCHRIFT 461: 135Google Scholar
Landis, JR and Koch, GG 1977 An application of hierarchical Kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 33: 362–374. https://doi.org/10.2307/2529786CrossRefGoogle ScholarPubMed
Lyons, C, Bruce, JM, Fowler, VR and English, PR 1995 A com-parison of productivity and welfare of growing pigs in four inten-sive systems. Livestock Production Science 43: 265–274. https://doi.org/10.1016/0301-6226(95)00050-UCrossRefGoogle Scholar
Martin, P and Bateson, P 2007 Measuring Behaviour: An Introductory Guide. University of Cambridge: Cambridge, UK. https://doi.org/10.1017/CBO9780511810893CrossRefGoogle Scholar
McGraw, KO and Wong, SP 1996 Forming inferences about some intraclass correlation coefficients. Psychological Methods 1: 30–46. https://doi.org/10.1037/1082-989X.1.1.30CrossRefGoogle Scholar
Mullan, S, Edwards, SA, Butterworth, A, Whay, HR and Main, DCJ 2011 Inter-observer reliability testing of pig welfare outcome measures proposed for inclusion within farm assurance schemes. The Veterinary Journal 190: 100–109. https://doi.org/10.1016/j.tvjl.2011.01.012CrossRefGoogle ScholarPubMed
Munsterhjelm, C, Heinonen, M and Valros, A 2015 Application of the Welfare Quality® animal welfare assessment system in Finnish pig production, part II: Associations between animal-based and environmental measures of welfare. Animal Welfare 24: 161–172. https://doi.org/10.7120/09627286.24.2.161CrossRefGoogle Scholar
Napolitano, F, de Rosa, G, Braghieri, A, Grasso, F, Bordi, A and Wemelsfelder, F 2008 The qualitative assessment of responsiveness to environmental challenge in horses and ponies. Applied Animal Behaviour Science 109: 342–354. https://doi.org/10.1016/j.applanim.2007.03.009CrossRefGoogle Scholar
Phythian, CJ, Cripps, PJ, Michalopoulou, E, Jones, PH, Grove-White, D, Clarkson, MJ, Winter, AC, Stubbings, LA and Duncan, JS 2012 Reliability of indicators of sheep welfare assessed by a group observation method. The Veterinary Journal 193: 257–263. https://doi.org/10.1016/j.tvjl.2011.12.006CrossRefGoogle ScholarPubMed
Phythian, CJ, Michalopoulou, E, Cripps, PJ, Duncan, JS and Wemelsfelder, F 2016 On-farm qualitative behaviour assess-ment in sheep: Repeated measurements across time, and association with physical indicators of flock health and welfare. Applied Animal Behaviour Science 175: 23–31. https://doi.org/10.1016/j.applanim.2015.11.013CrossRefGoogle Scholar
Phythian, CJ, Toft, N, Cripps, PJ, Michalopoulou, E, Winter, AC, Jones, PH, Grove-White, D and Duncan, JS 2013a Inter-observer agreement, diagnostic sensitivity and specificity of ani-mal-based indicators of young lamb welfare. Animal 7: 1182–1190. https://doi.org/10.1017/S1751731113000487CrossRefGoogle Scholar
Plesch, G, Broerkens, N, Laister, S, Winckler, C and Knierim, U 2010 Reliability and feasibility of selected measures concerning resting behaviour for the on-farm welfare assessment in dairy cows. Applied Animal Behaviour Science 126: 19–26. https://doi.org/10.1016/j.applanim.2010.05.003CrossRefGoogle Scholar
SAS® Institute Inc 2008 User’s Guide (release 9.4). SAS: Cary, USAGoogle Scholar
Scott, K, Binnendijk, GP, Edwards, SA, Guy, JH, Kiezebrink, MC and Vermeer, HM 2009 Preliminary evaluation of a proto-type welfare monitoring system for sows and piglets (Welfare Quality® project). Animal Welfare 18: 441–449CrossRefGoogle Scholar
Shrout, PE and Fleiss, JL 1979 Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin 86: 420–428. https://doi.org/10.1037/0033-2909.86.2.420CrossRefGoogle ScholarPubMed
Temple, D, Manteca, X, Dalmau, A and Velarde, A 2013 Assessment of test-retest reliability of animal-based measures on growing pig farms. Livestock Science 151: 35–45. https://doi.org/10.1016/j.livsci.2012.10.012CrossRefGoogle Scholar
Temple, D, Manteca, X, Velarde, A and Dalmau, A 2011 Assessment of animal welfare through behavioural parameters in Iberian pigs in intensive and extensive conditions. Applied Animal Behaviour Science 131: 29–39. https://doi.org/10.1016/j.applanim.2011.01.013CrossRefGoogle Scholar
Thorslund, CAH, Aaslyng, MD and Lassen, J 2017 Perceived importance and responsibility for market-driven pig welfare: Literature review. Meat Science 125: 37–45. https://doi.org/10.1016/j.meatsci.2016.11.008CrossRefGoogle ScholarPubMed
Tuyttens, FAM, de Graaf, S, Heerkens, JLT, Jacobs, L, Nalon, E, Ott, S, Stadig, L, van Laer, E and Ampe, B 2014 Observer bias in animal behaviour research: Can we believe what we score, if we score what we believe? Animal Behaviour 90: 273–280. https://doi.org/10.1016/j.anbehav.2014.02.007CrossRefGoogle Scholar
van Loo, EJ, Caputo, V, Nayga, RM and Verbeke, W 2014 Consumers’ valuation of sustainability labels on meat. Food Policy 49: 137–150. https://doi.org/10.1016/j.foodpol.2014.07.002CrossRefGoogle Scholar
Veissier, I, Winckler, C, Velarde, A, Butterworth, A, Dalmau, A and Keeling, LJ 2013 Development of welfare meas-ures and protocols for the collection of data on farms or at slaughter. In: Blokhuis H, Miele M, Veissier I and Jones B (eds) Improving Farm Animal Welfare. Science and Society Working Together: The Welfare Quality Approach pp 115–141. Wageningen Academic Publishers: Wageningen, The Netherlands. https://doi.org/10.3920/978-90-8686-770-7_6CrossRefGoogle Scholar
Velarde, A and Geers, R 2007 On-farm Monitoring of Pig Welfare. Wageningen Academic Publishers: Wageningen, The Netherlands. https://doi.org/10.3920/978-90-8686-591-8CrossRefGoogle Scholar
Walker, J, Dal, A, Waran, N, Clarke, N, Farnworth, M and Wemelsfelder, F 2010 The assessment of emotional expression in dogs using a Free Choice Profiling methodology. Animal Welfare 19: 75–84CrossRefGoogle Scholar
Walter, SD, Eliasziw, M and Donner, A 1998 Sample size and optimal designs for reliability studies. Statistics in Medicine 17: 101–110. https://doi.org/10.1002/(SICI)1097-0258(19980115)17:1<101::AID-SIM727>3.0.CO;2-E3.0.CO;2-E>CrossRefGoogle ScholarPubMed
Webster, J 2005 The assessment and implementation of animal welfare: theory into practice. Scientific and Technical Review of the Office International des Epizooties 24: 723–734. https://doi.org/10.20506/rst.24.2.1602CrossRefGoogle Scholar
Welfare Quality® 2009 Welfare Quality® Assessment Protocol for Pigs. Wageningen Academic Publishers: Wageningen, The NetherlandsGoogle Scholar
Wemelsfelder, F, Hunter, TEA, Mendl, MT and Lawrence, AB 2001 Assessing the ‘whole animal’. A free choice profiling approach. Animal Behaviour 62: 209–220. https://doi.org/10.1006/anbe.2001.1741CrossRefGoogle Scholar
Wemelsfelder, F, Hunter, EA, Mendl, MT, Lawrence, AB, Hunter, E, Mendl, M and Lawrence, A 2000 The spontaneous qualitative assessment of behavioural expressions in pigs: first explorations of a novel methodology for integrative animal wel-fare measurement. Applied Animal Behaviour Science 67: 193–215. https://doi.org/10.1016/S0168-1591(99)00093-3CrossRefGoogle Scholar
Wemelsfelder, F and Millard, F 2009 Qualitative behaviour assessment. In: Forkman B and Keeling LJ (eds) Welfare Quality®Reports: Assessment of Animal Welfare Measures for Dairy Cattle, Beef Bulls and Veal Calves. Cardiff University Press: Cardiff, UKGoogle Scholar
Wemelsfelder, F, Nevison, I and Lawrence, AB 2009 The effect of perceived environmental background on qualitative assessments of pig behaviour. Animal Behaviour 78: 477–484. https://doi.org/10.1016/j.anbehav.2009.06.005CrossRefGoogle Scholar
Windschnurer, I, Schmied, C, Boivin, X and Waiblinger, S 2008 Reliability and inter-test relationship of tests for on-farm assessment of dairy cows’ relationship to humans. Applied Animal Behaviour Science 114: 37–53. https://doi.org/10.1016/j.applan-im.2008.01.017CrossRefGoogle Scholar
Wirtz, M and Caspar, F 2002 Beurteilerübereinstimmung und Beurteilerreliabilität. Methoden zur Bestimmung und Verbesserung der Zuverlässigkeit von Einschätzungen mittels Kategoriensystemen und Ratingskalen. Hogrefe Verlag für Psychologie: Göttingen, Germany. [Title translation: Observer agreement and observer reliability. Methods for determining and improving the reliability of assessments using category systems and rating scales]Google Scholar
Supplementary material: File

Friedrich et al. supplementary material
Download undefined(File)
File 104.2 KB
Supplementary material: PDF

Download undefined(PDF)
PDF 104.2 KB