Skip to main content Accessibility help

Reliability of perceptions of voice quality: evidence from a problem asthma clinic population

  • C Sellars (a1), A E Stanton (a2) (a3), A McConnachie (a4), C P Dunnet (a1), L M Chapman (a1), C E Bucknall (a2) and K MacKenzie (a5)...



Methods of perceptual voice evaluation have yet to achieve satisfactory consistency; complete acceptance of a recognised clinical protocol is still some way off.

Materials and methods:

Three speech and language therapists rated the voices of 43 patients attending the problem asthma clinic of a teaching hospital, according to the grade-roughness-breathiness-asthenicity-strain (GRBAS) scale and other perceptual categories.

Results and analysis:

Use of the GRBAS scale achieved only a 64.7 per cent inter-rater reliability and a 69.6 per cent intra-rater reliability for the grade component. One rater achieved a higher degree of consistency. Improved concordance on the GRBAS scale was observed for subjects with laryngeal abnormalities. Raters failed to reach any useful level of agreement in the other categories employed, except for perceived gender.


These results should sound a note of caution regarding routine adoption of the GRBAS scale for characterising voice quality for clinical purposes. The importance of training and the use of perceptual anchors for reliable perceptual rating need to be further investigated.


Corresponding author

Address for correspondence: Mr Cameron Sellars, Dept of Speech and Language Therapy, Glasgow Royal Infirmary, Castle St, Glasgow G4 0SF, Scotland, UK. Fax: +44 (0)141 211 4821 E-mail:


Hide All
1 Blood, GW, Mahan, BW, Hyman, M. Judging personality and appearance from voice disorders. J Commun Disord 1979;12:63–7
2 DeGregorio, NJ, Polow, NG. Effect of teacher training sessions on listener perception of voice disorders. Lang Speech Hear Serv Sch 1985;16:25–8
3 Lallh, AK, Rochet, AP. The effect of information on listeners' attitudes toward speakers with voice or resonance disorders. J Speech Lang Hear Res 2000;43:782–95
4 Lass, NJ, Ruscello, DM, Bradshaw, KH, Blankenship, BL. Adolescents' perceptions of normal and voice-disordered children. J Commun Disord 1991;24:267–74
5 Ruscello, DM, Lass, NJ, Podbesek, J. Listeners' perceptions of normal and voice-disordered children. Folia Phoniatr 1988;40:290–6
6 Davis, CN, Harris, TB. Teachers' ability to accurately identify disordered voices. Lang Speech Hear Serv Sch 1992;23:136–40
7 Bonet, M, Casan, P. Evaluation of dysphonia in a children's choir. Folia Phoniatr Logop 1994;46:2734
8 Behrman, A. Common practices of voice therapists in the evaluation of patients. J Voice 2005;19:454–69
9 Bhuta, T, Patrick, L, Garnett, JD. Perceptual evaluation of voice quality and its correlation with acoustic measurements. J Voice 2004;18:299304
10 Carding, P, Carlson, E, Epstein, R, Mathieson, L, Shewell, C. Formal perceptual evaluation of voice quality in the United Kingdom. Logoped Phoniatr Vocol 2000;25:133–8
11 Eadie, TL, Doyle, PC. Classification of dysphonic voice: acoustic and auditory-perceptual measures. J Voice 2005;19:114
12 Speyer, R, Wieneke, GH, Dejonckere, PH. Documentation of progress in voice therapy: perceptual, acoustic, and laryngostroboscopic findings pretherapy and posttherapy. J Voice 2004;18:325–40
13 Webb, AL, Carding, PN, Deary, IJ, MacKenzie, K, Steen, N, Wilson, JA. The reliability of three perceptual evaluation scales for dysphonia. Eur Arch Otorhinolaryngol 2004;261:429–34
14 Kreiman, J, Gerratt, BR, Precoda, K, Berke, GS. Individual differences in voice quality perception. J Speech Hear Res 1992;35:512–20
15 Muñoz, J, Mendoza, E, Fresneda, MD, Carballo, G, Lopez, P. Acoustic and perceptual indicators of normal and pathological voice. Folia Phoniatr Logop 2003;55:102–14
16 Kreiman, J, Gerratt, BR. Sources of listener disagreement in voice quality assessment. J Acoust Soc Am 2000;108:1867–76
17 Hirano, M. Clinical Examination of Voice, 1st edn. Berlin, Heidelberg, New York: Springer, 1981
18 De Bodt, MS, Van de Heyning, PH, Wuyts, FL, Lambrechts, L. The perceptual evaluation of voice disorders. Acta Otorhinolaryngol Belg 1996;50:283–91
19 De Bodt, MS, Wuyts, FL, Van de Heyning, PH, Croux, C. Test-retest study of the GRBAS scale: influence of experience and professional background on perceptual rating of voice quality. J Voice 1997;11:7480
20 Dejonckere, PH, Obbens, C, de Moor, GM, Wieneke, GH. Perceptual evaluation of dysphonia: reliability and relevance. Folia Phoniatr Logop 1993;45:7683
21 Dejonckere, PH, Remacle, M, Fresnel-Elbaz, E, Woisard, V, Crevier-Buchman, L, Millet, B. Differentiated perceptual evaluation of pathological voice quality: reliability and correlations with acoustic measurements. Rev Laryngol Otol Rhinol 1996;117:219–24
22 Millet, B, Dejonckere, PH. What determines the differences in perceptual rating of dysphonia between experienced raters? Folia Phoniatr Logop 1998;50:305–10
23 Karnell, MP, Melton, SD, Childes, JM, Coleman, TC, Dailey, SA, Hoffman, HT. Reliability of clinician-based (GRBAS and CAPE-V) and patient-based (V-RQOL and IPVI) documentation of voice disorders. J Voice 2007;21:576–90
24 Piccirillo, JF, Painter, C, Haiduk, A, Fuller, D, Fredrickson, JM. Assessment of two objective voice function indices. Ann Otol Rhinol Laryngol 1998;107:396400
25 Lee, M, Drinnan, M, Carding, P. The reliability and validity of patient self-rating of their own voice quality. Clin Otolaryngol Allied Sci 2005;30:357–61
26 Ma, EP, Yiu, EM. Multiparametric evaluation of dysphonic severity. J Voice 2006;20:380–90
27 Murry, T, Medrado, R, Hogikyan, ND, Aviv, JE. The relationship between ratings of voice quality and quality of life measures. J Voice 2004;18:183192
28 Baker, BM, Baker, CD, Le, HT. Vocal quality, articulation and audiological characteristics of children and young adults with diagnosed allergies. Ann Otol Rhinol Laryngol 1982;91:277–80
29 Barnes, P. Corticosteroids. In: O'Byrne, PM, Thomson, NC, eds. Manual of Asthma Management. London: WB Saunders, 2000;173–96
30 Williamson, IJ, Matusiewicz, SP, Brown, PH, Greening, AP, Crompton, GK. Frequency of voice problems and cough in patients using pressurized aerosol inhaled steroid preparations. Eur Respir J 1995;8:590–2
31 Crompton, GK, Sanderson, R, Dewar, MH, Matusiewicz, SP, Ning, AC, Jamieson, AH et al. Comparison of pulmicort pMDI plus nebuhaler and pulmicort turbuhaler in asthmatic patients with dysphonia. Respir Med 2000;94:448–53
32 Dogan, M, Eryuksel, E, Kocak, I, Celikel, T, Sehitoglu, MA. Subjective and objective evaluation of voice quality in patients with asthma. J Voice 2007;21:224–30
33 Gallivan, GJ, Gallivan, KH, Gallivan, HK. Inhaled corticosteroids: hazardous effects on voice – an update. J Voice 2007;21:101–11
34 Lavy, JA, Wood, G, Rubin, JS, Harries, M. Dysphonia associated with inhaled steroids. J Voice 2000;14:581–8
35 Stanton, AE, Johnson, MK, Carter, R, MacKenzie, K, Bucknall, CE. Physiological evaluation of the upper airway in a problem asthma clinic. Eur Respir J 2004;24:P1712 Viewed 6/01/09
36 Stanton, AE, MacKenzie, K, Carter, R, Bucknall, CE. The spectrum of upper airway problems in a problem asthma clinic – the role of the larynx. Eur Respir J 2004;24:P1711 Viewed 6/01/09
37 Stanton, AE, McGarry, GW, Carter, R, Bucknall, CE. The spectrum of upper airway problems in a problem asthma clinic – the role of the nose. Eur Respir J 2004;24:1710 Viewed 6/01/09
38 Stanton, AE, Sellars, C, MacKenzie, K, McConnachie, A, Bucknall, CE. Perceived vocal morbidity in a problem asthma clinic. J Laryngol Otol 2009;123:96102
39 Wilson, JA, Webb, A, Carding, PN, Steen, IN, MacKenzie, K, Deary, IJ. The voice symptom scale (VoiSS) and the vocal handicap index (VHI): a comparison of structure and content. Clin Otolaryngol Allied Sci 2004;29:169–74
40 Fairbanks, G. Voice and Articulation Drillbook, 2nd edn. New York: Harper and Brothers, 1960
41 Dejonckere, PH, Remacle, M, Fresnel-Elbaz, E, Woisard, V, Crevier, L, Millet, B. Reliability and clinical relevance of perceptual evaluation of pathological voices. Rev Laryngol Otol Rhinol 1998;119:247–8
42 Gleser, GC, Cronbach, LJ, Rajaratnam, N. Generalizability of scores influences by multiple sources of variance. Psychometrika 1965;30:395418
43 Efron, B, Tibshirani, R. An Introduction to the Bootstrap. London: Chapman and Hall, 1993
44 Dedivitis, RA, Barros, AP, Queija, DS, Alexandre, JCM, Rezende, WTM, Crazza, VR et al. Interobserver perceptual analysis of smoker's voice. Clin Otolaryngol Allied Sci 2004;29:124–7
45 Carding, P, Carlson, E, Epstein, R, Mathieson, L, Shewell, C. Re: evaluation of voice quality. Int J Lang Commun Disord 2001;36:127–34
46 Dejonckere, PH. Perceptual and laboratory assessment of dysphonia. Otolaryngol Clin North Am 2000;33:731–50
47 Leslie, P, Drinnan, MJ, Finn, P, Ford, GA, Wilson, JA. Reliability and validity of cervical auscultation: a controlled comparison using videofluoroscopy. Dysphagia 2004;19:231–40
48 Stroud, AE, Lawrie, BW, Wiles, CM. Inter- and intra-rater reliability of cervical auscultation to detect aspiration in patients with dysphagia. Clin Rehabil 2002;16:640–5
49 Wolfe, VI, Martin, DP, Palmer, CI. Perception of dysphonic voice quality by naive listeners. J Speech Lang Hear Res 2000;43:697705
50 Eadie, TL, Baylor, CR. The effect of perceptual training on inexperienced listeners' judgments of dysphonic voice. J Voice 2006;20:527–44
51 Awan, SN, Lawson, LL. The effect of anchor modality on the reliability of vocal severity ratings. J Voice 2008 Viewed 16/10/08
52 Pribuisiene, R, Uloza, V, Kupcinskas, L, Jonaitis, L. Perceptual and acoustic characteristics of voice changes in reflux laryngitis patients. J Voice 2006;20:128–36
53 Bele, IV. Reliability in perceptual analysis of voice quality. J Voice 2005;19:555–73
54 Chan, KMK, Yiu, EM. A comparison of two perceptual voice evaluation training programs for naive listeners. J Voice 2006;20:229–41
55 Kreiman, J, Gerratt, B. Measuring vocal quality. In: Kent, RD, Ball, MJ, eds. Voice Quality Measurement, 1st edn. San Diego: Singular, 2000:73101
56 Shrivastav, R, Sapienza, CM, Nandur, V. Application of psychometric theory to the measurement of voice quality using rating scales. J Speech Lang Hear Res 2005;48:323–35
57 Shrivastav, R. The use of an auditory model in predicting perceptual ratings of breathy voice quality. J Voice 2003;17:502–12
58 Shrivastav, R. Multidimensional scaling of breathy voice quality: individual differences in perception. J Voice 2006;20:211–22
59 Zraick, RI, Wendel, K, Smith-Olinde, L. The effect of speaking task on perceptual judgment of the severity of dysphonic voice. J Voice 2005;19:574–81
60 Behrman, A, Sulica, L, He, T. Factors predicting patient perception of dysphonia caused by benign vocal fold lesions. Laryngoscope 2004;114:1693–700
61 Behrman, A. Evidence-based treatment of paralytic dysphonia: making sense of outcomes and efficacy data. Otolaryngol Clin North Am 2004;37:75104
62 Deary, IJ, Wilson, JA, Carding, PN, Mackenzie, K. The dysphonic voice heard by me, you and it: differential associations with personality and psychological distress. Clin Otolaryngol Allied Sci 2003;28:374–8


Reliability of perceptions of voice quality: evidence from a problem asthma clinic population

  • C Sellars (a1), A E Stanton (a2) (a3), A McConnachie (a4), C P Dunnet (a1), L M Chapman (a1), C E Bucknall (a2) and K MacKenzie (a5)...


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed