Skip to main content Accessibility help
Hostname: page-component-99c86f546-8r8mm Total loading time: 0.354 Render date: 2021-11-30T13:11:30.938Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

Patient information on schizophrenia on the internet

Published online by Cambridge University Press:  02 January 2018

Nicola J. Kalk
Academic Unit of Psychiatry, University of Bristol, Cotham House, Cotham Hill, Bristol BS6 6JL, email:
David D. Pothier
Department of Otolaryngology, St Michael's Hospital, Bristol
Rights & Permissions[Opens in a new window]


Aims and Methods

The internet is an important source of mental health information. Given variable literacy levels in the general public, patient information websites need to be easily readable to prevent misunderstanding and consequent misinformation about mental health problems being propagated. the aim was to ascertain the readability of websites containing patient information about schizophrenia. Twenty websites containing patient information about schizophrenia generated by Google® were analysed for Flesch Reading Ease and Flesch-Kincaid Grade Level.


According to standardised Flesch Reading Ease classification, 40% of the selected sites were classified as very difficult, 55% as difficult and 5% as fairly difficult. None were considered easy to read. There was a negative correlation of 70.798 (P < 0.001) between Flesch Reading Ease and Flesch-Kincaid Grade Level, which demonstrates the reliability of these results.

Clinical Implications

Easily accessed schizophrenia information websites do not score highly for readability. Those that produce websites should bear readability in mind when writing them in order to construct more readable sites. Ideally, these should be accredited by recognised organisations that evaluate readability. Clinicians should assess website information for readability before recommending them to patients or carers.

Original papers
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (, which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright © Royal College of Psychiatrists, 2008

The internet is used as a source of mental health information by over 10% of the general population and over 20% of those with a history of mental health problems (Reference PowellPowell, 2006). It is one of the top three most reliable sources of information about mental health issues for 24% of users (Reference PowellPowell, 2006). It has the advantages of accessibility and anonymity, and so it is not surprising that people with disorders carrying a stigma (such as mental health problems) are more likely to use the internet when looking for health-related information (Reference Berger, Wagner and BakerBerger et al, 2005). Such people are also more likely to seek medical help having first identified their symptoms using internet sites (Reference Berger, Wagner and BakerBerger et al, 2005).

Schizophrenia is a common mental health illness and a conceptually complex one. Along with drug dependence and alcoholism, it is one of the most stigmatised mental health disorders (Reference Crisp, Gelder and GoddardCrisp et al, 2005). It is also poorly understood by the lay public: when asked what they thought ‘schizophrenia’ meant, 42% of Britons associated it with ‘multiple personality’ or ‘split personality’ (Reference Luty, Fekadu and DhandayudhamLuty et al, 2006). The internet may be a valuable source of information to patients and their carers, as well as the general public, and help them engage with health services.

It is estimated that 45% of the UK population have limited literacy skills, reading at a level below the one necessary to cope with life and work (Organisation for Economic Co-operation and Development, 2003). The PRODIGY study estimated that the average reading age of the UK population was between 9 and 11 years (Reference Wilson, Kenny and ClarkWilson et al, 1998), while people with schizophrenia have significantly poorer language comprehension than controls (Reference Condray, Steinhauer and GoldsteinCondray et al, 1992). Specifically, they have trouble understanding complex syntax (Reference Morice and McNicolMorice & McNicol, 1985). When interviewed in a qualitative study on patient information materials, people with schizophrenia stated they found much of the available written information too difficult to understand (Reference Kingdon, Murray and DoyleKingdon et al, 2004).

Box 1.

Flesch Reading Ease

The output of the Flesch Reading Ease formula is a number from 0 to 100, with a higher score indicating easier reading.

The formula is as follows:

\batchmode \documentclass[fleqn,10pt,legalpaper]{article} \usepackage{amssymb} \usepackage{amsfonts} \usepackage{amsmath} \pagestyle{empty} \begin{document} \[\ 206.835-(1.015{\times}\mathrm{ASL})-(8.46{\times}\mathrm{ASW})\ \] \end{document}


ASL = average sentence length (the number of words divided by the number of sentences)

ASW = average number of syllables per word (the number of syllables divided by the number of words)

Flesch-Kincaid Grade Level

The Flesch-Kincaid Grade Level formula produces an output of the US school grade of literacy required to read the text.

The formula is as follows:

\batchmode \documentclass[fleqn,10pt,legalpaper]{article} \usepackage{amssymb} \usepackage{amsfonts} \usepackage{amsmath} \pagestyle{empty} \begin{document} \[\ (0.39{\times}\mathrm{ASL})+(11.8{\times}\mathrm{ASW})-15.59\ \] \end{document}


ASL = average sentence length (the number of words divided by the number of sentences)

ASW = average number of syllables per word (the number of syllables divided by the number of words).

Adapted from Reference FleschFlesch, 1973.

For these reasons, it is important that information about schizophrenia available on the internet be easily readable. Readability is not a measure of accuracy; rather, it is a measure of the simplicity of syntax and syllabic structure of a piece of text, and does not take into account the complexity of the subject matter presented. Readability of health-related information in other disciplines has been assessed using the Flesch-Kincaid Grade Level and Flesch Reading Ease (Reference FleschFlesch, 1973) score (Box 1). We applied readability criteria to UK-based internet sites with information on schizophrenia for patients.


We analysed the content of easily accessible internet sites about schizophrenia with information for patients. We used validated readability tools and generated descriptive statistics regarding readability.

Data acquisition

We chose schizophrenia for our study as it is a common, conceptually complex mental illness. There are established treatments likely to be consistent across sites. Because people with schizophrenia have problems with reading to advanced levels, clear patient information is of particular importance to them.

By using a single search term ‘schizophrenia’ on the Google search engine (, we identified the first 20 consecutive English language sites on schizophrenia on UK servers, ranked with the PageRank™ tool (Reference Brin and PageBrin & Page, 1998). For the purpose of homogeneity, ‘psychosis’ was not used as a search term; some of the sites accessed using this search term dealt exclusively with drug-induced psychosis. The aim of the study was to analyse the syntax and vocabulary of the written communication about a specific information set, rather than broadly overlapping information sets, therefore the search term had to be as specific as possible. Websites that did not contain patient information (e.g. specialist journal articles) were excluded. The search was filtered for language; HTML coding, internet links and figures were stripped from the original pages to produce a text-only version of the site.

Analysis of readability

Website information was imported into Microsoft Word and analysed using the Flesch Reading Ease and Flesch-Kincaid Grade Level scoring systems (Reference FleschFlesch, 1973). These are widely used validated tools which assess readability based on the syllabic and sentence structure of the text. Complexity of content is not considered. The reading ease scale ranges from 0 to 100, with specific intervals categorised from ‘very easy’ (90-100) to ‘very difficult’ (0-29). The Flesch-Kincaid Grade Level corresponds to the level a person having completed a specific US school grade would be able to read.

Statistical analysis

Flesch Reading Ease and Flesch-Kincaid Grade Level scores for the 20 websites were exported to SPSS 11 for Windows to generate descriptive statistics. The distributions were normal and the mean of each was calculated. The reliability of the assessment using the Flesch Reading Ease and Flesch-Kincaid Grade Level tools was assessed using Pearson's correlation coefficient.


The reading ages of the sites, derived from the Flesch-Kincaid Grade Level, ranged from 9 to 16 years with a mean of 12. None of the sites had a Flesch Reading Ease score of more than 60, the lower limit for ‘plain English’. One paper scored more than 50, which translates as ‘fairly difficult’, 55% were ‘difficult’ and 40% ‘very difficult’. The strong negative correlation between the scoring systems (Pearson=–0.789) supports the reliability of these results. These levels are well above the estimated reading capabilities of the UK population.


Our results show that although easily accessible, internet-based information about schizophrenia is not easily readable. A higher-than-average educational standard in literacy is required to understand the complex vocabulary and syntax used. The information was aimed at the mean reading age of 12, which is above the average UK reading age as estimated by PRODIGY. The sites analysed in this paper came from a broad range of organisations, including patient support groups, public broadcaster-based health information sites, commercial health information sites and professional bodies.

The internet, being accessible and anonymous, is an ideal vehicle for the provision of mental health information both to patients and the public. However, its potential can only be tapped if the information produced is readily understandable. Although the context and methods of explanation can make a text better understandable, readability is a necessary condition for a text to be understood and it does not depend on the context. Given that literacy skills, vocabulary and syntax comprehension in people with schizophrenia are impaired, information on the illness has to be as easily readable as possible. Mental health professionals providing internet-based information should seek to improve its readability, for example by using shorter sentences and words, active rather than passive voice, and simpler, more common words (Reference Horner, Surratt and JuliussonHorner et al, 2000). These can also reduce the chance of cultural bias in understanding (Reference Givaudan, Pick and De VengeurGivaudan et al, 2005).

One way in which the readability of internet sites can be improved and assessed as such is via accreditation by internet health information quality organisations. Sites that have such an accreditation have been found to be significantly more readable than those which do not (Reference PothierPothier, 2005). One of the most often prominent organisations is HONCode ( which is concerned primarily with accountability and the accuracy of the information presented. The Plain English Campaign ( checks readability and has established an accreditation system. None of the websites analysed were accredited by the above organisations or any other such organisations. Mental health professionals can recommend the more easily readable, accredited websites, thus improving the benefit to patients, their carers and families.

The content of internet pages tends to change often. Considering this, clinicians should not rely on lists of recommended websites, but should assess each website themselves before recommending it to patients and carers. For example, a member of the multidisciplinary team could discuss the internet information with the service user to check their understanding.


Easily-accessed websites containing patient information about schizophrenia do not score well when tested for readability. This has implications for service users and the general public. Mental health professionals involved in the production of websites with medical information should adapt them to the reading skills of their potential readers.

Declaration of interest



Berger, M., Wagner, T. H. & Baker, L. C. (2005) Internet use and stigmatised illness. Social Science and Medicine, 61, 18211827.CrossRefGoogle Scholar
Brin, S. & Page, L. (1998) The anatomy of a large-scale hypertexual web search engine. Computer Networks and ISDN Systems, 30, 107117.CrossRefGoogle Scholar
Condray, R., Steinhauer, S. R. & Goldstein, G. (1992) Language comprehension in schizophrenics and their brothers. Biological Psychiatry, 32, 790802.CrossRefGoogle ScholarPubMed
Crisp, A., Gelder, M., Goddard, E., et al (2005) Stigmatisation of people with mental illnesses: a follow-up study within the Changing Minds campaign of the Royal College of Psychiatrists. World Psychiatry, 4, 106113.Google Scholar
Flesch, R. (1973) The Art of Readable Writing. Harper & Row.Google Scholar
Givaudan, M., Pick, S., De Vengeur, M. T. T., et al (2005) Development of an Instrument to test the Cultural Adequacy of Health Related Written Material for Latinos in the USA. IMIFAP.Google Scholar
Horner, S., Surratt, D. & Juliusson, S. (2000) Improving readability of patient education materials. Journal of Community Health Nursing, 17, 1523.CrossRefGoogle ScholarPubMed
Kingdon, D., Murray, P. & Doyle, E. (2004) Reading about self-help for schizophrenia. Psychiatric Bulletin, 28, 349351.CrossRefGoogle Scholar
Luty, J., Fekadu, D. & Dhandayudham, A. (2006) Understanding of the term ‘schizophrenia’ by the British public. World Psychiatry, 5, 177178.Google ScholarPubMed
Morice, R. & McNicol, D. (1985) The comprehension and production of complex syntax in schizophrenia. Cortex, 21, 567580.CrossRefGoogle Scholar
Organisation for Economic Co-Operation and Development (2003) PISA 2003 Assessment Framework: Mathematics, Reading, Science and Problem Solving Knowledge and Skills. OECD.Google Scholar
Pothier, D. (2005) Patients and the internet: are websites on glue ear readable? Clinical Otolaryngology, 30, 566.CrossRefGoogle ScholarPubMed
Powell, J. (2006) Internet information-seeking in mental health. Population survey. British Journal of Psychiatry, 189, 273277.CrossRefGoogle ScholarPubMed
Wilson, R., Kenny, T., Clark, J., et al (1998) PIL's Project Summary Report: Ensuring the Readability and Understandability and Efficacy of Patient Information Leaflets. Sowerby Health Centre for Health Informatics at Newcastle.Google Scholar
Submit a response


No eLetters have been published for this article.
You have Access
Open access
Cited by

Send article to Kindle

To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Patient information on schizophrenia on the internet
Available formats

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Patient information on schizophrenia on the internet
Available formats

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Patient information on schizophrenia on the internet
Available formats

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *