Domains of nutrition literacy and the engagement in dietary behaviour scale

Health literacy is ‘the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions’⁽ Reference Nielsen-Bohlman, Panzer and Kindig ^{2 )}. Health literacy is claimed to be a stronger predictor of health than age, income, employment, education and cultural background^{( 3 )}.

Nutrition literacy, being an important dimension of people’s health literacy, has been defined as ‘the ability to find and elaborate on nutrition information and make conclusions regarding health issues’⁽ Reference Silk, Sherry and Winn ^{4 ,} Reference Nutbeam ^{5 )}. There are three major domains of nutrition literacy, which are referred to as functional nutrition literacy (FNL), interactive nutrition literacy (INL) and critical nutrition literacy (CNL)⁽ Reference Nutbeam ^{5 )}.

FNL refers to proficiency in applying basic literacy skills, while INL comprises the cognitive and interpersonal communication skills needed to seek nutrition information and interact appropriately with nutrition counsellors. The CNL domain covers the broad topics of ‘critically evaluating nutrition information and advice’ and ‘engagement in dietary behaviour’.

The first of these topics comprises the skills to ‘justify premises for and evaluate the sender of nutrition claims’ and ‘identify scientific nutrition claims’. Being critical nutrition literate therefore means being proficient in evaluating scientific enquiry and interpreting data and evidence scientifically, which actually means being scientifically literate as described by the Organisation for Economic Co-operation and Development’s Programme for International Student Assessment (PISA)^{( 6 )}. The second topic covered by the domain CNL includes the capability to ‘be concerned about dietary behaviours’ and ‘engage in processes to improve dietary behaviours’⁽ Reference Silk, Sherry and Winn ^{4 ,} Reference Pettersen ^{7 )}. The engagement in dietary behaviour (EDB) scale was developed to assess the EDB part of individuals’ CNL.

By hypothesising individuals’ scientific literacy as a predictor that facilitates the forming of persons’ CNL⁽ Reference Sykes, Wills and Rowlands ^{8 )}, and viewing scientific literacy as a mediator that helps implement the ideas on what scientific knowledge ‘is’ and how scientific knowledge forms and develops, we conducted analyses of an achievement test assessing ability in science and a scale measuring self-efficacy in science.

Self-efficacy in science, scientific literacy and socio-economic status

Self-efficacy (SE), being part of individuals’ self-regulation⁽ Reference Zimmerman ^{9 )}, represents the personal perception of external social factors⁽ Reference Bandura ^{10 ,} Reference Bandura ^{11 )}. In social-cognitive models of health behaviour change (see e.g. Schwarzer and Fuchs⁽ Reference Schwarzer and Fuchs ^{12 )}), SE is viewed as a predictor that facilitates the forming of intended behaviour, as a mediator that helps implement the intentions and as a moderator to help achieve the intended behaviour⁽ Reference Schwarzer ^{13 ,} Reference Gutiérrez-Doña, Lippke and Renner ^{14 )}. Consequently, different dimensions of SE tend to correlate. In education SE is viewed as part of individuals’ self-regulated learning⁽ Reference Zimmerman ^{9 )}.

While SE is a measure of students’ self-reported future expectations about achievement at the present time, an achievement test measures part of students’ scientific literacy. The assessment items in the applied achievement test operationalised the Norwegian natural science curriculum, which focuses on five main areas: ‘the budding researcher’, ‘diversity in nature’, ‘body and health’, ‘phenomena and substances’ and ‘technology and design’^{( 15 )}.

The achievement test items were also distributed across the cognitive domains ‘knowing’, ‘applying’ and ‘reasoning’. While knowing covers facts, concepts and procedures, applying involves direct application of knowledge and conceptual understanding. Items categorised as reasoning assess proficiency in evaluating scientific enquiry, interpreting data and evidence scientifically in unfamiliar situations and complex contexts.

Socio-economic status (SES) reflects social position in relation to others and the traditional indicators at the individual level have been income, education and occupation⁽ Reference Adler and Ostrove ^{16 )}. These are often used interchangeably even though they are only moderately correlated⁽ Reference Ostrove and Adler ^{17 ,} Reference Winkleby, Jatulis and Frank ^{18 )}.

The PISA survey^{( 19 )}, assessing 15-year-old students, included several measures of SES in the student questionnaire. Different measures of economic, cultural and social capital at home were applied. Among all these indicators, the number of books at home had the most powerful individual correlation with science ability^{( 19 )}. The number of books at home is also highly correlated with parental education and income⁽ Reference Ammermueller and Pischke ^{20 )}.

The unidimensional logistic Rasch model for polytomous data (PRM)

In the mathematical representation of the Rasch model for polytomous data (hereafter, ‘the polytomous Rasch model’; PRM), $P\{ X_{{ni}} =x\} =1/\gamma \left\{ {\exp \left[ {\kappa _{x} {\plus}x\left( {\beta _{n} {\minus}\delta _{i} } \right)} \right]} \right\}$ , where $\gamma =\mathop{\sum}\nolimits_{k=0}^m {\exp \left[ {\kappa _{k} {\plus}k\left( {\beta _{n} {\minus}\delta _{i} } \right)} \right]} $ is a normalisation factor ensuring $\mathop{\int}\nolimits_{{\minus}\infty}^\infty {P \cdot {\mathop{\rm d}\nolimits} \beta =1} $ , a person’s attitude is described by a single, unidimensional latent variable β _n defined so that −∞<β _n <∞⁽ Reference Andrich ^{21 ,} Reference Rasch ^{22 )}. The graphical representation of the PRM, referred to as the item characteristic curve, relates the probability P of person n with attitude β _n ticking off response category x on a polytomous item i with affective level δ _i ⁽ Reference Andrich ^{23 )}. The different κ refer to category coefficients. In the case of the achievement test, β _n refers to a person’s ability and δ _i to item difficulty.

Invariant measurement is not guaranteed if the data fit a two- or a three-parameter item response theory model. Only Rasch models provide invariant measurements and support construct validity if the data fit the model. Reliability and sufficiency are also provided when data fit a Rasch model.

The requirements of unidimensional Rasch models are that: (i) the raw scores contain all of the information on a person’s attitude (sufficiency); (ii) the response probability increases with higher attitude (monotonicity); (iii) the responses to items are independent (local independence); and (iv) the response probability depends on a dominant dimension (unidimensionality)⁽ Reference Smith ^{24 ,} Reference Linacre ^{25 )}. If factors other than the dominant dimension influence item responses, the data are biased.

Measurement bias – differential item functioning

Differential item functioning (DIF) means that an item has different affective levels for different groups of individuals such as males and females. Then the observed values for males and females are best described by two different curves similar to the theoretical item characteristic curve. If these curves are parallel the item discriminates similarly across the continuum for both groups and the DIF is said to be mainly uniform⁽ Reference Andrich and Hagquist ^{26 )}. Non-uniform DIF is an important factor for non-invariant measures. Items that show non-uniform DIF should be discarded while items mainly showing uniform DIF might be resolved⁽ Reference Brodersen, Meads and Kreiner ^{27 ,} Reference Looveer and Mulligan ^{28 )} by using the ‘person factor split’ procedure in the item analysis package RUMM2030^{( 29 )}.

The requirement of local independence

The local independence requirement implies that there are no dependencies among items other than those that are attributable to the latent trait. This means that after taking into account the person’s attitude (latent trait), responses to the questionnaire items should be independent. Likewise, taking into account the person’s ability (latent trait), responses to the achievement test items should be independent. Violations of local independence have been formalised as ‘response dependence’ and ‘trait dependence’, where the latter is also referred to as ‘multidimensionality’⁽ Reference Marais and Andrich ^{30 )}.

Response violations of local independence

Response dependence between items appears when two items share something more in common than can be accounted for by the latent trait. One example of response dependence is when two questionnaire items ask for more or less the same information, causing redundancy in the data. Another example is when a previous achievement test item offers clues that affect responses to a subsequent, dependent item⁽ Reference Smith ^{31 ,} Reference Andrich, Humphry and Marais ^{32 )}. Response dependence violates statistical independence and causes ‘response violations’ of local independence⁽ Reference Marais and Andrich ^{30 ,} Reference Andrich and Kreiner ^{33 ,} Reference Marais and Andrich ^{34 )}, meaning that the entire correlation between the items is not captured by the latent trait. The result of response dependency is deviations of the thresholds of the dependent item⁽ Reference Andrich, Humphry and Marais ^{32 )}.

A high correlation between a pair of item residuals (a residual is the difference between the observed and the expected value) is one way of generating a ‘post hoc’ hypothesis of response dependence⁽ Reference Smith ^{24 ,} Reference Marais and Andrich ^{30 )}. When two questionnaire items ask for the same information causing redundancy in the data, one would normally form a subtest, i.e. merge the two items into one composite item. Using the ‘item dependence split’ procedure in RUMM2030, the magnitude of the dependence of a pair of achievement test items, where one offers a clue for the response to the other, might be estimated^{( 29 )} and used to test the hypothesis of response dependence⁽ Reference Brodersen, Meads and Kreiner ^{27 ,} Reference Smith ^{31 )}.

Dimension violations of local independence

Multidimensionality or trait dependence means that ‘multiple’ latent variables or traits play a role and that some items measure one latent variable and other items measure another latent variable. One might form subtests and study whether the latent variables measure one overarching dimension or measure unique aspects. If the latent variables measure unique aspects, the theoretical composite construct might not find support in the empirical evidence as the data are not sufficiently unidimensional.

If, for example, the overarching dimension ‘ability in natural science’ is measured using different clusters or subsets of items assessing knowledge in biology, chemistry, geology and physics, each subset of items represents a latent variable. If, for example, the items assessing knowledge in biology and the items measuring knowledge in physics rank the students quite differently, the different subsets of items might form subscales that contribute with unique variance to the distribution of students’ score sums in natural science. Then the composite construct ‘ability in natural science’ is not sufficiently unidimensional and we should report one score in biology and one score in physics as opposed to a score in natural science – the overarching dimension. Therefore, if a theoretical composite construct is not sufficiently unidimensional, one might want to split the assessment instrument into as many parts as there are latent variables or subscales and do separate analyses. Principal component analysis of residuals might help investigate the dimensionality of the data.

Principal component analysis of residuals

A principal component analysis converts a set of observations (the data) of correlated variables (the items) into a set of linearly uncorrelated variables called principal components. The first principal component has the largest possible variance, i.e. accounts for as much of the variability in the data as possible, and each succeeding component in turn has the highest variance possible under the constraint that it be orthogonal to or uncorrelated with the other components. A principal component analysis therefore reveals the internal structure of the data in a way that best explains the variance in the data. Principal component analysis is closely related to factor analysis.

In a natural science achievement test the cluster of items in biology and the cluster of items in physics have ‘ability in science’ in common. If we remove the common latent trait from the data we are left with the residuals or the deviations from the Rasch model. If the residuals of the biology items correlate positively with the first principal component while the other items correlate negatively, the cluster of items in biology might share something else in common than the general underlying variable ‘natural science’ can ‘explain’. If so, the items in biology represent an additional latent trait that might violate the hypothesis of unidimensional data and hence violate local independence⁽ Reference Marais and Andrich ^{30 ,} Reference Andrich and Kreiner ^{33 –} Reference Ryan ^{35 )}.

Large variations in the percentage variance explained by each principal component is one way of generating a ‘post hoc’ hypothesis about multidimensionality in the data⁽ Reference Smith ^{24 ,} Reference Linacre ^{25 )}. In principle, such hypotheses should come from theoretical and conceptual considerations. The hypothesis might be tested by applying the equating tests and the t-test procedures in RUMM2030^{( 29 )}, and by estimating fractal indices based on a subtest analysis.

Fractal indices and reliability indices specific to a subtest analysis

A set of n items can be analysed either as n items or as two composite items (subscales) where each subscale takes on the role of an item. The subtest analysis takes account of multidimensionality in the data, and fractal indices (A, c and r) are estimated specific to the subtest. The value A describes the variance common to all subscales, the value c characterises the variance that is unique to the subscales and the variable r is the correlation between the two subscales^{( 29 )}. A subtest analysis performed on a data set with acceptable unidimensionality will return a high value for both A and r and a low value for c.

Reliability indices do not indicate whether a scale measures a unidimensional variable or not, but instead provide the value of the reliability on the assumption of unidimensionality^{( 29 )}. In the presence of a multidimensional subscale structure, the variance of person estimates and hence the reliability indices inflate⁽ Reference Marais and Andrich ^{34 )}.

Further, comparing the overall test-of-fit index, i.e. the total item χ ², obtained when the analyses uses (i) the discrete items and (ii) the subscales as two items might indicate changes in fit to the model taking the multidimensionality into account.

The parameterisations of the polytomous Rasch model, the thresholds and the likelihood ratio test

When the observed distance between the response categories on a rating scale is identical across the items, the data fit ‘the rating scale parameterisation’⁽ Reference Andrich ^{23 )} of the PRM best. If the distance is not the same across the items, ‘the partial credit parameterisation’⁽ Reference Wright and Masters ^{36 )} is indicated. When applying the partial credit parameterisation, the ‘thresholds’ should be ordered.

A threshold is defined as the person location at which the probability of responding in one of two adjacent response categories is equal, and in the special case of dichotomous data this probability is 0·50. A polytomous item with an m+1 number of response categories has m thresholds (τ _k ), where the index k takes on values from 1 to m and x takes on values from 0 to m+1. The score x indicates the number of m thresholds a respondent has passed⁽ Reference Andrich, de Jong and Sheridan ^{37 )}.

To treat the scales as linearly and directly related to the latent variable, where the succeeding response categories reflect successively more of the latent variable, we must examine whether the variables EDB and SE possess the properties of interval scales or are ordinal variables. If respondents use the rating scales in the questionnaire as expected, the observed succeeding thresholds should reflect successively more of the latent attitude and hence be ordered⁽ Reference Andrich ^{38 )}. Disordered thresholds in the data violate the hypothesised ordering of response categories, meaning that respondents have not used the scales as expected. If so, the variables cannot be treated as interval variables⁽ Reference Singh ^{39 )}.

The Fisher’s likelihood ratio test available in RUMM might be used to assess the efficiency of the partial credit parameterisation as compared with the rating scale parameterisation of the PRM. The parameterisations are compared against each other for the same model specifications.

Item discrimination, model fit, reliability and targeting

When an item, as part of a set of items, provides data that sufficiently fit a unidimensional Rasch model, the item provides an indication of attitude or ability along the latent variable. In Rasch analysis, this information is used to construct measures.

If the data do not fit the item characteristic curve – the theoretical expectation under the model – but rather approach a step function, the item is said to over-discriminate and the item might stratify the persons below and above a certain attitude estimate. If the data approach a constant function, the item is said to under-discriminate. Under-discriminating items tend to neither stratify nor measure. Strongly over- and under-discriminating items do not fit the Rasch model.

Fit residuals and item χ ² values are used to test how well the data fit the model⁽ Reference Smith and Plackner ^{40 )}. Negative and positive item fit residuals indicate whether items over- or under-discriminate. Similarly, a person fit residual indicates how well a person’s response pattern matches the expectation under the model⁽ Reference Andrich ^{41 ,} Reference Andrich ^{42 )}.

Large χ ² indicate that persons with different attitudes do not ‘agree on’ item affective estimates, thus compromising the required property of invariance. To adjust χ ² probabilities for the number of significant tests performed, the probabilities are Bonferroni-adjusted⁽ Reference Bland and Altman ^{43 )} using RUMM2030^{( 44 )}.

Estimates of Cronbach’s α and the person separation index (PSI) are used as indices of ‘reliability’⁽ Reference Traub and Rowley ^{45 )}. When the distribution of the items’ threshold estimates matches the distribution of the persons’ attitude estimates the instrument is well ‘targeted’. Well-targeted instruments help reduce the measurement error.

Frame of reference and data collection

One hundred randomly sampled public schools across Norway offering tenth grade were asked whether they could participate in a field-test trial for the ‘national sample test’ in science. The schools were contacted by regular mail on 21 November 2012, by email on 10 December 2012 and by telephone during the period 3–7 January 2013. As a result, 740 tenth-grade students with an age range from 14 to 15 years (48 % females and 9 % minority students) from twenty-seven public schools chose to take part in the voluntary field trial of the assessment instruments. The number of participating schools was low as no incentive was offered and some schools experienced technical problems when enrolling their students in the test administration system.

Twenty-two out of the twenty-seven schools reported the number of students in the participating class. At these schools the number of students who actually responded to the achievement test and the questionnaire ranged from 67 to 100 % of the students, with an average of 81 %.

The field-trial data were collected during the period 16 January–15 February 2013. When logging on to the applied electronic assessment tool, each student was assigned to one out of four different electronic ‘test booklets’. Each booklet contained science achievement test items and a student questionnaire that was completed at school within 90 min. Only one of these test booklets contained the EDB scale and the SE in science scale, and 178 students responded to this specific test booklet.

As the Scandinavian countries (Norway, Sweden and Denmark) have strong cultural and linguistic similarities, a student was defined as a majority student if at least one of his or her parents had been born in Scandinavia. Hence, a minority student in the present study is either an immigrant or a descendent of two immigrants (second generation).

The engagement in dietary behaviour and self-efficacy scales, the achievement test, the socio-economic status indicator and the items asking for the students’ cultural and linguistic background

All items in the EDB and the SE scales are reported in Table 1. The EDB scale, consisting of six items, is a revised version of the engagement in dietary habits scale reported by the authors⁽ Reference Guttersrud, Østerholt Dalane and Pettersen ^{1 )}. Items 68 and 69 are at the personal level, items 70 and 71 are at the social level, and items 72 and 73 are at the global level. The SE in science scale, consisting of five items, is based on the SE in science scale and the control expectation scale applied in PISA^{( 19 )}. Six-point rating scales, with the extreme response categories anchored with the phrases ‘strongly disagree’ (=1) and ‘strongly agree’ (=6), were applied for all items in the EDB and the SE scales.

Table 1 The wording of the items in the engagement in dietary behaviour (EDB) scale and the self-efficacy (SE) in science scale (originally stated in Norwegian)

The achievement test in the same field-test booklet as the EDB scale and the SE scale consisted of fifty-nine items, of which two were open-ended. The items were distributed across the competence aims in the science curriculum after grade ten and across the described cognitive categories.

An item asking for the number of books at home taken from the PIRLS (Progress in International Reading Literacy Study) student questionnaire^{( 46 )} was applied as an indicator for SES. The categories for number of books at home were 0–10, 11–25, 26–100, 101–200 and >200 books. To help students decide the number of books, pictures of how ten, twenty-five, 100 and 200 books might look in shelves were provided.

Student’s cultural background was obtained from an item asking for the students’ and the parents’ birth place. The three categories for birth place were: (i) Norway; (ii) Sweden or Denmark; and (iii) ‘Other’. The categories (i) and (ii) were merged into one category. The students also reported linguistic background: the language spoken at home most of the time. The two categories for linguistic background were: (i) Norwegian, Swedish or Danish; and (ii) ‘Other’. Students’ gender was available in the applied electronic national assessment tool.