Introduction
Adequate exposure assessment is essential in all epidemiological studies, although it may be particularly challenging in nutritional epidemiology(Reference Naska, Lagiou and Lagiou1). Dietary intake represents a complex and multidimensional exposure, largely conditioned by culture and geographic location. Being a dynamic exposure, dietary intake is prone to variation across days, weeks, seasons and the lifecycle(Reference Kirkpatrick, Baranowski and Subar2). However, the global recognition of diet as a major modifiable risk factor for various non-communicable diseases implies that measuring and targeting diet remains essential(Reference Medin, Carlsen and Hambly3). In the absence of objective biomarkers for overall diet, nutrition researchers typically administer methods to obtain self-reported dietary intake(Reference Kuhnle4), inevitably introducing measurement errors which should be considered during data analysis and interpretation(Reference Naska, Lagiou and Lagiou1,Reference Kirkpatrick, Vanderlee and Raffoul5) . Hence, assessing the validity of such dietary assessment tools is crucial to ensure that diet-disease associations are reported as precisely as possible(Reference Cade, Thompson and Burley6). Owing to the unfeasibility of accurately measuring the self-selected diet of individuals over longer time periods, studies often assess relative validity by comparing the test method with an alternative method for dietary intake assessment that has its own set of limitations(Reference Salvesen, Hillesund and Vik7,Reference Masson, McNeill and Tomany8) .
The food frequency questionnaire (FFQ) is designed to retrospectively measure habitual long-term dietary intake and is the most frequently applied dietary assessment tool in large-scale epidemiological studies(Reference Steinemann, Grize and Ziesemer9,Reference Kipnis, Midthune and Freedman10) . Despite the feasibility and cost-effectiveness of FFQs, various biomarker studies have raised concerns regarding the validity of FFQs and their ability to detect moderate diet-disease relationships(Reference Mitry, Wawro and Six-Merker11). To obtain adequate estimates for ranking individuals according to intake levels, de Boer et al. recommended administering at least two non-consecutive 24-hour dietary recall interviews (24-HDRs) combined with a food propensity questionnaire(Reference de Boer, Slimani and van ‘t Veer12). Although the 24-HDR does not produce unbiased data, its data have been reported to be less prone to bias than the FFQ in several populations. Hence, the 24-HDR may be used as a reference method in relative validation studies of the FFQ(Reference Freedman, Commins and Moler13,Reference Freedman, Commins and Moler14) .
The Hordaland Health Study (HUSK3) is a population-based survey designed to collect health data that may contribute to disease prevention. HUSK3 included the assessment of dietary intake, completed by the administration of an extensive web- and image-based FFQ (WebFFQ), as well as 24-HDRs. It is recommended that estimates of validity are obtained from subjects who are representative of the main study target population(Reference Cade, Thompson and Burley6,Reference Cade, Burley and Warm15,Reference Willett16) . Hence, in a subgroup of the HUSK3 study population, we aimed to assess the relative validity of the WebFFQ in estimating energy, nutrient and food intake by comparing its measures with those derived from repeated, non-consecutive 24-HDRs.
Methods
Study design and subjects
From 1992 to 1993, all residents of Hordaland County born in 1925–27 and 1950–52 were invited to participate in the Hordaland Homocysteine Study. From 1997 to 1999, participants belonging to the 1925–27 and 1950–51 birth cohorts were reinvited to participate in HUSK2. Dietary data collection was introduced in HUSK2 through a 169-item semi-quantitative paper-based FFQ (PB-FFQ), developed at the Department of Nutrition, University of Oslo, Norway.
In 2018, the follow-up health survey, HUSK3, was initiated as a joint project between the University of Bergen, Helse Bergen HF, and the Public Dental Health Service Competence Centre for Western Norway (TkVestland). In HUSK3, men and women born in 1950–51, who had previously taken part in HUSK2 were recruited. HUSK3 is a population-based observational study consisting of 2232 men and women. Participants were enrolled and health examinations were conducted from 2018 to 2020. HUSK3 included measurements of body weight, height, blood pressure, body composition, handgrip strength, as well as echocardiography, blood tests and assessment of dietary intake. Dietary intake information was collected from all HUSK3 participants through a single in-person 24-HDR on the HUSK3 survey day, and all participants received information on how to access the WebFFQ to be completed at home, following the HUSK3 survey day.
Within a subsample of HUSK3, we conducted a study with the aim of addressing methodological challenges in future analyses using dietary intake data collected in HUSK3 to assess diet-health associations. In the present study, we compare measures of nutrient and food intake from the HUSK3 WebFFQ with measures of nutrient and food intake from three repeated 24-HDRs. In the present comparison study, 175 men and women who underwent the HUSK3 health survey between January and March 2020 granted their informed written consent to participate. In addition to the in-person 24-HDR and WebFFQ administered to all HUSK3 participants, the participants in the comparison study agreed to provide additional dietary information through telephonic 24-HDRs.
On the HUSK3 survey day, the first 24-HDR was conducted, and participants were given a link to access the WebFFQ, which they were requested to complete at home. Hence, the WebFFQ was administered after the first 24-HDR, but prior to conducting the second and third 24-HDR for the comparison study. However, participants were recruited sequentially as they came to attend the HUSK3 survey at the Research Unit for Health Surveys (RUHS) – and therefore filled out the WebFFQ at different time points. Within the comparison study, the in-person 24-HDR conducted on the survey day was regarded as the first administration. The second administration of 24-HDR was completed via telephone by 121 participants included in the comparison study, while the third 24-HDR was completed via telephone by 68 of the included participants. The study was conducted in accordance with the Declaration of Helsinki, and dietary data collection was approved by the Regional Committees for Medical and Health Research Ethics West (REK 2017/294).
Data collection methods
The Web-based FFQ
The WebFFQ used in HUSK3 was developed by the Department of Nutrition, Institute of Basic Medical Sciences, University of Oslo. The validity of the WebFFQ has previously been assessed by Medin et al. in adults aged 18–70 years(Reference Medin, Carlsen and Hambly3). The self-administered WebFFQ consists of 279 items intended to measure the habitual intake of the following foods: bread, sandwich spreads, cereals, yoghurt, non-alcoholic and alcoholic beverages, hot dishes containing meat and fish/seafood, potatoes, rice, pasta, vegetables, sauces and condiments, fruits and berries, nuts and seeds, desserts, cakes and pastries, chocolate and sweets, types of cooking fat, and fat used on bread. Furthermore, the WebFFQ enquired about dietary supplement use. Finally, an open field allowed participants to report the consumption of foods or supplements that were not already included in the WebFFQ.
Frequency of consumption was inquired for all food items, ranging from never to multiple times daily, with the number of frequency response options and intervals varying depending on the food item in question. Portion sizes were also estimated for all food items, although in different ways. For some food items (e.g. bread, sausages, hamburgers, spring rolls, sushi, fish cakes, taco shells), the WebFFQ inquired about ‘number of slices/pieces’ consumed of an assumed standard portion size (e.g. number of sausages). For food items typically difficult to estimate in units (e.g. breakfast cereals, pizza, lasagna, stews, wok, nuts), portion sizes were estimated by using pictures of four different portion size alternatives. Each picture corresponded to a predefined quantity (grams) of food. The standard portion sizes applied in the WebFFQ are based on values from ‘Weights, measures and portion sizes for foods’(Reference Dalane, Bergvatn and Kielland17).
Information on the intake of food items, energy and nutrients was generated using KostBeregningsSystemet (KBS, version 7.4, database AE18), a food and nutrient composition database and calculation system developed at the Department of Nutrition, Institute of Basic Medical Sciences, University of Oslo. The AE18 database is based on the official Norwegian Food Composition Table version 2018, with added items, including approximately 3400 food items(18). When the consumption frequency was reported as a range (e.g. 1–2 times per week), the mean frequency was applied. Missing frequencies and portion estimates were avoided by built-in error checks, meaning that participants could not proceed to the next question unless all boxes were ticked off for each question. To estimate the daily intake of food, energy and nutrients, the recorded frequency was multiplied by the corresponding portion size. Fat used for bread and cooking, as well as supplements, was considered in the nutrient calculations. The mixed dishes were split into separate ingredients, which allowed each ingredient to be categorised into the most appropriate food group (e.g. pizza was split into cheese, meat, vegetables, flour). The splitting of foods did not affect the calculation of nutrients. The foods and nutrients included for the food/food group analysis were selected prior to data analysis. The included foods represent the main food groups enquired about in the WebFFQ, whereas nutrients were selected due to their presumed relevance to future studies of diet-disease associations(Reference Brantsæter, Knutsen and Johansen19,Reference Totland, Melnæs and Lundberg-Hallén20,Reference Madar, Heen and Hopstock21) .
The 24-hour dietary recall interviews
The 24-HDR was chosen as the reference method to assess the relative validity of the WebFFQ. The first 24-HDR was conducted in person as an integrated part of the HUSK3 health survey, using a paper-based approach. The in-person 24-HDR applied a three-part approach, approximately similar to that of the telephonic 24-HDRs described below. However, in contrast to the telephonic 24-HDRs, it did not apply a picture booklet to facilitate portion size estimation. Rather, the in-person 24-HDR required participants to report amounts in household measurements (e.g. cups, tablespoons), natural units (e.g. number of slices/pieces) and standard units of measurement (grams, decilitres). The second and third 24-HDR rounds were conducted via telephone using an integrated 24-hour multi-pass recall module in KBS, which was connected to its nutrition composition database (KBS, version 7.4, database AE18). The method has previously been described by Myhre et al. (Reference Myhre, Løken and Wandel22) and involves three steps. First, the participant provides a free description of the meals that were consumed the day prior to the interview. Second, the interviewer restates all reported items while adding follow-up questions regarding portion sizes, potential forgotten items and meals. Finally, the interviewer probes for frequently overlooked items such as snacks and supplements.
To facilitate portion estimation, the participants received a booklet containing pictures of different portion sizes prior to the telephonic 24-HDRs. The picture booklet used for the telephonic 24-HDRs was originally developed for use in Norkost3, which is the third national dietary survey in Norwegian adults conducted between 2010 and 2011(23). The booklet included pictures of plates, bowls, glasses and cups of different volumes. This was followed by pictures of bread rolls, baguettes and nine illustrations of bread slices of oval and squared shapes. Each illustration of bread referred to two to three different thickness alternatives. Finally, the booklet included pictures of thirty-three foods/dishes, with four different portion sizes ranging from small (A) to large (D) for each food. Each portion size alternative in the picture booklet corresponds to a weight in grams. The selection of pictures was based on experiences from a previously conducted Norwegian survey, Ungkost 2000(Reference Øverby and Andersen24), as well as a booklet used in the European validation project, EFCOVAL(Reference Slimani, Casagrande and Nicolas25). The illustrations of bread have been obtained from a picture booklet used in The Norwegian Women and Cancer Study (NOWAC) at The University of Tromsø(Reference Brustad, Skeie and Braaten26). To facilitate the reporting of bread type, the picture booklet included the Bread Scale symbol for the level of wholegrain content, developed by the Federation of Norwegian Bakers and Confectioners.
To standardise the process, all 24-HDRs included in the comparison study were performed by the same clinical dietitian (ZS). All recall interviews used in this study were collected between January 2020 and January 2021 and were spread out evenly to account for day-to-day and seasonal variations. Fifty-four (27 %) of the recall interviews included in this study reflected dietary intake on weekend days. To avoid conscious or subconscious alterations in dietary intake according to social desirability, the interviewees were not informed of the days they would be interviewed on in advance. KBS software and its food composition database were used to estimate food, energy and nutrient intake from the 24-HDRS. Mixed dishes were split into separate ingredients, allowing each ingredient to be categorised into the most appropriate food group.
Statistical analyses
Background characteristics are presented as medians with percentiles (p25–p75) for non-normally distributed variables. Categorical variables are presented as proportions (%). Due to the predominantly skewed distributions of nutrient and food intake data, these are presented as medians with the corresponding 25th and 75th percentiles. Normality was assessed using the Kolmogorov–Smirnov test and by examination of the associated Q-Q plots and histograms. Dietary supplements were included in the nutrient intake estimates from the WebFFQ and the 24-HDRs. Adjustment for total energy intake (kilojoules) was carried out using the simple nutrient density model as described by Willet et al. (Reference Willett, Howe and Kushi27).
The Wilcoxon signed-rank test was applied to assess the differences between the estimated nutrient and food intake from the WebFFQ and repeated 24-HDRs. The relative validity of the WebFFQ was assessed by computing Spearman's rank-order correlation coefficients, reflecting the degree to which the WebFFQ and the 24-HDRs ranked participants equally in terms of nutrient and food intake. In accordance with previous dietary validation studies(Reference Masson, McNeill and Tomany8,Reference Lombard, Steyn and Charlton28) , the following Spearman's rho cut-offs were applied to categorise the strength of the correlations: poor <0⋅20, acceptable 0⋅20–0⋅49 and strong ≥0⋅50.
Cross-classification tables were created to evaluate the extent to which the WebFFQ classified participants into the same quartiles of dietary intake as the repeated 24-HDRs. Due to non-reporting in the 24-HDRs, proper cross-classification tables could not be computed for alcohol and some foods. Agreement between methods is represented by the proportion of participants classified in the same/adjacent or opposite quartile by the WebFFQ compared with the 24-HDRs.
The agreement between the WebFFQ and 24-HDRs was also assessed using Bland–Altman plots, visualising the difference in intake between the two methods plotted against the mean intake of the two methods. Limits of agreement (LOAs) were calculated as the mean difference ± 1⋅96 sd, providing an interval that includes the difference between single measurements on the same participant by the two methods with 95 % probability. Calibration coefficients (λ) were derived using linear regression of the 24-HDR data (dependent variable) on the FFQ data (independent variable).
All reported P-values are two-sided, and a statistical significance level of P < 0⋅05 was applied. All statistical analyses were performed using Statistical Package for Social Sciences version 25, IBM Corporation (IBM Corp. Released 2017. IBM SPSS Statistics for Windows, version 25.0. Armonk, NY: IBM Corp).
Results
Among the sixty-eight participants who completed three repetitions of the 24-HDR, one participant was excluded due to a lack of WebFFQ data. The background characteristics of the participants included in the analysis are presented in Table 1. Among the sixty-seven participants who completed the WebFFQ and three repetitions of the 24-HDRs, 60 % were female, the median body mass index (BMI) was 24⋅9 kg/m2 and the median waist circumference was 104⋅5 cm in males and 86⋅0 cm in females. 5 % of the participants were current smokers. With regard to the mentioned characteristics, the participants completing the comparison substudy were representative of the main HUSK3 study population.
Table 1. Characteristics of the participants completing the comparison study and the total Hordaland Health Study 3 cohort
24-HDR, 24-hour dietary recall; WebFFQ, web-based food frequency questionnaire; BMI, body mass index.
a Median (25th percentile–75th percentile).
b Weight data available for 2182 participants. Height data available for 2178 participants. BMI data available for 2173 participants. Waist data available for 984 men and 1192 women.
Table 2 presents the absolute and energy-adjusted intakes of energy-providing nutrients including dietary fibre and alcohol, selected micronutrients and various food groups from the WebFFQ and the mean of three 24-HR dietary. Estimated absolute intakes from the WebFFQ were significantly higher for twenty-six out of thirty nutrients and foods. The largest overestimations in absolute intake by the WebFFQ were observed for ‘vegetables’, iodine and calcium. The only food significantly underestimated by the WebFFQ was ‘cheese’. No significant differences in absolute intakes between the WebFFQ and the 24-HDRs were evident for added sugar, alcohol and ‘eggs’. Energy adjustment of intake led to a prominent decline in the number of overestimated nutrients and foods by the WebFFQ, with only eleven nutrients and foods being significantly overestimated, seventeen being similar, and two being significantly underestimated.
Table 2. Absolute and energy-adjusted intakes of nutrients and foods, and Spearman's correlation coefficients between estimated absolute (rs) and energy-adjusted (ra) intakes from the WebFFQ and the mean of three 24-hour dietary recalls in a comparison study among Norwegian adults participating in the Hordaland Health Study 3
kJ, kilojoule; kcal, kilocalories; FFQ, food frequency questionnaire; 24-HDR, 24-hour dietary recall; p25, 25th percentile; p75, 75th percentile; MUFA, monounsaturated fatty acids; PUFA, polyunsaturated fatty acids; rs, Spearman's correlation coefficient; ra, energy-adjusted Spearman's correlation coefficient.
a kJ for absolute intake. kcal for energy-adjusted intake.
† Intake from the WebFFQ compared with intake from the repeated 24-HDRs using the Wilcoxon signed-rank test.
* Statistically significant difference from estimated WebFFQ intake | Statistically significant Spearman's rank correlation. The significance level was set at P < 0⋅05.
** Statistically significant Spearman's rank correlation. The significance level was set at P < 0⋅01.
Spearman's correlation coefficient between estimated intakes from the WebFFQ and 24-HDRs (Table 2) ranged from 0⋅19 (iodine) to 0⋅69 (vitamin D) for nutrients and from 0⋅31 (fatty fish) to 0⋅71 (juice) for foods. Correlations were statistically significant for all nutrients and foods, except iodine. All correlations remained statistically significant after energy adjustment. Improvements in correlation coefficients were observed for nine out of thirty nutrients and food when energy-adjusted, with the most prominent improvement for calcium, ‘meat, blood, offal’ and ‘vegetables’. However, energy adjustment led to the largest decline in the correlation coefficients for saturated fat, protein and carbohydrates.
The stability of quartile membership was assessed by cross-classification of intake estimates from the WebFFQ and 24-HDRs, as shown in Table 3. For most nutrients and foods, extreme misclassification did not exceed 5 %. The highest degree of similar classification was observed for ‘fruits and berries’ and vitamin D, while the lowest degree of similar classification was observed for iodine, ‘red meat’ and ‘meat, blood, offal’.
Table 3. Agreement of quartile membership, mean differences with limits of agreement between estimated absolute daily intakes of energy, nutrients and foods from the 24-HDRs and WebFFQ, and the calibration coefficient in the Hordaland Health Study 3
FFQ, food frequency questionnaire; 24-HDR, 24-hour dietary recall; sd, standard deviation; 95 % CI, 95 % confidence interval; LOA, limits of agreement; MUFA, monounsaturated fatty acids; PUFA, polyunsaturated fatty acids.
a Energy-adjusted calibration coefficient.
n/a, not available as cross-classification could not be computed for variables because of non-reporting in the 24-HDRs.
† WebFFQ – 24-HDR.
‡ Mean difference ± 1⋅96 sd of the difference.
Linear calibration coefficients ranged from 0⋅10 (wholegrain) to 0⋅81 (alcohol) for nutrients, and from 0⋅10 (bread) to 0⋅61 (cheese) for foods (Table 3). When energy-adjusted, the calibration coefficients increased for all nutrients and foods. All calibration coefficients were below 1, except for the energy-adjusted value for ‘fatty fish’.
LOAs were wide, and visual inspection of Bland–Altman plots indicated various patterns: (i) no relationship between differences and low and moderate mean intake, but increasing positive differences with increasing mean intake (e.g. fat and bread) (Fig. 1(a)), indicating greater overestimation by the WebFFQ compared with the 24-HDRs for high intake quantities, (ii) increasing positive differences with increasing mean intake values (e.g. vegetables and protein) (Fig. 1(b)), indicating that the overestimations by the WebFFQ increase consistently with increasing intake quantity and (iii) increased spacing of scatter with increasing intakes, implying that the agreement decreases with increasing intake quantities (e.g. meat and added sugar) (Fig. 1(c)). Few participants were found to lie beyond LOAs.
Fig. 1. (a) Bland–Altman plot of agreement between fat intake estimated from the web-based food frequency questionnaire (WebFFQ) and the three 24-hour dietary recall interviews (24-HDRs) (n 67) in the Hordaland Health Study 3. (b) Bland–Altman plot of agreement between vegetable intake estimated from the web-based food frequency questionnaire (WebFFQ) and the three 24-hour dietary recall interviews (24-HDRs) (n 67) in the Hordaland Health Study 3. (c) Bland–Altman plot of agreement between meat, blood, offal intake estimated from the web-based food frequency questionnaire (WebFFQ) and the three 24-hour dietary recall interviews (24-HDRs) (n 67) in the Hordaland Health Study 3. The solid line shows the mean difference between the two methods, and the dotted lines show limits of agreement (LOA) corresponding to ±1⋅96 sd.
Discussion
The results of the present comparison study suggest that the WebFFQ administered in HUSK3 performed reasonably well in estimating and ranking intakes of a variety of nutrients and foods when compared with estimated intakes from three repeated 24-HDRs.
The calculated intakes indicated that the WebFFQ significantly overestimated the absolute intakes of all nutrients and most foods, while it underestimated the intake of a few foods. Similar observations of overestimation by FFQs compared with other dietary assessment methods have been made in other cohort studies(Reference Medin, Carlsen and Hambly3,Reference Moghames, Hammami and Hwalla29–Reference Noor Hafizah, Ang and Yap33) , although some studies have reported underestimations by FFQs(Reference Horiuchi, Kusama and Sar34,Reference Knudsen, Hatch and Cueto35) . Our finding of general overestimation of nutrients and foods by the WebFFQ compared with repeated 24-HDRs is particularly consistent with the findings of Medin et al. (Reference Medin, Carlsen and Hambly3), who assessed the validity of the same WebFFQ in a different population using both doubly labelled water (DLW) and multiple 24-HDRs as reference methods.
The predominant overestimation of the absolute intake estimated by the WebFFQ in our study may be at least partly explained by the large number of food items listed and the inclusion of rarely consumed food items in the WebFFQ, which may not have been captured by the three repetitions of the 24-HDR(Reference Cade, Thompson and Burley6,Reference Krebs-Smith, Heimendinger and Subar36) . Furthermore, we observed that amount of intake seemed to be related to the number of items in the WebFFQ, with higher intakes generally being evident for food groups with a large number of listed items. Medin et al. (Reference Medin, Carlsen and Hambly3) observed that 24-HDR underestimated energy intake compared with DLW in their study, reminding us that the reference method, the 24-HDR, is also affected by sources of error(Reference Cade, Burley and Warm15). This may indicate that the difference in dietary intake estimates between the WebFFQ and the 24-HDRs in our study may have been further increased due to underestimation by the 24-HDRs. The differences in absolute intake estimates decreased when energy adjusted. As described by Willett et al. (Reference Willett, Howe and Kushi27), the intake of several nutrients, in particular energy-providing nutrients, are correlated with total energy intake. Although energy adjustment has mainly been discussed in the context of energy-providing nutrients, many dietary components tend to be positively correlated with total energy intake: Individuals who have/report a higher intake of foods generally will have a higher total energy intake – consequently leading to a higher intake of many nutrients(Reference Willett, Howe and Kushi27).
Additionally, the WebFFQ and 24-HDRs may have been disproportionately affected by social desirability bias. Considering that the WebFFQ was self-administered, whereas the 24-HDRs were interviewer-administered and conducted by a clinical dietitian, participants may have been more prone to underreporting their intake of energy-dense foods during the 24-HDRs(Reference Cade, Burley and Warm15,Reference Kreuter, Presser and Tourangeau37) .
Estimated correlation coefficients between the WebFFQ and 24-HDRs for absolute intake data ranged from 0⋅19 (iodine) to 0⋅69 (vitamin D) for nutrients and from 0⋅31 (fatty fish) to 0⋅71 (juice) for foods, with all correlations being categorised as acceptable or strong, apart from ‘iodine’. This is consistent with the findings of the study by Medin et al. (Reference Medin, Carlsen and Hambly3) in which correlation coefficients ranging from 0⋅19 (fibre) to 0⋅69 (juice) were reported. Similar to the findings of Medin et al., the highest correlation coefficient in our study was observed for ‘juice’(Reference Medin, Carlsen and Hambly3). A recent systematic review and meta-analysis reported FFQ validity correlation coefficients ranging from 0⋅22 to 0⋅77 for the sixty-six included studies that used 24-HDR as a reference method(Reference Cui, Xia and Wu38). Hence, the correlation coefficients in our study appear to be in accordance with those of other relative validation studies(Reference Streppel, de Vries and Meijboom39–Reference Zanini, Simonetto and Bertolotti41). Although better correlation estimates have been reported in some studies, such discrepancies may be attributed to methodological differences, such as differences in the length of the FFQ, the number of 24-HDRs used as the reference method, sample size, and the nutrients and foods included in the validity assessment.
The weaker ranking ability of our WebFFQ for iodine may be partly explained by differences in the time frame covered by the two methods. Along with milk and dairy products, lean fish are among the most important sources of iodine in the Norwegian diet(Reference Dahl, Johansson and Julshamn42,Reference Medin, Carlsen and Andersen43) . Adequate concentrations of iodine only occur naturally in a selection of foods, with the highest concentrations being found in marine fish(Reference Nerhus, Markhus and Nilsen44). While the WebFFQ covered dietary intake in the long run, the 24-HDR covered short-term intake and may not have been able to capture episodically consumed foods such as lean fish. Hence, the relative validity of FFQs is generally better for frequently ingested nutrients and foods.
While correlation coefficients may reflect the agreement in ranking between the WebFFQ and the repeated 24-HDRs(Reference Masson, McNeill and Tomany8), the Bland–Altman method is preferred when evaluating the degree to which the two methods agree across a range of intakes(Reference Cade, Thompson and Burley6). Mean differences were positive for all nutrients and foods except alcohol and ‘cheese’, corroborating the finding that the WebFFQ generally yielded a higher intake than the 24-HDRs. Although several patterns were evident in the plots, we generally observed that the mean differences between the WebFFQ and 24-HDRs increased with increasing intake quantities. Other studies have reported comparable findings, suggesting an increasing bias with increasing mean intake(Reference Brantsaeter, Haugen and Alexander30,Reference Zanini, Simonetto and Bertolotti41,Reference Næss, Aakre and Kjellevold45) .
As mentioned above, the validity of the WebFFQ has been assessed in a previous study, although in a different age group(Reference Medin, Carlsen and Hambly3). The background of the study participants may naturally influence the quality of response to dietary questionnaires which may in turn influence the magnitude of systematic and random errors(Reference Johansson, Hallmans and Wikman46). In addition, food preferences and availability may differ considerably between settings(Reference Moghames, Hammami and Hwalla29). Furthermore, completion of the WebFFQ administered in HUSK3 required a certain degree of digital literacy, which may be indicative of a high educational level and socioeconomic status among our study participants(Reference Hsieh, Rai and Keil47). Hence, validity estimates from one population may generally not be extrapolated to other populations, and it has, therefore, been recommended that questionnaires be validated in subsamples that are representative of the main study(Reference Sharma48). Such representativeness was ensured in the present study by including participants from the original HUSK3 cohort, in which the WebFFQ was administered. Representativeness was further corroborated by the similarity in BMI, waist circumference and smoking status between our study subgroup and the total HUSK3 cohort. Although the validity of the WebFFQ has previously been assessed by Medin et al. (Reference Medin, Carlsen and Hambly3), there are considerable differences between the two study populations, especially with regard to age. While the study by Medin et al. includes subjects between the ages of 18–70 years, HUSK3 operates with a narrow age group (67–70 years). An added advantage of conducting the comparison study in a subsample of the main study population in which risk analyses will be performed, is the opportunity to perform regression calibration to account for measurement errors in the dietary data(Reference Rosner, Spiegelman and Willett49,Reference Spiegelman, Schneeweiss and McDermott50) .
Another key strength of our study is the use of a WebFFQ specifically tailored to suit the Norwegian food culture. The use of images in the WebFFQ and picture booklets for the telephonic 24-HDRs may have led to a more accurate portion size estimation and reduced burden on participants. However, the use of the picture booklet was limited to the telephonic 24-HDRs. Using the picture booklet in the first in-person 24-HDR as well, may have contributed to further standardisation and improvements in portion size estimation by the reference method. The moderate agreement between the WebFFQ and 24-HDRs in our study may be attributed to day-to-day variations in dietary intake, which potentially means that three repetitions of the 24-HDR method might not have provided an adequate impression of habitual diet in the same manner as the WebFFQ. Although it has been proposed that increasing the number of recalls may enhance agreement between methods(Reference Willett51), it may also lead to lower response quality and increase the probability of withdrawal due to the excessive burden on participants. Inclusion of all participants who completed at least two 24-HDRs in the analysis led to a subtle decline in correlation coefficients for most of the nutrients and foods (data not shown), which may indicate the importance of conducting at least three repetitions of the 24-HDR. A key limitation in our study is the lack of biomarkers to examine the validity. Although the use of biomarkers may have provided added knowledge about the validity of the WebFFQ, it was deemed unfeasible due to the scarcity of biomarkers reflecting overall dietary intake, but also due to expenses and respondent burden(Reference Steinemann, Grize and Ziesemer9,Reference Thompson, Subar, Coulston, Boushey, Ferruzzi and Delahanty52) .
The current comparison study suggests that the WebFFQ administered in HUSK3 is a valuable tool for estimating habitual intake of most of the included nutrients and foods, whereas weaker ranking abilities were observed for others, e.g. iodine. Nutritional epidemiology typically aims to assess the association between different intake levels and health outcomes, implying that the acceptable ranking of individuals according to intake levels is more important than the absolute dietary intake(Reference Streppel, de Vries and Meijboom39). Dietary data collected in HUSK3 are mainly intended for use in analyses of diet and health-related outcomes. Hence, the present study fills an important knowledge gap in the assessment of dietary intake within this cohort and may provide a better understanding of diet-disease associations investigated in future studies using data from HUSK3.
Acknowledgements
The authors express their sincere gratitude to the study participants and fieldworkers at the Research Unit for Health Surveys (RUHS), University of Bergen, and to The Trond Mohn Foundation (TMS) which partly financed the RUHS. The authors are thankful to the Division of Nutritional Epidemiology, Institute of Basic Medical Sciences, University of Oslo for providing technical assistance and access to the KostBeregningsSystemet (KBS, version 7.4, database AE18).
Z. S., A. H., H. R. R. and J. D. conceptualised and designed the trial. Z. S. was responsible for data collection, data entry, statistical analyses and writing the manuscript for the present study. A. H. contributed to interpretation of the statistical analyses. All the authors have read and approved the final version of the manuscript.
HUSK3 was funded by the Western Norway Regional Health Authority (Helse Vest RHF). HUSK3 data collection in Bergen was performed at the Research Unit for Health Surveys (RUHS) at the University of Bergen, which received funding from the TMS (Grant ID BFS2017TMT02). The current project was funded by the Mohn Nutrition Research Laboratory. The funders were not involved in the design of the study, collection, entry, analysis, interpretation of data or writing of the article.
The authors declare that there is no conflict of interest.
This study was conducted in accordance with the principles of the Declaration of Helsinki. Dietary data collection within HUSK3 was approved by the Regional Committee for Medical and Health Research Ethics West (REK 2017/294). Participation was voluntary and withdrawal was possible at any time without further justification.
The datasets used in the current study can be obtained from the corresponding author upon reasonable request.
Open access
