Hostname: page-component-5d59c44645-lfgmx Total loading time: 0 Render date: 2024-03-03T17:05:12.984Z Has data issue: false hasContentIssue false

Assessment of Executive Functioning in Patients with Meningioma and Low-Grade Glioma: A Comparison of Self-Report, Proxy-Report, and Test Performance

Published online by Cambridge University Press:  08 November 2019

Sophie Dorothee van der Linden*
Department of Neurosurgery, Elisabeth-TweeSteden Hospital, 5022 GC Tilburg, The Netherlands Department of Cognitive Neuropsychology, Tilburg University, 5037 AB Tilburg, The Netherlands
Karin Gehring
Department of Neurosurgery, Elisabeth-TweeSteden Hospital, 5022 GC Tilburg, The Netherlands Department of Cognitive Neuropsychology, Tilburg University, 5037 AB Tilburg, The Netherlands
Wouter De Baene
Department of Cognitive Neuropsychology, Tilburg University, 5037 AB Tilburg, The Netherlands
Wilco Henricus Maria Emons
Department of Methodology and Statistics, Tilburg University, 5037 AB Tilburg, The Netherlands
Geert-Jan Maria Rutten
Department of Neurosurgery, Elisabeth-TweeSteden Hospital, 5022 GC Tilburg, The Netherlands
Margriet Maria Sitskoorn
Department of Cognitive Neuropsychology, Tilburg University, 5037 AB Tilburg, The Netherlands
*Correspondence and reprint requests to: Sophie D. van der Linden, Department of Cognitive Neuropsychology, Tilburg University, Room S201B, P.O. Box 90153, 5000 LE, Tilburg, The Netherlands. E-mails:;
Rights & Permissions [Opens in a new window]



This study aimed to examine: (1) patient–proxy agreement on executive functioning (EF) of patients with primary brain tumors, (2) the relationships between patient- and proxy-report with performance-based measures of EF, and (3) the potential influence of performance-based measures on the level of agreement.


Meningioma and low-grade glioma patients and their informal caregivers completed the Behavior Rating Inventory of Executive Function (BRIEF-A) 3 months after surgery. The two index scores of the BRIEF-A, Behavioral Regulation and Metacognition, were evaluated. Mean scores of patients and proxies were compared with normative values and with each other. Patient–proxy agreement was evaluated with Lin’s concordance correlation coefficients (CCCs) and Bland–Altman plots. Pearson correlation coefficients between reported EF and performance-based measures of EF were calculated. Multiple regression analysis was used to evaluate the potential influence of test performance on differences in dyadic reports.


A total of 47 dyads were included. Patients reported significantly more problems on the Metacognition Index compared to norms, and also in comparison with their proxies. Effect sizes indicated small differences. Moderate to substantial agreement was observed between patients and proxies, with CCCs of 0.57 and 0.61 for Metacognition and Behavioral Regulation, respectively. Correlations between reported EF and test performance ranged between −0.37 and 0.10. Dyadic agreement was not significantly influenced by test performance.


Patient–proxy agreement was found to be moderate. No clear associations were found between reported EF and test performance. Future studies should further explore the existing and new methods to assess everyday EF in brain tumor patients.

Regular Research
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (, which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright © INS. Published by Cambridge University Press, 2019


Over the last few decades, patients and their family members are increasingly being involved in care decisions (Colligan, Metzler, & Tiryaki, Reference Colligan, Metzler and Tiryaki2017; Kane, Halpern, Squiers, Treiman, & McCormack, Reference Kane, Halpern, Squiers, Treiman and McCormack2014). To be able to make informed decisions, patients need sufficient cognitive abilities and self-awareness, but these functions can be disturbed in patients with brain disorders (e.g., Day et al., Reference Day, Gillespie, Rooney, Bulbeck, Zienius, Boele and Grant2016; Kerrigan, Erridge, Liaquat, Graham, & Grant, Reference Kerrigan, Erridge, Liaquat, Graham and Grant2014; Sunderaraman & Consentino, Reference Sunderaraman and Consentino2017). More specifically, evidence emerging from patient studies, imaging research, and behavioral paradigms suggests that different components of executive functioning (EF) play a crucial role in decision-making and that impairments in EF seem to be associated with risky and ambiguous decision-making (Del Missier, Mäntylä, & Bruine de Bruin, Reference Del Missier, Mäntylä and Bruine de Bruin2010; Gleichgerrcht, Ibáñez, Roca, Torralya, & Manes, Reference Gleichgerrcht, Ibáñez, Roca, Torralya and Manes2010; Schiebener, Wegmann, Gathmann, Laier, Pawlikowski, & Brand, Reference Schiebener, Wegmann, Gathmann, Laier, Pawlikowski and Brand2014). Executive functions include several higher-order cognitive processes that enable people to control and regulate their own behavior. Key executive functions are inhibition (i.e., deliberate overriding of dominant responses), cognitive flexibility (i.e., shifting between different tasks or mental sets), and working memory (i.e., constant monitoring and updating of retained information) (Diamond & Ling, Reference Diamond and Ling2016; Karr et al., Reference Karr, Areshenkoff, Rast, Hofer, Iverson and Garcia-Barrera2018; Miyake & Friedman, Reference Miyake and Friedman2012). With respect to decision-making, executive functions are necessary for, among others, assessing probabilities, categorizing options, selecting decision-making strategies, and using feedback to revise strategies if necessary (Schiebener et al., Reference Schiebener, Wegmann, Gathmann, Laier, Pawlikowski and Brand2014).

Impairments in EF are among the most pronounced cognitive deficits in patients with primary brain tumors (Gehrke, Baisley, Sonck, Wronski, & Feuerstein, Reference Gehrke, Baisley, Sonck, Wronski and Feuerstein2013; Meskal, Gehring, Rutten, & Sitskoorn, Reference Meskal, Gehring, Rutten and Sitskoorn2016; Talacchi, Santini, Savazzi, & Gerosa, Reference Talacchi, Santini, Savazzi and Gerosa2011; Van Kessel, Baumfalk, Van Zandvoort, Robe, & Snijders, Reference Van Kessel, Baumfalk, Van Zandvoort, Robe and Snijders2017). Impairments in EF can have a negative impact on patients’ everyday lives but also on the lives of the people in their environment (Aaronson et al., Reference Aaronson, Taphoorn, Heimans, Postma, Gundy, Beute, Slotman and Klein2011; Nugent et al., Reference Nugent, Weimer, Choi, Bradley, Bender, Ryan, Gardner and Sherwood2014; Sterckx et al., Reference Sterckx, Coolbrandt, Dierckx de Casterle, Van den Heede, Decruyenaere, Borgenon, Mees and Clement2013). It is important to identify problems with EF at an early stage, so we can refer to intervention programs in time, including, for example, Goal Management Training (Richard et al., Reference Richard, Bernstein, Mason, Laperriere, Maurice, Millar, Shultz, Berlin and Edelstein2019).

From research and clinical practice, it is known that patients who experience cognitive complaints do not necessarily show lower scores on neuropsychological tests. And vice versa, if patients demonstrate lower test performance, they do not always experience cognitive complaints in their daily life (Gehring, Taphoorn, Sitskoorn, & Aaronson, Reference Gehring, Taphoorn, Sitskoorn and Aaronson2015; Hutchinson, Hosking, Kichenadasse, Mattiske, & Wilson, Reference Hutchinson, Hosking, Kichenadasse, Mattiske and Wilson2012). Therefore, both neuropsychological tests and self-report questionnaires on EF are often used to get a full picture of a person’s EF. When patients are unable to complete assessments, due to, for example, language problems or paralysis, information from informal caregivers can be of added value. When using proxy-report as an addition to, or as a substitute for self-report, it is crucial to know how these measures are related.

A few studies in brain tumor patients evaluated patient–proxy agreement with respect to patients’ quality of life. Overall, moderate to high levels of concordance were observed, if patients did not suffer from cognitive dysfunction (Brown et al., Reference Brown, Decker, Rummans, Clark, Frost, Ballman, Arusell and Buckner2008; Ediebah et al., Reference Ediebah, Reijneveld, Taphoorn, Coens, Zikos, Aaronson, Heimans, Bottomley and Klein2017; Giesinger et al., Reference Giesinger, Golser, Erharter, Kemmler, Schauer-Maurer, Stockhammer, Muigg, Hutterer, Rumpold and Holzner2009; Sneeuw et al., Reference Sneeuw, Aaronson, Osoba, Muller, Hsu, Yung, Brada and Newlands1997). Also, high congruence was found in the study of Armstrong et al. (Reference Armstrong, Wefel, Gning, Acquave, Vera-Bolanos, Gilbert, Cleeland and Mendoza2012) on a brain tumor symptom checklist MD Anderson Symptom Inventory – Brain Tumor Module that was administered in 115 brain tumor patients and their caregivers. Additionally, a study in 60 brain tumor patients showed high level of agreement between patients and proxies on the Dexamethasone Symptom Questionnaire-Chronic (Agar et al., Reference Agar, Koh, Gibbs, Barnes, Hovey, Livingstone, Sawkins, Chye, Lovell, Clark, Vardy and King2016). By contrast, the study of Rooney and colleagues demonstrated substantial disagreement between glioma patients and their proxies in the evaluation of depressive symptoms (Rooney et al., Reference Rooney, McNamara, Mackinnon, Fraser, Rampling, Carson and Grant2013), with proxies reporting more depressive symptoms than patients themselves. These authors suggested that patients and proxies generally seem to agree on objective signs and overt behavior of patients, but that agreement is limited on subjective, internal symptoms, such as mood and emotional functioning (Rooney et al., Reference Rooney, McNamara, Mackinnon, Fraser, Rampling, Carson and Grant2013).

It is important to know how brain tumor patients perceive their EF and how this relates to the experience of their informal caregivers and to performance-based measures of EF. In the current study, the main objective was to evaluate the level of agreement between patient-report and proxy-report of patients’ EF. Second, associations of reported measures of EF with performance-based measures of EF (i.e., neuropsychological tests) were examined, as well as the influence of these performance-based measures on the level of patient–proxy agreement. Since from previous studies it is known that brain tumor patients suffer from executive deficits, we hypothesized that this may also influence their self-assessment, potentially resulting in disagreement among patients and proxies. We also expected that lower patients’ scores on performance-based measures would be associated with greater patient–proxy discrepancies.


Design and Procedure

Data of this study were gathered at preintervention assessments in a feasibility study (Van der Linden, Sitskoorn, Rutten, & Gehring, Reference Van der Linden, Sitskoorn, Rutten and Gehring2018a) and a randomized controlled trial (RCT; Van der Linden, Sitskoorn, Rutten, & Gehring, Reference Van der Linden, Sitskoorn, Rutten and Gehring2018b) on cognitive rehabilitation in brain tumor patients, initiated at the Elisabeth-TweeSteden Hospital Tilburg, The Netherlands. This research was conducted in accordance with the Declaration of Helsinki (World Medical Association, 2013). The cognitive rehabilitation study (i.e., feasibility study and RCT) was approved by the local ethical review board (METC Brabant: NL 51152.028.14) and registered with (NTC03373487) and the Dutch Trial Register (NTR5392).

Before and 3 months after surgery, patients underwent neuropsychological assessment (NPA) as part of usual clinical care in the hospital. Directly after the 3-month NPA, patients who participated in the RCT were randomized to an intervention or control condition. During an appointment with the researcher 3 months after surgery, participants and proxies completed the adult version of the Behavior Rating Inventory of Executive Function (BRIEF-A).


Adult patients who were scheduled for resective surgery for a meningioma or low-grade glioma were invited to participate in one of the cognitive rehabilitation studies. Patients undergoing only biopsy were not eligible. Exclusion criteria were tumor resection in the last year; chemotherapy or radiotherapy in the last 2 years; presence/history of progressive neurological disease; severe psychiatric disorder or substance abuse; diagnosis of acute neurological or mild psychiatric disorders in the last 2 years (e.g., cerebrovascular accident); multiple (>1) tumors; lack of basis proficiency in Dutch; Karnofsky Performance Score below 70; IQ below 85 or (very) low cognitive skills; and insufficient reading skills, visual impairment, or motor impairment. Patients were also excluded if they had severe surgery-related complications or if they were referred to formal cognitive rehabilitation.

Study participants were invited to involve an informal caregiver (e.g., spouse, family member, or close friend) to the study. No exclusion criteria were applied to the proxies. If patients agreed, informed consent was obtained from the proxies. Only participants who involved an informal caregiver were included in the present analyses.


The BRIEF-A assesses problems in an adult’s EF as experienced in his or her daily life (Roth, Isquith, & Gioia, Reference Roth, Isquith and Gioia2005). The self-report and informant-report versions are available, and both questionnaires consist of 75 items. Patients and proxies were asked to assess the extent to which certain behavior of the patient occurred during the past month. The informant version of the BRIEF-A uses a proxy–proxy perspective: proxies need to report on how they think the patient is functioning. Answers are given on a three-point scale (i.e., never = 1, sometimes = 2, often = 3). Two index scores, Behavioral Regulation and Metacognition, can be calculated based on 30 and 40 items, respectively. Behavioral Regulation contains four subscales (i.e., Inhibit, Shift, Emotional Control, and Self-monitor), and Metacognition has five subscales (i.e., Initiate, Working Memory, Plan/Organize, Task-monitor, and Organization of Materials). Three validity scales (i.e., Negativity, Infrequency, and Inconsistency) were checked, and in case of invalid responses, patients were excluded from analyses. Cases were also excluded if there were ≥5 missing answers. Missing values were handled in accordance with the manual (Scholte & Noens, Reference Scholte and Noens2011) and scores of “1” were imputed in case of missing values. Raw scores on the indices were converted into T scores, using representative norms of the Dutch/Flemish population (self-report: n = 1600, informant-report: n = 1082) (Scholte & Noens, Reference Scholte and Noens2011). These standardized T scores were converted into Z scores and also reversed, to coincide with performance-based scores, so that lower Z scores indicate lower subjective EF. Z scores ≤−1.5 were considered as low (Scholte & Noens, Reference Scholte and Noens2011). Psychometric properties of the Dutch version of the self-report and informant-report are good, with Cronbach’s α above 0.90 and intraclass correlation coefficients (ICCs) between 0.73 and 0.81, for the two indices (Scholte & Noens, Reference Scholte and Noens2011).

Three neuropsychological tests from the 3-month assessment were included to measure different aspects of performance-based EF. Response inhibition and cognitive flexibility were measured, respectively, with the Stroop Test and the Shifting Attention Test (SAT) of the computerized test battery Central Nervous System Vital Signs (CNS VS LCC, Morrisville, NC, USA). Regarding validity and reliability, Gualtieri and Johnson (Reference Gualtieri and Johnson2006) concluded that psychometric characteristics of the CNS VS’ tests were comparable with the conventional tests on which they were based (Gualtieri & Johnson, Reference Gualtieri and Johnson2006). Additionally, previous studies demonstrated sufficient sensitivity of the CNS VS in the detection of (mild) cognitive deficits and change in cognitive function in patients with brain tumors (Meskal et al., Reference Meskal, Gehring, van der Linden, Rutten and Sitskoorn2015; Rijnen et al., Reference Rijnen, Meskal, Bakker, De Baene, Rutten, Gehring and Sitskoorn2019; Van Loenen et al., Reference Van Loenen, Rijnen, Bruijn, Rutten, Gehring and Sitskoorn2018). Furthermore, the Digit Span Backward of the Wechsler Adult Intelligence Scale was used to assess working memory (Wechsler, Reference Wechsler2015). Patients’ scores on the neuropsychological tests were compared to representative, recently collected Dutch norms (Rijnen et al., Reference Rijnen, Meskal, Emons, Campman, Van Der Linden, Gehring and Sitskoorn2017; Rijnen, Van der Linden, Emons, Sitskoorn, & Gehring, Reference Rijnen, Van der Linden, Emons, Sitskoorn and Gehring2018; CAR study A,, reference nr. NCT02953756). Patient test scores were corrected for sex, age, educational level, and practice effects and converted into Z scores, with lower scores indicating worse EF performance. Again, Z scores ≤−1.5 were considered as low.

Data Analysis

Descriptive statistics of the sample were calculated and compared with patients who were not included in the current study (i.e., patients who did not include a proxy in the cognitive rehabilitation study) using independent-samples t-tests and Chi-square tests.

Mean scores of patients and proxies were compared with normative values [M = 0, standard deviation (SD) = 1] using two-tailed one-sample z-tests. The standardized mean differences between patients and the normative values, and those between proxies and normative values, are interpreted as Glass’ delta effect sizes, with 0.20–0.49 indicating small effects, 0.50–0.79 medium effects, and ≥0.80 reflecting large effects (Glass, McGaw, & Smith, Reference Glass, McGaw and Smith1981). Subsequently, means of the patients were compared with means of the proxies using two-tailed paired-sample t-tests, to investigate potential systematic differences, and effect sizes [Cohen’s d (Cohen, Reference Cohen1988)] were calculated.

To examine the level of concordance between patient- and proxy-reports, Lin’s concordance CCCs (Lin, Reference Lin1989, Reference Lin2000) were calculated for the two indices of the BRIEF-A. CCCs were interpreted as follows: 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and 0.81–1.00 almost perfect to perfect agreement (Altman, Reference Altman1991). As described in the manual of the BRIEF-A (Scholte & Noens, Reference Scholte and Noens2011), in order to be able to consider patient-report and proxy-report as interchangeable, ICCs of at least 0.60 are required. Bland–Altman plots were generated, plotting the difference between measurements (i.e., patient score – proxy score) against the mean of the two measurements [i.e., (mean patient score + mean proxy score)/2] (Bland & Altman, Reference Bland and Altman1999). By visualizing these differences, patterns of agreement, types of bias, and outliers can be further identified. We computed 95% confidence envelopes around the mean difference (i.e., mean ± 1.96 × SD of difference scores), which shows the range of difference scores that covers 95% of the observations. Patient and proxy scores can be used interchangeably if the range is small given the application envisaged. Furthermore, patient and proxy scores on the two indices were dichotomized denoting low (Z score ≤ −1.5) or normal (Z score > −1.5) EF. Percentages of cases below the cut-off were calculated, and percentages of dyadic agreement about the presence or absence of reported EF impairment were determined.

Furthermore, performance-based measures of EF were examined, with standardized mean scores of the patients being compared to normative values (M = 0, SD = 1) with z-tests. The relationship between performance-based measures of EF and reported EF was explored by calculating Pearson’s product moment correlation coefficients between standardized performance scores and index scores of both patients and proxies. Correlation coefficients between 0.10 and 0.29 were considered as small, 0.30–0.49 were considered as medium, and 0.50–1.0 reflected large correlation coefficients (Cohen, Reference Cohen1988).

Finally, the association between objective EF and the level of agreement was examined using multiple regression analysis. The observed difference scores (patient self-report – proxy-report score) on the two BRIEF-A indices served as the dependent variables. All independent variables (standardized scores on the Stroop Test, SAT and Digit Span Backward of the 3-month NPA) were entered at the same time for each of the two analyses. Assumptions were evaluated using Durbin–Watson tests (Durbin & Watson, Reference Durbin and Watson1951), scatterplots and histograms of residuals, variance inflation indices (not exceeding 0.80 and 10; Field, Reference Field2009), and Cook’s distances (≤1; Cook & Weisberg, Reference Cook and Weisberg1982).

Statistical analyses were conducted using SPSS Statistics (version 24.0, IBM corporation, Armonk, NY, USA). All statistical tests were performed at an α level of 0.05.


Participants’ Characteristics

Three months after surgery, 47 out of 75 participants (63%) of the cognitive rehabilitation studies chose to involve a proxy. Regarding age, years of education, sex, tumor histology, and patient-reported EF, no significant differences were observed between patients who did include a proxy and those who did not (p-values > .05). Included patients (n = 47) had a mean age of 51 years and 26 patients (55%) were female. Twenty-eight patients (60%) were diagnosed with a meningioma and 19 (40%) with a low-grade glioma. In 26 patients (55%), the tumor was located in the frontal lobe. In the large majority (94%), the relationship of the proxy to the patient was partner. Three patients involved their mother, sister, or brother.

All BRIEF-A questionnaires met the validity criteria. There were no questionnaires with ≥5 missing values. In this study, Cronbach’s α of the indices ranged between 0.90 and 0.96 in patients and proxies, indicating good internal consistency (Gliem & Gliem, Reference Gliem and Gliem2003).

Patient-Report and Proxy-Report of Patients’ EF

Patients scored on average significantly lower on the Metacognition Index in comparison with normative values (M = −0.47, SD = 1.04, z = −3.25, p = .001). Glass’ delta effect size of 0.47 suggested a small effect. On the Behavioral Regulation Index, the average patient score did not significantly differ from normative values (M = 0.02, SD = 1.01, z = 0.13, p = .896). There were no significant differences between proxy evaluations of patients’ EF and normative values for both indices (Table 1).

Table 1. Mean levels of reported EF: Comparison with normative values and differences between patients (n = 47) and proxies (n = 47)

a Lower scores indicate lower reported EF/more experienced problems.

b Sample means compared to normative values (Mean = 0, SD = 1).

* p < .05

When comparing mean patient scores with mean proxy scores using paired-sample t-tests (Table 1), a significant mean difference was observed on the Metacognition Index. Patients reported significantly lower EF (i.e., more concerns) than their proxies (t(46) = −2.40, p = .021, Cohen’s d = −0.35), with Cohen’s d indicating a small effect. On the Behavioral Regulation Index, no significant mean difference was observed between patients and proxies (t(46) = −0.83, p = .703, Cohen’s d = −0.06).

On the Metacognition and Behavioral Regulation Index, 23% and 9% of the patients scored below the cut-off (Z score ≤ −1.5), respectively. According to the report of the proxies, 13% and 6% of the patients would score below the cut-off of the Metacognition Index and Behavioral Regulation Index, respectively.

Level of Agreement Between Patients and Proxies

Lin’s CCCs, listed in Table 2, indicated moderate to substantial agreement among patients and proxies (CCCs of 0.57 for Metacognition and 0.61 for Behavioral Regulation). Figure 1 shows the Bland–Altman plots. The lower and upper limits were −2.11 and 1.47 for Metacognition and −1.76 and 1.67 for Behavioral Regulation. The Bland–Altman plots showed that 93.6% of the data points (i.e., 44/47) lied within these limits of agreement. Inspection of the plots revealed no particular trends in agreement with respect to the level of reported EF. When the cut-off for reported EF impairment (i.e., Z score ≤−1.5) was applied, congruence was observed in 94% (44/47) dyads on Behavioral Regulation and in 77% (36/47) dyads on Metacognition (Table 3). Based on the dichotomized data of the Metacognition Index, 8 proxies of the 11 nonagreeing dyads did not rate the patient’s EF as impaired, whereas the patient did (Table 3).

Table 2. Level of agreement between patient-report and proxy-report of EF

a CCC = Concordance correlation coefficient. 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, 0.81–1.00 almost perfect to perfect agreement (Altman, Reference Altman1991).

b Limits of agreement: Meandiff ± 1.96*SDdiff.

Fig. 1. Bland–Altman plot for agreement on the (a) Behavioral Regulation Index, and (b) Metacognition Index. Note: Dashed lines represent the upper and lower limits of the 95 % confidence intervals.

Table 3. Dyadic agreement on the presence or absence of reported EF impairment

Note: Z scores ≤ −1.5 were considered as low (Scholte & Noens, Reference Scholte and Noens2011).

Performance-Based Versus Reported EF

Compared to normative values of the general population, patients scored significantly lower on the Stroop Test (M = −0.91, SD = 1.71, z = −6.25, p < .001), the SAT (M = −1.09, SD = 1.34, z = −7.26, p < .001), and Digit Span Backward (M = −0.84, SD = 0.96, z = −75, p < .001) 3 months after surgery (Table 3).

Using a cut-off of Z ≤ −1.5, 15 patients (32%) had low scores on the Stroop Test, 14 (30%) on the SAT, and 12 (26%) patients scored low on Digit Span Backward. In total, 31 patients (66%) had Z scores ≤ −1.5 on one or more tests.

Correlation coefficients of neuropsychological tests scores with patient-report and proxy-report are listed in Table 4. Overall, nonsignificant very small correlations were observed, with r ranging from −0.29 to 0.10, except for Digit Span Backward, where significant negative correlations were observed with patient-report of EF (r of −0.33 and −0.37 for Behavioral Regulation and Metacognition, respectively).

Table 4. Performance-based measures of EF: Mean scores compared to normative values and correlations with reported EF

a Sample means compared to normative values (Mean = 0, SD = 1).

b Correlation coefficients between 0.10 and 0.29 were considered as small, 0.30–0.49 as medium, and 0.50–1.0 as large (Cohen, Reference Cohen1988).

* p < .05

Association Between EF Test Performance and Patient–Proxy Differences

Multiple regression analyses indicated that scores on performance-based measures of EF (Stroop Test, SAT, and Digit Span Backward) had no significant influence on the patient–proxy difference scores of Behavioral Regulation [F (3,47) = 0.109, p = .954, R 2 = 0.008] and Metacognition [F (3,47) = 0.924, p = .412, R 2 = 0.065].


The present study was performed to evaluate patient–proxy agreement with respect to report of EF in patients with meningioma and low-grade glioma. In addition, the relationship between reported EF with performance-based measures of EF was investigated, as was the effect of EF test performance on the level of agreement. Since patients are increasingly involved in health-care decisions, it is important to gain more insight in their metacognitive abilities and how their experiences relate to experiences of their informal caregivers and to performance-based measures of EF.

Firstly, compared to normative data, lowered patient scores were observed on reported EF (Metacognition Index) and on EF test performance (Stroop Test, SAT and Digit Span Backward). These findings contribute to the large body of evidence showing that patients with primary brain tumors experience complaints and deficits in EF after neurosurgery. Moreover, it stresses the importance of careful assessment of EF. Since problems in EF can have major impact on everyday life of patients and health-care decisions, attention should be paid to EF of patients throughout their disease trajectory, in both clinical care and research settings.

In this study, patients reported significantly more problems than their proxies on the Metacognition Index. A similar mismatch between patients and partners has been reported in patients with dementia. It has been suggested that in the early stages of the disease, patients experience subtle cognitive complaints that are not always noticed by their partners (e.g., Fogarty, Almklov, Borrie, Wells, & Roth, Reference Fogarty, Almklov, Borrie, Wells and Roth2017; Rueda et al., Reference Rueda, Lau, Saito, Harvey, Risacher, Aisen, Petersen, Saykin and Farias2015). However, when the disease progresses, diminished disease awareness often arises (Sunderaraman & Consentino, Reference Sunderaraman and Consentino2017), leading to the reversed pattern: partners often report more problems than patients (Rueda et al., Reference Rueda, Lau, Saito, Harvey, Risacher, Aisen, Petersen, Saykin and Farias2015). Our sample consists of brain tumor patients who have slow-growing tumors and a relatively favorable prognosis. It is possible that self-reported EF problems may decrease as the disease progresses and that studies in patients with high-grade glioma or metastases with less favorable prognosis would render different results. However, more research is needed to further explore this hypothesis. Furthermore, no systematic differences between patients and proxies were observed in group means of the Behavioral Regulation Index. Possibly, this is due to the fact that this index generally contains more items that are aimed at the report of concrete, overt behavior (e.g., “I drum my fingers or wiggle my legs”; “I start things on the last moment”), which makes it easier for proxies to assess patients’ functioning (Rooney et al., Reference Rooney, McNamara, Mackinnon, Fraser, Rampling, Carson and Grant2013).

Regarding the degree of concordance as measured with Lin’s CCCs, the findings indicate moderate to substantial agreement among dyads on patients’ EF, with CCCs of 0.61 and 0.57 for the Behavioral Regulation Index and Metacognition Index. Based on these CCCs, we can conclude that proxy-reports are a reasonable estimate of patient-reported EF. The additional information from the Bland–Altman plots and cross tables suggests less agreement between patients and proxies on the Metacognition Index compared with the Behavioral Regulation Index. This can also be explained by the more concrete and observable symptoms assessed by the Behavioral Regulation Index, as described above.

Although patients showed lower EF test scores, no clear associations were observed between patients’ self-reports and proxy-reports with performance-based measures of EF. Overall, (very) small correlations were observed in this study, which is in line with previous research in brain tumor patients and other neurologic populations (Lanni et al., Reference Lanni, Ross, Higginson, Dressler, Sigvardt, Zhang, Malhado-Chang and Disbrow2014; Pranckeviciene, Deltuva, Tamasauskas, & Bunevicius, Reference Pranckeviciene, Deltuva, Tamasauskas and Bunevicius2017; Rabin, Wang, Katz, Derby, Buschke, & Lipton, Reference Rabin, Wang, Katz, Derby, Buschke and Lipton2012; Schiehser et al., Reference Schiehser, Delis, Filoteo, Delano-Wood, Han, Jak, Drake and Bondi2011). Based solely on the absence of associations of patients’ self-report and performance-based results, one might conclude that patients are not able to estimate the objective level of their EF, possibly due to a lack of insight or reduced cognitive capabilities. However, the current findings render this explanation more unlikely, since we found that there was concordance between patient and proxies, and proxy-report was also not significantly correlated with patients’ test performance. Thus, there is a mismatch between reported EF and test performance, but we do not know which of the measures is the most accurate.

There are many difficulties involved in measuring EF, and an objective gold standard is missing. Based on the finding that reported EF was not correlated with EF test performance in both clinical and nonclinical samples, Toplak, West, and Stanovich (Reference Toplak, West and Stanovich2013) concluded in their review that rating measures of EF and performance-based measures of EF appear to capture different underlying mental constructs and therefore, they cannot be interpreted as equivalent nor be used interchangeably. Tests often focus on a particular domain of EF and are administered in a quiet and controlled environment on a certain moment in time, while EF in everyday life requires complex, integrated, and dynamic skills. Ecological validity of performance-based measures of EF is limited, and thus will not fully capture experienced problems in patients’ day-to-day functioning. On the other hand, several studies demonstrated that self-reported EF is associated with psychological variables, such as personality and mood (Buchanan, Reference Buchanan2016; Meltzer et al., Reference Meltzer, Kapoor, Fogel, Elbulok-Charcape, Roth, Katz and Rabin2016; Pranckeviciene et al., Reference Pranckeviciene, Deltuva, Tamasauskas and Bunevicius2017), and that self-report response biases can affect validity and reliability (Althubaiti, Reference Althubaiti2016).

In addition to neuropsychological tests, Noll, Bradshaw, Weinberg, and Wefel (Reference Noll, Bradshaw, Weinberg and Wefel2018) made use of measures of functional independence (i.e., the Functional Independence Measure and the Karnofsky Performance Status) and found significant correlations between functional independence and performance-based measures of EF (r between 0.28 and 0.42). A potential promising alternative is using an Experience Sampling Method to measure self-reported EF, at multiple times throughout the day, actually integrated in the daily lives of patients (Moore, Schwendsen, & Depp, Reference Moore, Schwendsen and Depp2017; Myin-Germeys et al., Reference Myin-Germeys, Kasanova, Vaessen, Vachon, Kirtley, Viechtbauer and Reininghaus2018). By gathering real-time data, some biases that occur through reporting in retrospect on conventional self-report questionnaires can be avoided. Furthermore, virtual environments are increasingly being used to enhance neuropsychological testing, so that they better reflect situations patients face in the outside world (Parsey & Schmitter-Edgecombe, Reference Parsey and Schmitter-Edgecombe2013; Parsons, Carlew, Magtoto, & Stonecipher, Reference Parsons, Carlew, Magtoto and Stonecipher2015). These relatively new assessment techniques, with potentially improved validity and reliability, could also be explored in future studies in patients with primary brain tumors.

In this study, no effect of executive test performance on dyadic agreement was found, which is in contrast with our expectations based on previous research on health-related quality of life in brain tumor patients (Brown et al., Reference Brown, Decker, Rummans, Clark, Frost, Ballman, Arusell and Buckner2008; Ediebah et al., Reference Ediebah, Reijneveld, Taphoorn, Coens, Zikos, Aaronson, Heimans, Bottomley and Klein2017; Sneeuw et al., Reference Sneeuw, Aaronson, Osoba, Muller, Hsu, Yung, Brada and Newlands1997) and studies in other populations (Hart, Whyte, Kim, & Vaccaro, Reference Hart, Whyte, Kim and Vaccaro2005; Howland, Allan, Carlton, Tatsuoka, Smyth, & Sajatociv, Reference Howland, Allan, Carlton, Tatsuoka, Smyth and Sajatociv2017). It should be noted that the instruments used to assess cognitive performance differed across studies (e.g., the Mini Mental State Examination or extensive neuropsychological test batteries), as well as the statistical procedures used to examine the possible effect of test performance on dyadic agreement. Moreover, it is possible that deficits in EF are more likely to relate to patient–proxy agreement when patients report less problems than proxies (potentially resulting from impaired insight), instead of more problems, as was observed in our sample of meningioma and low-grade glioma patients.

Besides EF test performance, other factors may also have modulated the level of agreement between patients and proxies, including patient- or disease-related factors, proxy-related factors or factors related to the assessment of EF. The small sample size of this study limited our possibilities to explore the potential influence of, for example, tumor location or tumor histology, with valid statistical testing. Also, we had little information available on the informal caregivers or the quality of the relationship with the patient, restricting the possibility to take this into account. Moreover, informal caregivers were involved in the cognitive rehabilitation study, only if patients (and proxies) were willing to. Possibly, this may have led to a selection bias (i.e., sampling bias), namely, the inclusion of dyads who are highly motivated and might agree on the patient’s functioning more than dyads not included in this study. In addition, all patients in this study were participants in a cognitive rehabilitation trial, with strict in- and exclusion criteria, which may also hamper generalizability of the findings to the low-grade glioma and meningioma patient population as a whole.

In conclusion, patients with primary brain tumors experience executive deficits after neurosurgery. Congruence between brain tumor patients and their proxies was observed in the evaluation of patients’ EF, but correlations with performance-based measures of EF were low. Since problems in EF are commonly observed in brain tumor patients and can lead to lowered quality of life, it is important to assess EF carefully throughout the disease trajectory, using both questionnaires and performance-based measures of EF. At the same time, research on the use of innovative methods to assess EF in patients with primary brain tumors should be expanded.


This work was supported by the Dutch organization for health research and innovation (ZonMw) (grant number 842003009).


The authors declare they have no conflict of interests.



Aaronson, N.K., Taphoorn, M.J.B., Heimans, J.J., Postma, T.J., Gundy, C.M., Beute, G.N., Slotman, B.J., & Klein, M. (2011). Compromised health-related quality of life in patients with low-grade glioma. Journal of Clinical Oncology, 29(33), 44304435. doi: 10.1200/JCO.2011.35.5750 CrossRefGoogle ScholarPubMed
Agar, M., Koh, E.S., Gibbs, E., Barnes, E.H., Hovey, E., Livingstone, A., Sawkins, K., Chye, R., Lovell, M.R., Clark, K., Vardy, J., & King, M. (2016). Validating self-report and proxy-reports of the Dexamethasone Symptom Questionnaire-Chronic for the evaluation of longer-term corticosteroid toxicity. Supportive Care in Cancer, 24, 12091218. doi: 10.1007/s00520-015-2897-0 CrossRefGoogle ScholarPubMed
Althubaiti, A. (2016). Information bias in health research: Definition, pitfalls and adjustment methods. Journal of Multidisciplinaire Healthcare, 9, 211217. doi: 10.2147/JMDH.S104807 CrossRefGoogle ScholarPubMed
Altman, D.G. (1991). Practical Statistics for Medical Research. London, UK: Chapman and Hall.Google Scholar
Armstrong, T.S., Wefel, J.S., Gning, I., Acquave, A., Vera-Bolanos, E., Gilbert, M.R., Cleeland, C.S., & Mendoza, T. (2012). Congruence of primary brain tumor patients and caregiver symptom report. Cancer, 118, 50265037. doi: 10.1002/cncr.27483 CrossRefGoogle ScholarPubMed
Bland, J.M. & Altman, D.G. (1999). Measuring agreement in method comparison studies. Statistical Methods in Medical Research, 8, 135–60. doi: 10.1191/096228099673819272 CrossRefGoogle ScholarPubMed
Brown, P.D., Decker, P.A., Rummans, T.A., Clark, M.M., Frost, M.H., Ballman, K.V., Arusell, R.M., & Buckner, J.C. (2008). A prospective study of quality of life in adults with newly diagnosed high-grade gliomas: Comparison of patient and caregiver ratings of quality of life. American Journal of Clinical Oncology, 31, 163168. doi: 10.1097/COC.0b013e318149f1d3 CrossRefGoogle ScholarPubMed
Buchanan, T. (2016). Self-report measures of executive function problems correlate with personality, not performance-based executive function measures, in nonclinical samples. Psychological Assessment, 28(4), 372385. doi: 10.1037/pas0000192 CrossRefGoogle Scholar
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Colligan, E., Metzler, A., & Tiryaki, E. (2017). Shared decision-making in multiple sclerosis. Multiple Sclerosis Journal, 23(2), 185190. doi: 10.1177/1352458516671204.CrossRefGoogle ScholarPubMed
Cook, D.R. & Weisberg, S. (1982). Residuals and Influence in Regression. New York, NY: Chapman & Hall.Google Scholar
Day, J., Gillespie, D.C., Rooney, A.G., Bulbeck, H.J., Zienius, K., Boele, F., & Grant, R. (2016). Neurocognitive deficits and neurocognitive rehabilitation in adult brain tumors. Current Treatment Options in Neurology, 18, 22. doi: 10.1007/s11940-016-0406-5 CrossRefGoogle ScholarPubMed
Del Missier, F., Mäntylä, T., & Bruine de Bruin, W. (2010). Executive functions in decision making: An individual differences approach. Thinking & Reasoning, 16(2), 6997. doi: 10.1080/13546781003630117 CrossRefGoogle Scholar
Diamond, A. & Ling, D.S. (2016). Conclusions about interventions, programs, and approaches for improving executive functions that appear justified and those that, despite much hype, do not. Developmental Cognitive Neuroscience, 18, 3448. doi: 10.1016/j.dcn.2015.11.005 CrossRefGoogle ScholarPubMed
Durbin, J. & Watson, G.S. (1951). Testing for serial correlation in least squares regression II. Biometrika, 38(1/2), 159177. doi: 10.2307/2332325 CrossRefGoogle ScholarPubMed
Ediebah, D.E., Reijneveld, J.C., Taphoorn, M.J., Coens, C., Zikos, E., Aaronson, N.K., Heimans, J.J., Bottomley, A., & Klein, M. (2017). Impact of neurocognitive deficits on patient-proxy agreement regarding health-related quality of life in low-grade glioma patients. Quality of Life Research, 26, 869880. doi: 10.1007/s11136-016-1426-z CrossRefGoogle ScholarPubMed
Field, A. (2009). Discovering Statistics Using SPSS (3rd ed.). London, UK: SAGE publications Ltd.Google Scholar
Fogarty, J., Almklov, E., Borrie, M., Wells, J., & Roth, R.M. (2017). Subjective rating of executive functions in mild Alzheimer’s disease. Aging & Mental Health, 21, 11841191. doi: 10.1080/13607863.2016.1207750 CrossRefGoogle ScholarPubMed
Gehring, K., Taphoorn, M.J., Sitskoorn, M.M., & Aaronson, N.K. (2015). Predictors of subjective versus objective cognitive functioning in patients with stable grades II and III glioma. Neuro-Oncol Practice, 2(1), 2031. doi: 10.1093/nop/npu035 CrossRefGoogle ScholarPubMed
Gehrke, A.K., Baisley, M.C., Sonck, A.L., Wronski, S.L., & Feuerstein, M. (2013). Neurocognitive deficits following primary brain tumor treatment: Systematic review of a decade of comparative studies. Journal of Neuro-Oncology, 115(2), 135142. doi: 10.1007/s11060-013-1215-2 CrossRefGoogle ScholarPubMed
Giesinger, J.M, Golser, M., Erharter, A., Kemmler, G., Schauer-Maurer, G., Stockhammer, G., Muigg, A., Hutterer, M., Rumpold, G., & Holzner, B. (2009). Do neuro-oncological patients and their significant others agree on quality of life ratings? Health and Quality of Life Outcomes, 7, 87. doi: 10.1186/1477-7525-7-87 CrossRefGoogle Scholar
Glass, G.V., McGaw, B., & Smith, M.L. (1981). Meta-Analysis in Social Research. Beverly Hills, CA: Sage.Google Scholar
Gleichgerrcht, E., Ibáñez, A., Roca, M., Torralya, T., & Manes, F. (2010). Decision-making cognition in neurodegenerative diseases. Nature Reviews Neurology, 6, 611623. doi: 10.1038/nrneurol.2010.148 CrossRefGoogle ScholarPubMed
Gliem, J.A. & Gliem, R.R. (2003) Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for likert-type scales. In Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education, The Ohio State University, Columbus, OH, 8–10 October, (pp. 8288).Google Scholar
Gualtieri, C.T., & Johnson, L.G. (2006). Reliability and validity of a computerized neurocognitive test battery, CNS Vital Signs. Archives of Clinical Neuropsychology, 21, 623643. doi: 10.1016/j.acn.2006.05.007 CrossRefGoogle ScholarPubMed
Hart, T., Whyte, J., Kim, J., & Vaccaro, M. (2005). Executive function and self-awareness of “real-world” behavior and attention deficits following traumatic brain injury. The Journal of Head Trauma Rehabilitation, 20(4), 333347.CrossRefGoogle ScholarPubMed
Howland, M., Allan, K.C., Carlton, C.E., Tatsuoka, C., Smyth, K.A., & Sajatociv, M. (2017). Patient-rated versus proxy-rated cognitive and functional measures in older adults. Patient Related Outcome Measures, 8, 3342. doi: 10.2147/PROM.S126919 CrossRefGoogle ScholarPubMed
Hutchinson, A.D., Hosking, J.R., Kichenadasse, G., Mattiske, J.K., & Wilson, C. (2012). Objective and subjective cognitive impairment following chemotherapy for cancer: a systematic review. Cancer Treatment Reviews, 38(7), 926934. doi: 10.1016/j.ctrv.2012.05.002 CrossRefGoogle ScholarPubMed
Kane, H.L., Halpern, M.T., Squiers, L.B., Treiman, K.A., & McCormack, L.A. (2014). Implementing and evaluating shared decision making in oncology practice. CA: A Cancer Journal for Clinicians, 64(6), 377388. doi: 10.3322/caac.21245 Google ScholarPubMed
Karr, J.E., Areshenkoff, C.N., Rast, P., Hofer, S.M., Iverson, G.L., & Garcia-Barrera, M.A. (2018). The unity and diversity of executive functions: A systematic review and re-analysis of latent variable studies. Psychological Bulletin, 144, 11471185. doi: 10.1037/bul0000160 CrossRefGoogle ScholarPubMed
Kerrigan, S., Erridge, S., Liaquat, I., Graham, C., & Grant, R. (2014). Mental incapacity in patients undergoing neuro-oncologic treatment: A cross-sectional study. Neurology, 83, 537541. doi: 10.1212/WNL.0000000000000671 CrossRefGoogle ScholarPubMed
Lanni, K.E., Ross, J.M., Higginson, C.I., Dressler, E.M., Sigvardt, K.A., Zhang, L., Malhado-Chang, N., & Disbrow, E.A. (2014). Perceived and performance-based executive dysfunction in Parkinson’s disease. Journal of Clinical and Experimental Neuropsychology, 36, 342355. doi: 10.1080/13803395.2014.892059 CrossRefGoogle ScholarPubMed
Lin, L.I. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 45, 255268.CrossRefGoogle ScholarPubMed
Lin, L.I. (2000). A note on the concordance correlation coefficient. Biometrics, 56, 324325.Google Scholar
Meltzer, E.P., Kapoor, A., Fogel, J., Elbulok-Charcape, M.M., Roth, R.M., Katz, M.J., … Rabin, L.A. (2016). Association of psychological, cognitive, and functional variables with self-reported executive functioning in a sample of nondemented community-dwelling older adults. Applied Neuropsychology: Adult, 24(4), 364375. doi: 10.1080/23279095.2016.1185428 CrossRefGoogle Scholar
Meskal, I., Gehring, K., Rutten, G.J.M., & Sitskoorn, M.M. (2016). Cognitive functioning in meningioma patients: A systematic review. Journal of Neuro-Oncology, 128(2), 195205. doi: 10.1007/s11060-016-2115-z CrossRefGoogle ScholarPubMed
Meskal, I., Gehring, K., van der Linden, S.D., Rutten, G.J., & Sitskoorn, M.M. (2015). Cognitive improvement in meningioma patients after surgery: Clinical relevance of computerized testing. Journal of Neuro-Oncology, 121(3), 617625. doi: 10.1007/s11060-014-1679-8 CrossRefGoogle ScholarPubMed
Miyake, A., & Friedman, N.P. (2012). The nature and organization of individual differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21, 814.CrossRefGoogle ScholarPubMed
Moore, R.C., Schwendsen, J., & Depp, C.A. (2017). Applications for self-administered mobile cognitive assessments in clinical research: A systematic review. International Journal of Methods in Psychiatric Research, 26(4), 118. doi: 10.1002/mpr.1562 CrossRefGoogle ScholarPubMed
Myin-Germeys, I., Kasanova, Z., Vaessen, T., Vachon, H., Kirtley, O., Viechtbauer, W., & Reininghaus, U. (2018). Experience sampling methodology in mental health research: New insights and technical developments. World Psychiatry, 17, 123132. doi: 10.1002/wps.20513 CrossRefGoogle ScholarPubMed
Noll, K.R., Bradshaw, M.E., Weinberg, J.S., & Wefel, J.S. (2018). Neurocognitive functioning is associated with functional independence in newly diagnosedpatients with temporal lobe glioma. Neuro-Oncology Practice, 5(3), 184193. doi: 10.1093/nop/npx028 CrossRefGoogle ScholarPubMed
Nugent, B.D., Weimer, J., Choi, C.J., Bradley, C.J., Bender, C.M., Ryan, C.M., Gardner, P., & Sherwood, P.R. (2014). Work productivity and neuropsychological function in persons with skull base tumors. Neuro-Oncology Practice, 1(3), 106113. doi: 10.1093/nop/npu015 CrossRefGoogle ScholarPubMed
Parsey, C.M., & Schmitter-Edgecombe, M. (2013). Applications of technology in neuropsychological assessment. The Clinical Neuropsychologist, 27(8), 13281361. doi: 10.1080/13854046.2013.834971 CrossRefGoogle ScholarPubMed
Parsons, T.D., Carlew, A.R., Magtoto, J., & Stonecipher, K. (2015). The potential of functional-led virtual environments for ecologically valid measures of executive function in experimental and clinical neuropsychology. Neuropsychological Rehabilitation, 27, 777807. doi: 10.1080/09602011.2015.1109524 CrossRefGoogle ScholarPubMed
Pranckeviciene, A., Deltuva, V.P., Tamasauskas, A., & Bunevicius, A. (2017). Association between psychological distress, subjective cognitive complaints and objective neuropsychological functioning in brain tumor patients. Clinical Neurology and Neurosurgery, 163, 1823. doi: 10.1016/j.clineuro.2017.10.007 CrossRefGoogle ScholarPubMed
Rabin, L.A., Wang, C., Katz, M.J., Derby, C.A., Buschke, H., & Lipton, R.B. (2012). Predicting Alzheimer’s disease: Neuropsychological tests, self- reports, and informant reports of cognitive difficulties. Journal of the American Geriatrics Society, 60, 11281134. doi: 10.1111/j.1532-5415.2012.03956.x CrossRefGoogle ScholarPubMed
Richard, N.M., Bernstein, L.J., Mason, W.P., Laperriere, N., Maurice, C., Millar, B., Shultz, D.B., Berlin, A., & Edelstein, K. (2019). Cognitive rehabilitation for executive dysfunction in brain tumor patients: A pilot randomized controlled trial. Journal of Neuro-Oncology, 142, 565575. doi: 10.1007/s11060-019-03130-1 CrossRefGoogle ScholarPubMed
Rijnen, S.J.M., Meskal, I., Bakker, M., De Baene, W., Rutten, G.J.M., Gehring, K., & Sitskoorn, M.M. (2019). Cognitive outcomes in meningioma patients undergoing surgery: Individual changes over time and predictors of late cognitive functioning. Neuro-Oncology, 21, 911922. doi: 10.1093/neuonc/noz039.CrossRefGoogle ScholarPubMed
Rijnen, S.J.M, Meskal, I., Emons, W.H.M., Campman, C.A.M., Van Der Linden, S.D., Gehring, K., & Sitskoorn, M.M. (2017). Evaluation of normative data of a widely used computerized neuropsychological battery: Applicability and effects of sociodemographic variables in a Dutch sample. Assessment, 115 (epub ahead of print) doi: 10.1177/1073191117727346 CrossRefGoogle Scholar
Rijnen, S.J.M., Van der Linden, S.D., Emons, W.H.M., Sitskoorn, M.M., & Gehring, K. (2018). Test-retest reliability and practice effects of the computerized neuropsychological battery CNS Vital Signs: A solution-oriented approach. Psychological Assessment, 30(12), 16521662. doi: 10.1037/pas0000618 CrossRefGoogle ScholarPubMed
Rooney, A.G., McNamara, S., Mackinnon, M., Fraser, M., Rampling, R., Carson, A., & Grant, R. (2013). Screening for major depressive disorder in adults with glioma using the PHQ-9: A comparison of patient versus proxy reports. Journal of Neuro-Oncology, 113, 4955. doi: 10.1007/s11060-013-1088-4 CrossRefGoogle ScholarPubMed
Roth, R.M., Isquith, P.K., & Gioia, G.A. (2005). BRIEF-A. Behavior Rating Inventory of Executive Function- Adult Version. Lutz, FL: Psychological Assessment Resources, Inc.Google Scholar
Rueda, A.D., Lau, K.M., Saito, N., Harvey, D., Risacher, S.L., Aisen, P.S., Petersen, R.C., Saykin, A.J., & Farias, S.T. (2015). Self-rated and informant-rated everyday function in comparison to objective markers of Alzheimer’s disease. Alzheimer’s & Dementia, 11, 10801089. doi: 10.1016/j.jalz.2014.09.002 CrossRefGoogle ScholarPubMed
Schiebener, J., Wegmann, E., Gathmann, B., Laier, C., Pawlikowski, M., & Brand, M. (2014). Among three different executive functions, general executive control ability is a key predictor of decision making under objective risk. Frontiers in Psychology, 5, 1386. doi: 10.3389/fpsyg.2014.01386 CrossRefGoogle ScholarPubMed
Schiehser, D.M., Delis, D.C., Filoteo, J.V., Delano-Wood, L., Han, S.D., Jak, A.J., Drake, A.I., & Bondi, M.W. (2011). Are self-reported symptoms of executive dysfunction associated with objective executive function performance following mild to moderate traumatic brain injury? Journal of Clinical and Experimental Neuropsychology, 33(6), 704714. doi: 10.1080/13803395.2011.553587.CrossRefGoogle ScholarPubMed
Scholte, E. & Noens, I. (2011). BRIEF-A. Vragenlijst over executief functioneren bij volwassenen. Handleiding. Amsterdam, The Netherlands: Hogrefe.Google Scholar
Sneeuw, K.C., Aaronson, N.K., Osoba, D., Muller, M.J., Hsu, M.A., Yung, W.K., Brada, M., & Newlands, E.S. (1997). The use of significant others as proxy raters of the quality of life of patients with brain cancer. Medical Care, 35, 490506.CrossRefGoogle ScholarPubMed
Sterckx, W., Coolbrandt, A., Dierckx de Casterle, B., Van den Heede, K., Decruyenaere, M., Borgenon, S., Mees, A., & Clement, P. (2013). The impact of a high-grade glioma on everyday life: A systematic review from the patients and caregiver’s perspective. European Journal of Oncology Nursing, 17(1), 107117. doi: 10.1016/j.ejon.2012.04.006 CrossRefGoogle ScholarPubMed
Sunderaraman, P. & Consentino, S. (2017). Integrating the constructs of anosognosia and metacognition: A review of recent findings in dementia. Current Neurology and Neuroscience Reports, 17(3), 27. doi: 10.1007/s11910-017-0734-1 CrossRefGoogle ScholarPubMed
Talacchi, A., Santini, B., Savazzi, S., & Gerosa, M. (2011). Cognitive effects of tumour and surgical treatment in glioma patients. Journal of Neuro-Oncology, 103(3), 541549. doi: 10.1007/s11060-010-0417-0 CrossRefGoogle ScholarPubMed
Toplak, M.E., West, R.F., & Stanovich, K.E. (2013). Practitioner review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry, 54(2), 131143. doi: 10.1111/jcpp.12001 CrossRefGoogle ScholarPubMed
Van der Linden, S.D., Sitskoorn, M.M., Rutten, G.J.M., & Gehring, K. (2018a). Feasibility of the evidence-based cognitive telerehabilitation program remind for patients with primary brain tumors. Journal of Neuro-Oncology, 137, 523532. doi: 10.1007/s11060-017-2738-8 CrossRefGoogle ScholarPubMed
Van der Linden, S.D., Sitskoorn, M.M., Rutten, G.J.M., & Gehring, K. (2018b). Study protocol for a randomized controlled trial evaluating the efficacy of an evidence-based app for cognitive rehabilitation in patients with primary brain tumours. Neurosurgery, 85(2), 273279. doi: 10.1093/neuros/nyy254 CrossRefGoogle Scholar
Van Kessel, E., Baumfalk, A.E., Van Zandvoort, M.J.E., Robe, P.A., & Snijders, T.J. (2017). Tumor-related neurocognitive dysfunction in patients with diffuse glioma: A systematic review of neurocognitive functioning prior to anti-tumor treatment. Journal of Neuro-Oncology, 134, 918. doi: 10.1007/s11060-017-2503-z CrossRefGoogle ScholarPubMed
Van Loenen, I.S., Rijnen, S.J.M., Bruijn, J., Rutten, G.J.M., Gehring, K., & Sitskoorn, M.M. (2018). Group changes in cognitive performance after surgery mask changes in individual patients with glioblastoma. World Neurosurgery, 117, e172e179. doi: 10.1016/j.wneu.2018.05.232 CrossRefGoogle ScholarPubMed
Wechsler, D. (2015). Wechsler Adult Intelligence Scale (3rd ed.). Nederlandstalige bewerking. Technische handleiding. Amsterdam, The Netherlands: Harcourt Test Publishers.Google Scholar
World Medical Association. (2013). World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects. Journal of the American Medical Association, 310(20), 21912194.CrossRefGoogle Scholar
Figure 0

Table 1. Mean levels of reported EF: Comparison with normative values and differences between patients (n = 47) and proxies (n = 47)

Figure 1

Table 2. Level of agreement between patient-report and proxy-report of EF

Figure 2

Fig. 1. Bland–Altman plot for agreement on the (a) Behavioral Regulation Index, and (b) Metacognition Index. Note: Dashed lines represent the upper and lower limits of the 95 % confidence intervals.

Figure 3

Table 3. Dyadic agreement on the presence or absence of reported EF impairment

Figure 4

Table 4. Performance-based measures of EF: Mean scores compared to normative values and correlations with reported EF