Pre-operative functional magnetic resonance imaging (fMRI) has emerged as valuable clinical tool to help surgically manage patients diagnosed with brain tumours. Surgical decision-making may be significantly improved through the provision of fMRI, however its clinical usage is contingent on the level of agreement with direct cortical stimulation (DCS). While previous studies have been undertaken to investigate the spatial agreement between fMRI and DCS, the influence that various factors may have on fMRI sensitivity and specificity is not fully clear. Thus, in a group of eight brain tumour patients who underwent pre-operative fMRI followed intra-operative DCS during an awake craniotomy procedure, we measured the agreement between the two brain mapping techniques looking at the influence of behavioural task, statistical threshold, and task standardization. Results: There were significant differences between motor and language mapping, where agreement was better for the former. Sensitivity and specificity shared an inverse relationship with increasing fMRI threshold, and were significantly reduced in the case where tasks were not standardized. Lastly, false positive occurrences were identified as the dominate source of error in comparison to false negative occurrences. Conclusion: Thus, the results from this work suggest that fMRI can predict intraoperative findings with good accuracy, however, sources of variability may significantly reduce the quality of fMRI data at the single-subject level. Neurosurgeons should carefully evaluate fMRI data with these considerations prior to its inclusion in the surgical-decision making process.