Skip to main content Accessibility help

From mental representations to neural codes: A multilevel approach

  • Jon Gauthier (a1), João Loula (a1), Eli Pollock (a1), Tyler Brooke Wilson (a2) and Catherine Wong (a1)...


Representation and computation are the best tools we have for explaining intelligent behavior. In our program, we explore the space of representations present in the mind by constraining them to explain data at multiple levels of analysis, from behavioral patterns to neural activity. We argue that this integrated program assuages Brette's worries about the study of the neural code.



Hide All
Anderson, J. R. (1989) A rational analysis of human memory. In: Varieties of memory and consciousness: Essays in honour of Endel Tulving, ed. Roediger, H. L. III & Craik, F. I. M., pp. 195210. Lawrence Erlbaum Associates.
Chang, M. B., Ullman, T., Torralba, A., & Tenenbaum, J. B. (2017) A compositional object-based approach to learning physical dynamics. Presented at the 5th International Conference on Learning Representations (ICLR 2017), April 24–26, 2017, Toulon, France. Available at:
Fragkiadaki, K., Agrawal, P., Levine, S. & Malik, J. (2016) Learning visual predictive models of physics for playing billiards. Presented at the International Conference on Learning Representations, San Juan, Puerto Rico. Available at:
Frank, M. J., Seeberger, L. C. & O'Reilly, R. C. (2004) By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science 306(5703):1940–43.
Gao, P., Trautmann, E., Yu, B., Santhanam, G., Ryu, S., Shenoy, K. & Ganguli, S. (2017) A theory of multineuronal dimensionality, dynamics and measurement. bioRxiv 214262. doi:
Gulordava, K., Bojanowski, P., Grave, E., Linzen, T. & Baroni, M. (2018) Colorless green recurrent networks dream hierarchically. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, Vol. 1, pp. 11951205. Association for Computational Linguistics. Available at:
Hudson, D. A. & Manning, C. D. (2018) Compositional attention networks for machine reasoning. Presented at the International Conference on Learning Representations. arXiv preprint arXiv:1803.03067.
Jazayeri, M., & Afraz, A. (2017) Navigating the neural space in search of the neural code. Neuron 93(5):1003–14. doi:10.1016/j.neuron.2017.02.019.
Johnson, J., Hariharan, B., van der Maaten, L., Hoffman, J., Fei-Fei, L., Lawrence Zitnick, C. & Girshick, R. (2017) Inferring and executing programs for visual reasoning. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2989–2998. IEEE.
Mastrogiuseppe, F. & Ostojic, S. (2018) Linking connectivity, dynamics, and computations in low-rank recurrent neural networks. Neuron 99(3):609–23.e29.
McClelland, J. L. & Rogers, T. T. (2003) The parallel distributed processing approach to semantic cognition. Nature Reviews Neuroscience 4(4):310.
Młynarski, W. & McDermott, J. H. (2018) Learning midlevel auditory codes from natural sound statistics. Neural Computation 30(3):631–69.
Montague, P. R., Dayan, P. & Sejnowski, T. J. (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience 16(5):1936–47.
Niv, Y. (2009) Reinforcement learning in the brain. Journal of Mathematical Psychology 53(3):139–54.
Rajalingham, R., Issa, E. B., Bashivan, P., Kar, K., Schmidt, K. & DiCarlo, J. J. (2018) Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks. Journal of Neuroscience 38(33):7255–69.
Saxena, S. & Cunningham, J. P. (2019) Towards the neural population doctrine. Current Opinion in Neurobiology 55:103–11. doi:10.1016/j.conb.2019.02.002.
Schultz, W., Dayan, P. & Montague, P. R. (1997) A neural substrate of prediction and reward. Science 275(5306):1593–9.
Smolensky, P. & Legendre, G. (2006) The harmonic mind: From neural computation to optimality-theoretic grammar: Vol. 1. Cognitive architecture. MIT Press.
Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K. & Janak, P. H. (2013) A causal link between prediction errors, dopamine neurons and learning. Nature Neuroscience 16(7):966–73.
Yi, K., Wu, J., Gan, C., Torralba, A., Kohli, P. & Tenenbaum, J. (2018) Neural-symbolic VQA: Disentangling reasoning from vision and language understanding. In: Advances in Neural Information Processing Systems, Volume 31, ed. Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N. & Garnett, R., pp. 1031–42. Neural Information Processing Systems Foundation.
Eliasmith, C. & Anderson, C. H. (2004) Neural engineering: Computation, representation, and dynamics in neurobiological systems. MIT Press.

From mental representations to neural codes: A multilevel approach

  • Jon Gauthier (a1), João Loula (a1), Eli Pollock (a1), Tyler Brooke Wilson (a2) and Catherine Wong (a1)...


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed