The visually guided development of facial representations in the primate ventral visual pathway: A computer modeling study.

Citation

Eguchi, A., Humphreys, G. W., & Stringer, S. M. (2016). The visually guided development of facial representations in the primate ventral visual pathway: A computer modeling study. Psychological Review, 123(6), 696-739.

http://dx.doi.org/10.1037/rev0000042

Abstract

Experimental studies have shown that neurons at an intermediate stage of the primate ventral visual pathway, occipital face area, encode individual facial parts such as eyes and nose while neurons in the later stages, middle face patches, are selective to the full face by encoding the spatial relations between facial features. We have performed a computer modeling study to investigate how these cell firing properties may develop through unsupervised visually guided learning. A hierarchical neural network model of the primate’s ventral visual pathway is trained by presenting many randomly generated faces to the network while a local learning rule modifies the strengths of the synaptic connections between neurons in successive layers. After training, the model is found to have developed the experimentally observed cell firing properties. In particular, we have shown how the visual system forms separate representations of facial features such as the eyes, nose, and mouth as well as monotonically tuned representations of the spatial relationships between these facial features. We also demonstrated how the primate brain learns to represent facial expression independently of facial identity. Furthermore, based on the simulation results, we propose that neurons encoding different global attributes simply represent different spatial relationships between local features with monotonic tuning curves or particular combinations of these spatial relations. (PsycINFO Database Record (c) 2016 APA, all rights reserved)

Unique Identifier

2016-51969-002

Title

The visually guided development of facial representations in the primate ventral visual pathway: A computer modeling study.

Publication Date

Nov 2016

Publication History

Accepted: Jul 12, 2016

Revised: Jul 12, 2016

First Submitted: Sep 5, 2015

Language

English

Author Identifier

Eguchi, Akihiro; Humphreys, Glyn W.; Stringer, Simon M.

Email

Eguchi, Akihiro: akihiro.eguchi@psy.ox.ac.uk

Correspondence Address

Eguchi, Akihiro: Department of Experimental Psychology, Oxford University, Tinbergen Building, 9 South Parks Road, Oxford, United Kingdom, OX1 3UD, akihiro.eguchi@psy.ox.ac.uk

Affiliation

Eguchi, Akihiro: Department of Experimental Psychology, Oxford University, Oxford, United Kingdom

Humphreys, Glyn W.: Department of Experimental Psychology, Oxford University, Oxford, United Kingdom

Stringer, Simon M.: Department of Experimental Psychology, Oxford University, Oxford, United Kingdom

Source

Psychological Review, Vol 123(6), Nov 2016, 696-739.

NLM Title Abbreviation

Psychol Rev

ISSN

1939-1471(Electronic); 0033-295X(Print)

Publisher

US: American Psychological Association

Other Publishers

US: Macmillan & Company

US: Psychological Review Company

US: The Macmillan Company

US: The Review Publishing Company

Format Covered

Electronic

Publication Type

Journal; Peer Reviewed Journal

Document Type

Journal Article

Digital Object Identifier

http://dx.doi.org/10.1037/rev0000042

Keywords

primate ventral visual pathway; face processing; facial expression; neural network model; trace learning

Index Terms

*Face Perception; *Facial Expressions; *Models; *Neural Networks; *Primates (Nonhuman); Basal Ganglia; Learning

PsycINFO Classification

2520 Neuropsychology & Neurology

Population Group

Animal

Copyright

Holder: American Psychological Association

Year: 2016

Methodology

Mathematical Model

Grant Sponsorship

Sponsor: Oxford Foundation for Theoretical Neuroscience and Artificial Intelligence, United Kingdom
Recipient: No recipient indicated

Conference

Annual Computational Neuroscience Meeting: CNS2014, 23rd, Quebec City, PQ, Canada; Some of the data and the ideas reported in this article have been presented at the aforementioned conference.

Release Date

20161031 (PsycINFO); 20161031 (PsycARTICLES)

References

Number of Citations: 79, Number of Citations Displayed: 79

• Baker, C. I., Behrmann, M., & Olson, C. R. (2002). Impact of learning on representation of parts and wholes in monkey inferotemporal cortex. Nature Neuroscience, 5(11), 1210-1216. http://dx.doi.org/10.1038/nn960
• Benson, P. J., Perrett, D. I. (1991). Synthesising continuous-tone caricatures. Image and Vision Computing, 9, 123-129. http://dx.doi.org/10.1016/0262-8856(91)90022-H
• Biederman, I., Kalocsai, P. (1997). Neurocomputational bases of object and face recognition. Philosophical Transactions of the Royal Society B: Biological Sciences, 352, 1203-1219.
• Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94(2), 115-147. http://dx.doi.org/10.1037/0033-295X.94.2.115
• Bruce, V., Young, A. (2011). Face perception. New York, NY: Psychology Press, London.
• Bruce, V., & Young, A. (1986). Understanding face recognition. British Journal of Psychology, 77(3), 305-327. http://dx.doi.org/10.1111/j.2044-8295.1986.tb02199.x
• Calder, A. J., Burton, A. M., Miller, P., Young, A. W., & Akamatsu, S. (2001). A principal component analysis of facial expressions. Vision Research, 41(9), 1179-1208. http://dx.doi.org/10.1016/S0042-6989(01)00002-5
• Chance, J. E., Turner, A. L., & Goldstein, A. G. (1982). Development of differential recognition for own- and other-race faces. The Journal of Psychology: Interdisciplinary and Applied, 112(1), 29-37. http://dx.doi.org/10.1080/00223980.1982.9923531
• Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36(3), 181-204. http://dx.doi.org/10.1017/S0140525X12000477
• Cumming, B. G., & Parker, A. J. (1999). Binocular neurons in V1 of awake monkeys are selective for absolute, not relative, disparity. The Journal of Neuroscience, 19(13), 5602-5618.
• Eguchi, A., Mender, B. M. W., Evans, B., Humphreys, G., Stringer, S. (2015). Computational modeling of the neural representation of object shape in the primate ventral visual system. Frontiers in Computational Neuroscience, 9, 100.
• Eguchi, A., Neymotin, S. A., Stringer, S. M. (2014). Color opponent receptive fields self-organize in a biophysical model of visual cortex via spike-timing dependent plasticity. Frontiers in Neural Circuits, 8, 16. http://dx.doi.org/10.3389/fncir.2014.00016
• Eliasmith, C., Stewar, T. C., Choo, X., Bekolay, T., DeWolf, T., Tang, C., & Rasmussen, D. (2012). A large-scale model of the functioning brain. Science, 338(6111), 1202-1205. http://dx.doi.org/10.1126/science.1225266
• Engell, A. D., & Haxby, J. V. (2007). Facial expression and gaze-direction in human superior temporal sulcus. Neuropsychologia, 45(14), 3234-3241. http://dx.doi.org/10.1016/j.neuropsychologia.2007.06.022
• Földiák, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3(2), 194-200. http://dx.doi.org/10.1162/neco.1991.3.2.194
• Freeman, J. B., Rule, N. O., Adams, R. B., Jr., & Ambady, N. (2010). The neural basis of categorical face perception: Graded representations of face gender in fusiform and orbitofrontal cortices. Cerebral Cortex, 20(6), 1314-1322. http://dx.doi.org/10.1093/cercor/bhp195
• Freeman, J., & Simoncelli, E. P. (2011). Metamers of the ventral stream. Nature Neuroscience, 14(9), 1195-1201. http://dx.doi.org/10.1038/nn.2889
• Freiwald, W. A., Tsao, D. Y., & Livingstone, M. S. (2009). A face feature space in the macaque temporal lobe. Nature Neuroscience, 12(9), 1187-1196. http://dx.doi.org/10.1038/nn.2363
• Friesen, W. V., Ekman, P. (1976). Pictures of facial affect. Palo Alto, CA: Consulting Psychologists Press.
• Gillebert, C. R., Op de Beeck, H. P., Panis, S., & Wagemans, J. (2009). Subordinate categorization enhances the neural selectivity in human object-selective cortex for fine shape differences. Journal of Cognitive Neuroscience, 21(6), 1054-1064. http://dx.doi.org/10.1162/jocn.2009.21089
• Gosselin, F., & Schyns, P. G. (2001). Bubbles: A technique to reveal the use of information in recognition tasks. Vision Research, 41(17), 2261-2271. http://dx.doi.org/10.1016/S0042-6989(01)00097-9
• Gross, C. G., Rocha-Miranda, C. E., & Bender, D. B. (1972). Visual properties of neurons in inferotemporal cortex of the macaque. Journal of Neurophysiology, 35(1), 96-111.
• Hasselmo, M. E., Rolls, E. T., & Baylis, G. C. (1989). The role of expression and identity in the face-selective responses of neurons in the temporal visual cortex of the monkey. Behavioural Brain Research, 32(3), 203-218. http://dx.doi.org/10.1016/S0166-4328(89)80054-3
• Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4(6), 223-233. http://dx.doi.org/10.1016/S1364-6613(00)01482-0
• Henriksson, L., Mur, M., & Kriegeskorte, N. (2015). Faciotopy—A face-feature map with face-like topology in the human occipital face area. Cortex: A Journal Devoted to the Study of the Nervous System and Behavior, 72, 156-167. http://dx.doi.org/10.1016/j.cortex.2015.06.030
• Homola, G. A., Jbabdi, S., Beckmann, C. F., & Bartsch, A. J. (2012). A brain network processing the age of faces. PLoS ONE, 7(11). Article ID e49451. http://dx.doi.org/10.1371/journal.pone.0049451
• Jones, J. P., Palmer, L. A. (1987). The two-dimensional spatial structure of simple receptive fields in cat striate cortex. Journal of Neurophysiology, 58, 1187-1211.
• Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: A module in human extrastriate cortex specialized for face perception. The Journal of Neuroscience, 17(11), 4302-4311.
• Khaligh-Razavi, S.-M., Kriegeskorte, N. (2014). Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Computational Biology, 10, e1003915. http://dx.doi.org/10.1371/journal.pcbi.1003915
• Kiani, R., Esteky, H., Mirpour, K., & Tanaka, K. (2007). Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. Journal of Neurophysiology, 97(6), 4296-4309. http://dx.doi.org/10.1152/jn.00024.2007
• Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43, 59-69. http://dx.doi.org/10.1007/BF00337288
• Kriegeskorte, N., Mur, M., Bandettini, P. A., Kriegeskorte, N., Mur, M., Bandettini, P. (2008a). Representational similarity analysis: Connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 4.
• Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., Bandettini, P. A. (2008b). Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron, 60, 1126-1141.
• Lades, M., Vorbruggen, J., Buhmann, J., Lange, J., von der Malsburg, C., Wurtz, R., Konen, W. (1993). Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers, 42, 300-311.
• Lawrence, S., Giles, C., Tsoi, A. C., Back, A. (1997). Face recognition: A convolutional neural-network approach. IEEE Transactions on Neural Networks, 8, 98-113.
• Leder, H., & Bruce, V. (1998). Local and relational aspects of face distinctiveness. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 51A(3), 449-473. http://dx.doi.org/10.1080/027249898391486
• Lisetti, C. L., Rumelhart, D. E. (1998). Facial expression recognition using a neural network. In FLAIRS Conference, May 1998, Sanabal Island, FL, pp. 328-332.
• Mangini, M. C., & Biederman, I. (2004). Making the ineffable explicit: Estimating the information employed for face classifications. Cognitive Science, 28(2), 209-226. http://dx.doi.org/10.1016/j.cogsci.2003.11.004
• Masquelier, T., Thorpe, S. J. (2007). Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Computational Biology, 3, e31. http://dx.doi.org/10.1371/journal.pcbi.0030031
• Maurer, D., Le Grand, R., & Mondloch, C. J. (2002). The many faces of configural processing. Trends in Cognitive Sciences, 6(6), 255-260. http://dx.doi.org/10.1016/S1364-6613(02)01903-4
• Morin, E. L., Hadj-Bouziane, F., Stokes, M., Ungerleider, L. G., & Bell, A. H. (2015). Hierarchical encoding of social cues in primate inferior temporal cortex. Cerebral Cortex, 25(9), 3036-3045.
• de Beeck, H. P. O., Baker, C. I., DiCarlo, J. J., & Kanwisher, N. G. (2006). Discrimination training alters object representations in human extrastriate cortex. The Journal of Neuroscience, 26(50), 13025-13036. http://dx.doi.org/10.1523/JNEUROSCI.2481-06.2006
• Pasupathy, A. (2006). Neural basis of shape representation in the primate brain. Progress in Brain Research, 154, 293-313. http://dx.doi.org/10.1016/S0079-6123(06)54016-6
• Perrett, D. I., Hietanen, J. K., Oram, M. W., Benson, P. J. (1992). Organization and functions of cells responsive to faces in the temporal cortex. Philosophical transactions of the Royal Society of London, Series B: Biological Sciences, 335, 23-30. http://dx.doi.org/10.1098/rstb.1992.0003
• Petkov, N., Kruizinga, P. (1997). Computational models of visual neurons specialised in the detection of periodic and aperiodic oriented visual stimuli: Bar and grating cells. Biological Cybernetics, 76, 83-96. http://dx.doi.org/10.1007/s004220050323
• Pettet, M. W., Gilbert, C. D. (1992). Dynamic changes in receptive-field size in cat primary visual cortex. Proceedings of the National Academy of Sciences of the United States of America, 89, 8366-8370. http://dx.doi.org/10.1073/pnas.89.17.8366
• Pitcher, D., Walsh, V., & Duchaine, B. (2011). The role of the occipital face area in the cortical face perception network. Experimental Brain Research, 209(4), 481-493. http://dx.doi.org/10.1007/s00221-011-2579-1
• Rhodes, G. (1997). Superportraits: Caricatures and recognition (1st ed.). Hove, East Sussex: Psychology Press.
• Rolls, E. T. (2000). Functions of the primate temporal lobe cortical visual areas in invariant visual object and face recognition. Neuron, 27, 205-218. http://dx.doi.org/10.1016/S0896-6273(00)00030-1
• Rolls, E. T., & Milward, T. (2000). A model of invariant object recognition in the visual system: Learning rules, activation functions, lateral inhibition, and information-based performance measures. Neural Computation, 12(11), 2547-2572. http://dx.doi.org/10.1162/089976600300014845
• Rolls, E. T., Treves, A. (1998). Neural networks and brain function (1st ed.). New York, NY: Oxford University Press.
• Rolls, E. T., Treves, A., Tovee, M. J., & Panzeri, S. (1997). Information in the neuronal representation of individual stimuli in the primate temporal visual cortex. Journal of Computational Neuroscience, 4(4), 309-333. http://dx.doi.org/10.1023/A:1008899916425
• Royer, S., & Paré, D. (2003). Conservation of total synaptic weight through balanced synaptic depression and potentiation. Nature, 422(6931), 518-522. http://dx.doi.org/10.1038/nature01530
• Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533-536. http://dx.doi.org/10.1038/323533a0
• Serre, T., Kouh, M., Cadieu, C., Knoblich, U., Kreiman, G., Poggio, T. (2005). A theory of object recognition: Computations and circuits in the feedforward path of the ventral stream in primate visual cortex. Cambridge, MA: MIT CSAIL.
• Sormaz, M., Watson, D. M., Smith, W. A. P., Young, A. W., & Andrews, T. J. (2016). Modelling the perceptual similarity of facial expressions from image statistics and neural responses. NeuroImage, 129, 64-71. http://dx.doi.org/10.1016/j.neuroimage.2016.01.041
• Stork, D. (1989). Is backpropagation biologically plausible? International Joint Conference on Neural Networks, IJCNN, Washington, DC, pp. 241-246, vol. 2. http://dx.doi.org/10.1109/IJCNN.1989.118705
• Stork, D. (1989). Is backpropagation biologically plausible? International Joint Conference on Neural Networks, IJCNN, Washington, DC, pp. 241-246, vol. 2. http://dx.doi.org/10.1109/IJCNN.1989.118705
• Stringer, S. M., Perry, G., Rolls, E. T., Proske, J. H. (2006). Learning invariant object recognition in the visual system with continuous transformations. Biological cybernetics, 94, 128-142.
• Stringer, S. M., & Rolls, E. T. (2008). Learning transform invariant object recognition in the visual system with multiple stimuli present during training. Neural Networks, 21(7), 888-903. http://dx.doi.org/10.1016/j.neunet.2007.11.004
• Stringer, S. M., Rolls, E. T., Tromans, J. M. (2007). Invariant object recognition with trace learning and multiple stimuli present during training. Network, 18, 161-187. http://dx.doi.org/10.1080/09548980701556055
• Taigman, Y., Yang, M., Ranzato, M., Wolf, L. (2014). DeepFace: Closing the gap to human-level performance in face verification. 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Washington, DC, pp. 1701-1708. http://dx.doi.org/10.1109/CVPR.2014.220
• Tanaka, J. W., & Farah, M. J. (1993). Parts and wholes in face recognition. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 46A(2), 225-245. http://dx.doi.org/10.1080/14640749308401045
• Tromans, J. M. (2012). Computational neuroscience of natural scene processing in the ventral visual pathway (Doctoral thesis). University of Oxford, Oxford, England.
• Tromans, J. M., Harris, M., & Stringer, S. M. (2011). A computational model of the development of separate representations of facial identity and expression in the primate visual system. PLoS ONE, 6(10). Article ID e25616. http://dx.doi.org/10.1371/journal.pone.0025616
• Tromans, J. M., Page, H. J. I., & Stringer, S. M. (2012). Learning separate visual representations of independently rotating objects. Network: Computation in Neural Systems, 23(1-2), 1-23.
• Tsao, D. Y., Freiwald, W. A., Tootell, R. B. H., & Livingstone, M. S. (2006). A Cortical Region Consisting Entirely of Face-Selective Cells. Science, 311(5761), 670-674. http://dx.doi.org/10.1126/science.1119983
• von der Malsburg, C., Schneider, W. (1986). A neural cocktail-party processor. Biological Cybernetics, 54, 29-40.
• von der Malsburg, C. (1973). Self-organization of orientation sensitive cells in the striate cortex. Kybernetik, 14, 85-100. http://dx.doi.org/10.1007/BF00288907
• von der Malsburg, C. (1981). The correlation theory of brain function. Departmental Technical Report, MPI.
• Wallis, G., Rolls, E. T. (1997). Invariant face and object recognition in the visual system. Progress in Neurobiology, 51, 167-194. http://dx.doi.org/10.1016/S0301-0082(96)00054-8
• Wallis, G. (2013). Toward a unified model of face and object recognition in the human visual system. Frontiers in Psychology, 4, Article ID 497. http://dx.doi.org/10.3389/fpsyg.2013.00497
• Wegrzyn, M., Riehle, M., Labudda, K., Woermann, F., Baumgartner, F., Pollmann, S., . . . Kissler, J. (2015). Investigating the brain basis of facial expression perception using multi-voxel pattern analysis. Cortex: A Journal Devoted to the Study of the Nervous System and Behavior, 69, 131-140. http://dx.doi.org/10.1016/j.cortex.2015.05.003
• Xu, X., Biederman, I., & Shah, M. P. (2014). A neurocomputational account of the face configural effect. Journal of Vision, 14(8). Article ID 9.
• Yankouskaya, A., Humphreys, G. W., & Rotshtein, P. (2014). Differential interactions between identity and emotional expression in own and other-race faces: Effects of familiarity revealed through redundancy gains. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(4), 1025-1038. http://dx.doi.org/10.1037/a0036259
• Yin, R. K. (1969). Looking at upside-down faces. Journal of Experimental Psychology, 81(1), 141-145. http://dx.doi.org/10.1037/h0027474
• Yue, X., Tjan, B. S., & Biederman, I. (2006). What makes faces special? Vision Research, 46(22), 3802-3811. http://dx.doi.org/10.1016/j.visres.2006.06.017
• Zhang, J., Li, X., Song, Y., & Liu, J. (2012). The fusiform face area is engaged in holistic, not parts-based, representation of faces. PLoS ONE, 7(7). Article ID e40390. http://dx.doi.org/10.1371/journal.pone.0040390
• Zhou, H., Friedman, H. S., & von der Heydt, R. (2000). Coding of border ownership in monkey visual cortex. The Journal of Neuroscience, 20(17), 6594-6611.