Copyright © 1999 Massachusetts Institute of Technology
The term alone suggests a field of study that is pregnant and full of promise. It is a large field of study, uniting concepts and techniques from many disciplines, and its boundaries are rangy and often loosely defined. At the heart of cognitive neuroscience, however, lies the fundamental question of knowledge and its representation by the brain -- a relationship characterized not inappropriately by WILLIAM JAMES (1842-1910) as "the most mysterious thing in the world" (James 1890 vol. 1, 216). Cognitive neuroscience is thus a science of information processing. Viewed as such, one can identify key experimental questions and classical areas of study: How is information acquired (sensation), interpreted to confer meaning (perception and recognition), stored or modified (learning and memory), used to ruminate (thinking and consciousness), to predict the future state of the environment and the consequences of action (decision making), to guide behavior (motor control), and to communicate (language)? These questions are, of course, foundational in cognitive science generally, and it is instructive to consider what distinguishes cognitive neuroscience from cognitive science and psychology, on the one hand, and the larger field of neuroscience, on the other.
The former distinction is perhaps the fuzzier, depending heavily as it does upon how one defines cognitive science. A neurobiologist might adopt the progressive (or naive) view that the workings of the brain are the subject matter of both, and the distinction is therefore moot. But this view evidently has not prevailed (witness the fact that neuroscience is but one of the subdivisions of this volume); indeed the field of cognitive science was founded upon and continues to press the distinction between software (the content of cognition) and hardware (the physical stuff, for example, the brain) upon which cognitive processes are implemented. Much has been written on this topic, and one who pokes at the distinction too hard is likely to unshelve as much dusty political discourse as true science. In any case, for present purposes, we will consider both the biological hardware and the extent to which it constrains the software, and in doing so we will discuss answers to the questions of cognitive science that are rooted in the elements of biological systems.
The relationship between cognitive neuroscience and the umbrella of modern neuroscience is more straightforward and less embattled. While the former is clearly a subdivision of the latter, the questions of cognitive neuroscience lie at the root of much of neuroscience's turf. Where distinctions are often made, they arise from the fact that cognitive neuroscience is a functional neuroscience -- particular structures and signals of the nervous system are of interest inasmuch as they can be used to explain cognitive functions.
There being many levels of explanation in biological systems -- ranging from cellular and molecular events to complex behavior -- a key challenge of the field of cognitive neuroscience has been to identify the relationships between different levels and the train of causality. In certain limited domains, this challenge has met with spectacular success; in others, it is clear that the relevant concepts have only begun to take shape and the necessary experimental tools are far behind. Using examples drawn from well-developed areas of research, such as vision, memory, and language, we illustrate concepts, experimental approaches, and general principles that have emerged -- and, more specifically, how the work has answered many of the information processing questions identified above. Our contemporary view of cognitive neuroscience owes much to the heights attained by our predecessors; to appreciate the state of this field fully, it is useful to begin with a consideration of how we reached this vantage point.
Legend has it that the term "cognitive neuroscience" was coined by George A. Miller -- the father of modern cognitive psychology -- in the late 1970s over cocktails with Michael Gazzaniga at the Rockefeller University Faculty Club. That engaging tidbit of folklore nevertheless belies the ancient history of this pursuit. Indeed, identification of the biological structures and events that account for our ability to acquire, store, and utilize knowledge of the world was one of the earliest goals of empirical science. The emergence of the interdisciplinary field of cognitive neuroscience that we know today, which lies squarely at the heart of twentieth-century neuroscience, can thus be traced from a common stream in antiquity, with many tributaries converging in time as new concepts and techniques have evolved (Boring 1950).
The focal point of the earliest debates on the subject -- and a topic that has remained a centerpiece of cognitive neuroscience to the present day -- is localization of the material source of psychological functions. With Aristotle as a notable exception (he thought the heart more important), scholars of antiquity rightly identified the brain as the seat of intellect. Relatively little effort was made to localize specific mental functions to particular brain regions until the latter part of the eighteenth century, when the anatomist Franz Josef Gall (1758-1828) unleashed the science of phrenology. Although flawed in its premises, and touted by charlatans, phrenology focused attention on the CEREBRAL CORTEX and brought the topic of localization of function to the forefront of an emerging nineteenth century physiology and psychology of mind (Zola-Morgan 1995). The subsequent HISTORY OF CORTICAL LOCALIZATION of function (Gross 1994a) is filled with colorful figures and weighty confrontations between localizationists and functional holists (antilocalizationists). Among the longest shadows is that cast by PAUL BROCA (1824 - 1880), who in 1861 reported that damage to a "speech center" in the left frontal lobe resulted in loss of speech function, and was thus responsible for the first widely cited evidence for localization of function in the cerebral cortex. An important development of a quite different nature came in the form of the Bell-Magendie law, discovered independently in the early nineteenth century by the physiologists Sir Charles Bell (1774 - 1842) and François Magendie (1783 - 1855). This law identified the fact that sensory and motor nerve fibers course through different roots (dorsal and ventral, respectively) of the spinal cord. Although far from the heavily contested turf of the cerebral cortex, the concept of nerve specificity paved the way for the publication in 1838 by Johannes Muller (1801 - 1858) of the law of specific nerve energies, which included among its principles the proposal that nerves carrying different types of sensory information terminate in distinct brain loci, perhaps in the cerebral cortex.
Persuasive though the accumulated evidence seemed at the dawn of the twentieth century, the debate between localizationists and antilocalizationists raged on for another three decades. By this time the chief experimental tool had become the "lesion method," through which the functions of specific brain regions are inferred from the behavioral or psychological consequences of loss of the tissue in question (either by clinical causes or deliberate experimental intervention). A central player during this period was the psychologist KARL SPENCER LASHLEY (1890-1958) -- often inaccurately characterized as professing strong antilocalizationist beliefs, but best known for the concept of equipotentiality and the law of mass action of brain function. Lashley's descendants include several generations of flag bearers for the localizationist front -- Carlyle Jacobsen, John Fulton, Karl Pribram, Mortimer Mishkin, Lawrence Weiskrantz, and Charles Gross, among others -- who established footholds for our present understanding of the cognitive functions of the frontal and temporal lobes.
These later efforts to localize cognitive functions using the lesion method were complemented by studies of the effects of electrical stimulation of the human brain on psychological states. The use of stimulation as a probe for cognitive function followed its more pragmatic application as a functional brain mapping procedure executed in preparation for surgical treatment of intractable epilepsy. The neurosurgeon WILDER PENFIELD (1891-1976) pioneered this approach in the 1930s at the legendary Montreal Neurological Institute and, with colleagues Herbert Jasper and Brenda Milner, subsequently began to identify specific cortical substrates of language, memory, emotion, and perception.
The years of the mid-twentieth century were quarrelsome times for the expanding field of psychology, which up until that time had provided a home for much of the work on localization of brain function. It was from this fractious environment, with inspiration from the many successful experimental applications of the lesion method and a growing link to wartime clinical populations, that the field of neuropsychology emerged -- and with it the wagons were drawn up around the first science explicitly devoted to the relationship between brain and cognitive function. Early practitioners included the great Russian neuropsychologist ALEKSANDR ROMANOVICH LURIA (1902-1977) and the American behavioral neurologist NORMAN GESCHWIND (1926 - 1984), both of whom promoted the localizationist cause with human case studies and focused attention on the role of connections between functionally specific brain regions. Also among the legendary figures of the early days of neuropsychology was HANS-LUKAS TEUBER (1916 - 1977). Renowned scientifically for his systematization of clinical neuropsychology, Teuber is perhaps best remembered for having laid the cradle of modern cognitive neuroscience in the 1960s MIT Psychology Department, through his inspired recruitment of an interdisciplinary faculty with a common interest in brain structure and function, and its relationship to complex behavior (Gross 1994b).
Although the earliest antecedents of modern cognitive neuroscience focused by necessity on the macroscopic relationship between brain and psychological function, the last 50 years have seen a shift of focus, with major emphasis placed upon local neuronal circuits and the causal link between the activity of individual cells and behavior. The payoff has been astonishing, but one often takes for granted the resolution of much hotly debated turf. The debates in question focused on the elemental units of nervous system structure and function. We accept these matter-of-factly to be specialized cells known as NEURONS, but prior to the development of techniques to visualize cellular processes, their existence was mere conjecture. Thus the two opposing views of the nineteenth century were reticular theory, which held that the tissue of the brain was composed of a vast anastomosing reticulum, and neuron theory, which postulated neurons as differentiated cell types and the fundamental unit of nervous system function. The ideological chasm between these camps ran deep and wide, reinforced by ties to functional holism in the case of reticular theory, and localizationism in the case of neuron theory. The deadlock broke in 1873 when CAMILLO GOLGI (1843-1926) introduced a method for selective staining of individual neurons using silver nitrate, which permitted their visualization for the first time. (Though this event followed the discovery of the microscope by approximately two centuries, it was the Golgi method's complete staining of a minority of neurons that enabled them to be distinguished from one another.) In consequence, the neuron doctrine was cast, and a grand stage was set for studies of differential cellular morphology, patterns of connectivity between different brain regions, biochemical analysis, and, ultimately, electrophysiological characterization of the behavior of individual neurons, their synaptic interactions, and relationship to cognition.
Undisputedly, the most creative and prolific applicant of the Golgi technique was the Spanish anatomist SANTIAGO RAMÓN Y CAJAL (1852-1934), who used this new method to characterize the fine structure of the nervous system in exquisite detail. Cajal's efforts yielded a wealth of data pointing to the existence of discrete neuronal elements. He soon emerged as a leading proponent of the neuron doctrine and subsequently shared the 1906 Nobel Prize in physiology and medicine with Camillo Golgi. (Ironically, Golgi held vociferously to the reticular theory throughout his career.)
Discovery of the existence of independent neurons led naturally to investigations of their means of communication. The fine-scale stereotyped contacts between neurons were evident to Ramón y Cajal, but it was Sir Charles Scott Sherrington (1857-1952) who, at the turn of the century, applied the term "synapses" to label them. The transmission of information across synapses by chemical means was demonstrated experimentally by Otto Loewi (1873 - 1961) in 1921. The next several decades saw an explosion of research on the nature of chemical synaptic transmission, including the discovery of countless putative NEUROTRANSMITTERS and their mechanisms of action through receptor activation, as well as a host of revelations regarding the molecular events that are responsible for and consequences of neurotransmitter release. These findings have provided a rich foundation for our present understanding of how neurons compute and store information about the world (see COMPUTING IN SINGLE NEURONS).
The ability to label neurons facilitated two other noteworthy developments bearing on the functional organization of the brain: (1) cytoarchitectonics, which is the use of coherent regional patterns of cellular morphology in the cerebral cortex to identify candidates for functional specificity; and (2) neuroanatomical tract tracing, by which the patterns of connections between and within different brain regions are established. The practice of cytoarchitectonics began at the turn of the century and its utility was espoused most effectively by the anatomists Oscar Vogt (1870-1950), Cecile Vogt (1875 - 1962), and Korbinian Brodmann (1868 - 1918). Cytoarchitectonics never fully achieved the functional parcellation that it promised, but clear histological differences across the cerebral cortex, such as those distinguishing primary visual and motor cortices from surrounding tissues, added considerable reinforcement to the localizationist camp.
By contrast, the tracing of neuronal connections between different regions of the brain, which became possible in the late nineteenth century with the development of a variety of specialized histological staining techniques, has been an indispensable source of knowledge regarding the flow of information through the brain and the hierarchy of processing stages. Recent years have seen the emergence of some remarkable new methods for tracing individual neuronal processes and for identifying the physiological efficacy of specific anatomical connections (Callaway 1998), the value of which is evidenced most beautifully by studies of the CELL TYPES AND CONNECTIONS IN THE VISUAL CORTEX.
The neuron doctrine also paved the way for an understanding of the information represented by neurons via their electrical properties, which has become a cornerstone of cognitive neuroscience in the latter half of the twentieth century. The electrical nature of nervous tissue was well known (yet highly debated) by the beginning of the nineteenth century, following advancement of the theory of "animal electricity" by Luigi Galvani (1737-1798) in 1791. Subsequent work by Emil du Bois-Reymond (1818 - 1896), Carlo Matteucci (1811 - 1862), and HERMANN LUDWIG FERDINAND VON HELMHOLTZ (1821 - 1894) established the spreading nature of electrical potentials in nervous tissue (nerve conduction), the role of the nerve membrane in maintaining and propagating an electrical charge ("wave of negativity"), and the velocity of nervous conduction. It was in the 1920s that Lord Edgar Douglas Adrian (1889 - 1977), using new cathode ray tube and amplification technology, developed the means to record "action potentials" from single neurons. Through this means, Adrian discovered the "all-or-nothing property" of nerve conduction via action potentials and demonstrated that action potential frequency is the currency of information transfer by neurons. Because of the fundamental importance of these discoveries, Adrian shared the 1932 Nobel Prize in physiology and medicine with Sherrington. Not long afterward, the Finnish physiologist Ragnar Granit developed techniques for recording neuronal activity using electrodes placed on the surface of the skin (Granit discovered the electroretinogram, or ERG, which reflects large-scale neuronal activity in the RETINA). These techniques became the foundation for non-invasive measurements of brain activity (see ELECTROPHYSIOLOGY, ELECTRIC AND MAGNETIC EVOKED FIELDS), which have played a central role in human cognitive neuroscience over the past 50 years.
With technology for SINGLE-NEURON RECORDING and large-scale electrophysiology safely in hand, the mid-twentieth century saw a rapid proliferation of studies of physiological response properties in the central nervous system. Sensory processing and motor control emerged as natural targets for investigation, and major emphasis was placed on understanding (1) the topographic mapping of the sensory or motor field onto central target zones (such as the retinotopic mapping in primary visual cortex), and (2) the specific sensory or motor events associated with changes in frequency of action potentials. Although some of the earliest and most elegant research was directed at the peripheral auditory system -- culminating with Georg von Bekesy's (1889-1972) physical model of cochlear function and an understanding of its influence on AUDITORY PHYSIOLOGY -- it is the visual system that has become the model for physiological investigations of information processing by neurons.
The great era of single-neuron studies of visual processing began in the 1930s with the work of Haldan Keffer Hartline (1903-1983), whose recordings from the eye of the horseshoe crab (Limulus) led to the discovery of neurons that respond when stimulated by light and detect differences in the patterns of illumination (i.e., contrast; Hartline, Wagner, and MacNichol 1952). It was for this revolutionary advance that Hartline became a corecipient of the 1967 Nobel Prize in physiology and medicine (together with Ragnar Granit and George Wald). Single-neuron studies of the mammalian visual system followed in the 1950s, with the work of Steven Kuffler (1913 - 1980) and Horace Barlow, who recorded from retinal ganglion cells. This research led to the development of the concept of the center-surround receptive field and highlighted the key role of spatial contrast detection in early vision (Kuffler 1953). Subsequent experiments by Barlow and Jerome Lettvin, among others, led to the discovery of neuronal FEATURE DETECTORS for behaviorally significant sensory inputs. This set the stage for the seminal work of David Hubel and Torsten Wiesel, whose physiological investigations of visual cortex, beginning in the late 1950s, profoundly shaped our understanding of the relationship between neuronal and sensory events (Hubel and Wiesel 1977).
The rise of neuroscience from its fledgling origins in the nineteenth century was paralleled by the growth of experimental psychology and its embracement of sensation and perception as primary subject matter. The origins of experimental psychology as a scientific discipline coincided, in turn, with the convergence and refinement of views on the nature of the difference between sensation and perception. These views, which began to take their modern shape with the concept of "associationism" in the empiricist philosophy of John Locke (1632-1704), served to focus attention on the extraction of meaning from sensory events and, not surprisingly, lie at the core of much twentieth century cognitive neuroscience.
The proposition that things perceived cannot reflect directly the material of the external world, but rather depend upon the states of the sense organs and the intermediary nerves, is as old as rational empiricism itself. Locke's contribution to this topic was simply that meaning -- knowledge of the world, functional relations between sensations, nee perception -- is born from an association of "ideas," of which sensation was the primary source. The concept was developed further by George Berkeley (1685-1753) in his "theory of objects," according to which a sensation has meaning -- that is, a reference to an external material source -- only via the context of its relationship to other sensations. This associationism was a principal undercurrent of Scottish and English philosophy for the next two centuries, the concepts refined and the debate further fueled by the writings of James Mill and, most particularly, John Stuart Mill. It was the latter who defined the "laws of association" between elemental sensations, and offered the useful dictum that perception is the belief in the "permanent possibilities of sensation." By so doing, Mill bridged the gulf between the ephemeral quality of sensations and the permanence of objects and our experience of them: it is the link between present sensations and those known to be possible (from past experience) that allows us to perceive the enduring structural and relational qualities of the external world.
In the mid-nineteenth century the banner of associationism was passed from philosophy of mind to the emerging German school of experimental psychology, which numbered among its masters Gustav Fechner (1801-1887), Helmholtz, WILHELM WUNDT (1832 - 1920), and the English- American disciple of that tradition Edward Titchener (1867 - 1927). Fechner's principal contribution in this domain was the introduction of a systematic scientific methodology to a topic that had before that been solely the province of philosophers and a target of introspection. Fechner"s Elements of Psychophysics, published in 1860, founded an "exact science of the functional relationship . . . between body and mind," based on the assumption that the relationship between brain and perception could be measured experimentally as the relationship between a stimulus and the sensation it gives rise to. PSYCHOPHYSICS thus provided the new nineteenth- century psychology with tools of a rigorous science and has subsequently become a mainstay of modern cognitive neuroscience. It was during this move toward quantification and systematization that Helmholtz upheld the prevailing associationist view of objects as sensations bound together through experience and memory, and he advanced the concept of unconscious inference to account for the attribution of perceptions to specific environmental causes. Wundt pressed further with the objectification and deconstruction of psychological reality by spelling out the concept -- implicit in the manifestoes of his associationist predecessors -- of elementism. Although Wundt surely believed that the meaning of sensory events lay in the relationship between them, elementism held that any complex association of sensations -- any perception -- was reducible to the sensory elements themselves. Titchener echoed the Wundtian view and elaborated upon the critical role of context in the associative extraction of meaning from sensation.
It was largely in response to this doctrine of elementism, its spreading influence, and its corrupt reductionistic account of perceptual experience that GESTALT PSYCHOLOGY was born in the late nineteenth century. In simplest terms, the Gestalt theorists, led by the venerable trio of Max Wertheimer (1880-1943), Wolfgang Kohler (1887 - 1967), and Kurt Koffka (1886 - 1941), insisted -- and backed up their insistence with innumerable compelling demonstrations -- that our phenomenal experience of objects, which includes an appreciation of their meanings and functions, is not generally reducible to a set of elemental sensations and the relationships between them. Moreover, rather than accepting the received wisdom that perception amounts to an inference about the world drawn from the associations between sensations, the Gestalt theorists held the converse to be true: perception is native experience and efforts to identify the underlying sensory elements are necessarily inferential (Koffka 1935). In spite of other flaws and peculiarities of the broad-ranging Gestalt psychology, this holistic view of perception, its distinction from sensation, and the nature of meaning, has become a central theme of modern cognitive neuroscience.
At the time the early associationist doctrine was being formed, there emerged a physiological counterpart in the form of Johannes Muller's (1801-1858) law of specific nerve energies, which gave rise in turn to the concept of specific fiber energies, and, ultimately, our twentieth- century receptive fields and feature detectors. Muller's law followed, intellectually as well as temporally, the Bell-Magendie law of distinct sensory and motor spinal roots, which set a precedent for the concept of specificity of nerve action. Muller's law was published in his 1838 Handbook of Physiology and consisted of several principles, those most familiar being the specificity of the sensory information (Muller identified five kinds) carried by different nerves and the specificity of the site of termination in the brain (a principle warmly embraced by functional localizationists of the era). For present discussion, the essential principle is that "the immediate objects of the perception of our senses are merely particular states induced in the nerves, and felt as sensations either by the nerves themselves or by the sensorium" (Boring 1950). Muller thus sidestepped the ancient problem of the mind"s access to the external world by observing that all it can hope to access is the state of its sensory nerves. Accordingly, perception of the external world is a consequence of the stable relationship between external stimuli and nerve activation, and -- tailing the associationist philosophers -- meaning is granted by the associative interactions between nerves carrying different types of information. The concept was elaborated further by Helmholtz and others to address the different submodalities (e.g., color vs. visual distance) and qualities (e.g., red vs. green) of information carried by different fibers, and is a tenet of contemporary sensory neurobiology and cognitive neuroscience. The further implications of associationism for an understanding of the neuronal basis of perception -- or, more precisely, of functional knowledge of the world -- are profound and, as we shall see, many of the nineteenth-century debates on the topic are being replayed in the courts of modern single-neuron physiology.
And so it was from these ancient but rapidly converging lines of inquiry, with the blush still on the cheek of a young cognitive science, that the modern era of cognitive neuroscience began. The field continues to ride a groundswell of optimism borne by new experimental tools and concepts -- particularly single-cell electrophysiology, functional brain imaging, molecular genetic manipulations, and neuronal computation -- and the access they have offered to neuronal operations underlying cognition. The current state of the field and its promise of riches untapped can be summarized through a survey of the processes involved in the acquisition, storage, and use of information by the nervous system: sensation, perception, decision formation, motor control, memory, language, emotions, and consciousness.
We acquire knowledge of the world through our senses. Not surprisingly, sensory processes are among the most thoroughly studied in cognitive neuroscience. Systematic explorations of these processes originated in two domains. The first consisted of investigations of the physical nature of the sensory stimuli in question, such as the wave nature of light and sound. Sir Isaac Newton's (1642-1727) Optiks is an exemplar of this approach. The second involved studies of the anatomy of the peripheral sense organs, with attention given to the manner in which anatomical features prepared the physical stimulus for sensory transduction. Von Bekesy's beautiful studies of the structural features of the cochlea and the relation of those features to the neuronal frequency coding of sound is a classic example (for which he was awarded the 1961 Nobel Prize in physiology and medicine). Our present understanding of the neuronal bases of sensation was further enabled by three major developments: (1) establishment of the neuron doctrine, with attendant anatomical and physiological studies of neurons; (2) systematization of behavioral studies of sensation, made possible through the development of psychophysics; and (3) advancement of sophisticated theories of neuronal function, as embodied by the discipline of COMPUTATIONAL NEUROSCIENCE. For a variety of reasons, vision has emerged as the model for studies of sensory processing, although many fundamental principles of sensory processing are conserved across modalities.
Initial acquisition of information about the world, by all sensory modalities, begins with a process known as transduction, by which forms of physical energy (e.g., photons) alter the electrical state of a sensory neuron. In the case of vision, phototransduction occurs in the RETINA, which is a specialized sheet-like neural network with a regular repeating structure. In addition to its role in transduction, the retina also functions in the initial detection of spatial and temporal contrast (Enroth-Cugell and Robson 1966; Kaplan and Shapley 1986) and contains specialized neurons that subserve COLOR VISION (see also COLOR, NEUROPHYSIOLOGY OF). The outputs of the retina are carried by a variety of ganglion cell types to several distinct termination sites in the central nervous system. One of the largest projections forms the "geniculostriate" pathway, which is known to be critical for normal visual function in primates. This pathway ascends to the cerebral cortex by way of the lateral geniculate nucleus of the THALAMUS.
The cerebral cortex itself has been a major focus of study during the past forty years of vision research (and sensory research of all types). The entry point for ascending visual information is via primary visual cortex, otherwise known as striate cortex or area V1, which lies on the posterior pole (the occipital lobe) of the cerebral cortex in primates. The pioneering studies of V1 by Hubel and Wiesel (1977) established the form in which visual information is represented by the activity of single neurons and the spatial arrangement of these representations within the cortical mantle ("functional architecture"). With the development of increasingly sophisticated techniques, our understanding of cortical VISUAL ANATOMY AND PHYSIOLOGY, and their relationships to sensory experience, has been refined considerably. Several general principles have emerged:
Receptive Field This is an operationally defined attribute of a sensory neuron, originally offered by the physiologist Haldan Keffer Hartline, which refers to the portion of the sensory field that, when stimulated, elicits a change in the electrical state of the cell. More generally, the receptive field is a characterization of the filter properties of a sensory neuron, which are commonly multidimensional and include selectivity for parameters such as spatial position, intensity, and frequency of the physical stimulus. Receptive field characteristics thus contribute to an understanding of the information represented by the brain, and are often cited as evidence for the role of a neuron in specific perceptual and cognitive functions.
Contrast Detection The elemental sensory operation, that is, one carried out by all receptive fields -- is detection of spatial or temporal variation in the incoming signal. It goes without saying that if there are no environmental changes over space and time, then nothing in the input is worthy of detection. Indeed, under such constant conditions sensory neurons quickly adapt. The result is a demonstrable loss of sensation -- such as "snow blindness" -- that occurs even though there may be energy continually impinging on the receptor surface. On the other hand, contrast along some sensory dimension indicates a change in the environment, which may in turn be a call for action. All sensory modalities have evolved mechanisms for detection of such changes.
Topographic Organization Representation of spatial patterns of activation within a sensory field is a key feature of visual, auditory, and tactile senses, which serves the behavioral goals of locomotor navigation and object recognition. Such representations are achieved for these modalities, in part, by topographically organized neuronal maps. In the visual system, for example, the retinal projection onto the lateral geniculate nucleus of the thalamus possesses a high degree of spatial order, such that neurons with spatially adjacent receptive fields lie adjacent to one another in the brain. Similar visuotopic maps are seen in primary visual cortex and in several successively higher levels of processing (e.g., Gattass, Sousa, and Covey 1985). These maps are commonly distorted relative to the sensory field, such that, in the case of vision, the numbers of neurons representing the central portion of the visual field greatly exceed those representing the visual periphery. These variations in "magnification factor" coincide with (and presumably underlie) variations in the observer"s resolving power and sensitivity.
Modular and Columnar Organization The proposal that COLUMNS AND MODULES form the basis for functional organization in the sensory neocortex is a natural extension of the nineteenth-century concept of localization of function. The 1970s and 1980s saw a dramatic rise in the use of electrophysiological and anatomical tools to subdivide sensory cortices -- particularly visual cortex -- into distinct functional modules. At the present time, evidence indicates that the visual cortex of monkeys is composed of over thirty such regions, including the well-known and heavily studied areas V1, V2, V3, V4, MT, and IT, as well as some rather more obscure and equivocal designations (Felleman and Van Essen 1991). These efforts to reveal order in heterogeneity have been reinforced by the appealing computational view (e.g., Marr 1982) that larger operations (such as seeing) can be subdivided and assigned to dedicated task-specific modules (such as ones devoted to visual motion or color processing, for example). The latter argument also dovetails nicely with the nineteenth-century concept of elementism, the coincidence of which inspired a fevered effort to identify visual areas that process specific sensory "elements." Although this view appears to be supported by physiological evidence for specialized response properties in some visual areas -- such as a preponderance of motion-sensitive neurons in area MT (Albright 1993) and color-sensitive neurons in area V4 (Schein and Desimone 1990) -- the truth is that very little is yet known of the unique contributions of most other cortical visual areas.
Modular organization of sensory cortex also occurs at a finer spatial scale, in the form of regional variations in neuronal response properties and anatomical connections, which are commonly referred to as columns, patches, blobs, and stripes. The existence of a column-like anatomical substructure in the cerebral cortex has been known since the early twentieth century, following the work of Ramón y Cajal, Constantin von Economo (1876-1931), and Rafael Lorente de Nó. It was the latter who first suggested that this characteristic structure may have some functional significance (Lorento de Nó 1938). The concept of modular functional organization was later expanded upon by the physiologist Vernon B. Mountcastle (1957), who obtained the first evidence for columnar function through his investigations of the primate somatosensory system, and offered this as a general principle of cortical organization. The most well known examples of modular organization of the sort predicted by Mountcastle are the columnar systems for contour orientation and ocular dominance discovered in primary visual cortex in the 1960s by David Hubel and Torsten Wiesel (1968). Additional evidence for functional columns and for the veracity of Mountcastle's dictum has come from studies of higher visual areas, such as area MT (Albright, Desimone, and Gross 1984) and the inferior temporal cortex (Tanaka 1997). Other investigations have demonstrated that modular representations are not limited to strict columnar forms (Born and Tootell 1993; Livingstone and Hubel 1984) and can exist as relatively large cortical zones in which there is a common feature to the neuronal representation of sensory information (such as clusters of cells that exhibit a greater degree of selectivity for color, for example).
The high incidence of columnar structures leads one to wonder why they exist. One line of argument, implicit in Mountcastle's original hypothesis, is based on the need for adequate "coverage" -- that is, nesting the representation of one variable (such as preferred orientation of a visual contour) across changes in another (such as the topographic representation of the visual field) -- which makes good computational sense and has received considerable empirical support (Hubel and Wiesel 1977). Other arguments include those based on developmental constraints (Swindale 1980; Miller 1994; Goodhill 1997) and computational advantages afforded by representation of sensory features in a regular periodic structure (see COMPUTATIONAL NEUROANATOMY; Schwartz 1980).
Hierarchical Processing A consistent organizational feature of sensory systems is the presence of multiple hierarchically organized processing stages, through which incoming sensory information is represented in increasingly complex or abstract forms. The existence of multiple stages has been demonstrated by anatomical studies, and the nature of the representation at each stage has commonly been revealed through electrophysiological analysis of sensory response properties. As we have seen for the visual system, the first stage of processing beyond transduction of the physical stimulus is one in which a simple abstraction of light intensity is rendered, namely a representation of luminance contrast. Likewise, the outcome of processing in primary visual cortex is, in part, a representation of image contours -- formed, it is believed, by a convergence of inputs from contrast-detecting neurons at earlier stages (Hubel and Wiesel 1962). At successively higher stages of processing, information is combined to form representations of even greater complexity, such that, for example, at the pinnacle of the pathway for visual pattern processing -- a visual area known as inferior temporal (IT) cortex -- individual neurons encode complex, behaviorally significant objects, such as faces (see FACE RECOGNITION).
Parallel Processing In addition to multiple serial processing stages, the visual system is known to be organized in parallel streams. Incoming information of different types is channeled through a variety of VISUAL PROCESSING STREAMS, such that the output of each serves a unique function. This type of channeling occurs on several scales, the grossest of which is manifested as multiple retinal projections (typically six) to different brain regions. As we have noted, it is the geniculostriate projection that serves pattern vision in mammals. The similarly massive retinal projection to the midbrain superior colliculus (the "tectofugal" pathway) is known to play a role in orienting responses, OCULOMOTOR CONTROL, and MULTISENSORY INTEGRATION. Other pathways include a retinal projection to the hypothalamus, which contributes to the entrainment of circadian rhythms by natural light cycles.
Finer scale channeling of visual information is also known to exist, particularly in the case of the geniculostriate pathway (Shapley 1990). Both anatomical and physiological evidence (Perry, Oehler, and Cowey 1984; Kaplan and Shapley 1986) from early stages of visual processing support the existence of at least three subdivisions of this pathway, known as parvocellular, magnocellular, and the more recently identified koniocellular (Hendry and Yoshioka 1994). Each of these subdivisions is known to convey a unique spectrum of retinal image information and to maintain that information in a largely segregated form at least as far into the system as primary visual cortex (Livingstone and Hubel 1988).
Beyond V1, the ascending anatomical projections fall into two distinct streams, one of which descends ventrally into the temporal lobe, while the other courses dorsally to the parietal lobe. Analyses of the behavioral effects of lesions, as well as electrophysiological studies of neuronal response properties, have led to the hypothesis (Ungerleider and Mishkin 1982) that the ventral stream represents information about form and the properties of visual surfaces (such as their color or TEXTURE) -- and is thus termed the "what" pathway -- while the dorsal stream represents information regarding motion, distance, and the spatial relations between environmental surfaces -- the so-called "where" pathway. The precise relationship, if any, between the early-stage channels (magno, parvo, and konio) and these higher cortical streams has been a rich source of debate and controversy over the past decade, and the answers remain far from clear (Livingstone and Hubel 1988; Merigan and Maunsell 1993).
Perception reflects the ability to derive meaning from sensory experience, in the form of information about structure and causality in the perceiver"s environment, and of the sort necessary to guide behavior. Operationally, we can distinguish sensation from perception by the nature of the internal representations: the former encode the physical properties of the proximal sensory stimulus (the retinal image, in the case of vision), and the latter reflect the world that likely gave rise to the sensory stimulus (the visual scene). Because the mapping between sensory and perceptual events is never unique -- multiple scenes can cause the same retinal image -- perception is necessarily an inference about the probable causes of sensation.
As we have seen, the standard approach to understanding the information represented by sensory neurons, which has evolved over the past fifty years, is to measure the correlation between a feature of the neuronal response (typically magnitude) and some physical parameter of a sensory stimulus (such as the wavelength of light or the orientation of a contour). Because the perceptual interpretation of a sensory event is necessarily context-dependent, this approach alone is capable of revealing little, if anything, about the relationship between neuronal events and perceptual state. There are, however, some basic variations on this approach that have led to increased understanding of the neuronal bases of perception.
Origins of a Neuron Doctrine for Perceptual Psychology The first strategy involves evaluation of neuronal responses to visual stimuli that consist of complex objects of behavioral significance. The logic behind this approach is that if neurons are found to be selective for such stimuli, they may be best viewed as representing something of perceptual meaning rather than merely coincidentally selective for the collection of sensory features. The early studies of "bug detectors" in the frog visual system by Lettvin and colleagues (Lettvin, Maturana, MCCULLOCH, and PITTS 1959) exemplify this approach and have led to fully articulated views on the subject, including the concept of the "gnostic unit" advanced by Jerzy Konorski (1967) and the "cardinal cell" hypothesis from Barlow"s (1972) classic "Neuron Doctrine for Perceptual Psychology." Additional evidence in support of this concept came from the work of Charles Gross in the 1960s and 1970s, in the extraordinary form of cortical cells selective for faces and hands (Gross, Bender, and Rocha-Miranda 1969; Desimone et al. 1984). Although the suggestion that perceptual experience may be rooted in the activity of single neurons or small neuronal ensembles has been decried, in part, on the grounds that the number of possible percepts greatly exceeds the number of available neurons, and is often ridiculed as the "grandmother-cell" hypothesis, the evidence supporting neuronal representations for visual patterns of paramount behavioral significance, such as faces, is now considerable (Desimone 1991; Rolls 1992).
Although a step in the right direction, the problem with this general approach is that it relies heavily upon assumptions about how the represented information is used. If a cell is activated by a face, and only a face, then it seems likely that the cell contributes directly to the perceptually meaningful experience of face recognition rather than simply representing a collection of sensory features (Desimone et al. 1984). To some, that distinction is unsatisfactorily vague, and it is, in any case, impossible to prove that a cell only responds to a face. An alternative approach that has proved quite successful in recent years is one in which an effort is made to directly relate neuronal and perceptual events.
Neuronal Discriminability Predicts Perceptual Discriminability In the last quarter of the twentieth century, the marriage of single-neuron recording with visual psychophysics has yielded one of the dominant experimental paradigms of cognitive neuroscience, through which it has become possible to explain behavioral performance on a perceptual task in terms of the discriminative capacity of sensory neurons. The earliest effort of this type was a study of tactile discrimination conducted by Vernon Mountcastle in the 1960s (Mountcastle et al. 1967). In this study, thresholds for behavioral discrimination performance were directly compared to neuronal thresholds for the same stimulus set. A later study by Tolhurst, Movshon, and Dean (1983) introduced techniques from SIGNAL DETECTION THEORY that allowed more rigorous quantification of the discriminative capacity of neurons and thus facilitated neuronal-perceptual comparisons. Several other studies over the past ten years have significantly advanced this cause (e.g., Dobkins and Albright 1995), but the most direct approach has been that adopted by William Newsome and colleagues (e.g., Newsome, Britten, and Movshon 1989). In this paradigm, behavioral and neuronal events are measured simultaneously in response to a sensory stimulus, yielding by brute force some of the strongest evidence to date for neural substrates of perceptual discriminability.
Decoupling Sensation and Perception A somewhat subtler approach has been forged by exploiting the natural ambiguity between sensory events and perceptual experience (see ILLUSIONS). This ambiguity is manifested in two general forms: (1) single sensory events that elicit multiple distinct percepts, a phenomenon commonly known as "perceptual metastability," and (2) multiple sensory events -- "sensory synonyms" -- that elicit the same perceptual state. Both of these situations, which are ubiquitous in normal experience, afford opportunities to experimentally decouple sensation and perception.
The first form of sensory-perceptual ambiguity (perceptual metastability) is a natural consequence of the indeterminate mapping between a sensory signal and the physical events that gave rise to it. A classic and familiar example is the Necker Cube, in which the three-dimensional interpretation -- the observer"s inference about visual scene structure -- periodically reverses despite the fact that the retinal image remains unchanged. Logothetis and colleagues (Logothetis and Schall 1989) have used a form of perceptual metastability known as binocular rivalry to demonstrate the existence of classes of cortical neurons that parallel changes in perceptual state in the face of constant retinal inputs.
The second type of sensory-perceptual ambiguity, in which multiple sensory images give rise to the same percept, is perhaps the more common. Such effects are termed perceptual constancies, and they reflect efforts by sensory systems to reconstruct behaviorally significant attributes of the world in the face of variation along irrelevant sensory dimensions. Size constancy -- the invariance of perceived size of an object across different retinal sizes -- and brightness or color constancy -- the invariance of perceived reflectance or color of a surface in the presence of illumination changes -- are classic examples. These perceptual constancies suggest an underlying neuronal invariance across specific image changes. Several examples of neuronal constancies have been reported, including invariant representations of direction of motion and shape across different cues for form (Albright 1992; Sary et al. 1995).
Contextual Influences on Perception and its Neuronal Bases One of the most promising new approaches to the neuronal bases of perception is founded on the use of contextual manipulations to influence the perceptual interpretation of an image feature. As we have seen, the contextual dependence of perception is scarcely a new finding, but contextual manipulations have been explicitly avoided in traditional physiological approaches to sensory coding. As a consequence, most existing data do not reveal whether and to what extent the neuronal representation of an image feature is context dependent. Gene Stoner, Thomas Albright, and colleagues have pioneered the use of contextual manipulations in studies of the neuronal basis of the PERCEPTION OF MOTION (e.g., Stoner and Albright 1992, 1993). The results of these studies demonstrate that context can alter neuronal filter properties in a manner that predictably parallels its influence on perception.
Several lines of evidence suggest that there may be multiple steps along the path to extracting meaning from sensory signals. These steps are best illustrated by examples drawn from studies of visual processing. Sensation itself is commonly identified with "early" or "low-level vision." Additional steps are as follows.
Mid-Level Vision This step involves a reconstruction of the spatial relationships between environmental surfaces. It is implicit in the accounts of the perceptual psychologist JAMES JEROME GIBSON (1904-1979), present in the computational approach of DAVID MARR (1945 - 1980), and encompassed by what has recently come to be known as MID-LEVEL VISION. Essential features of this processing stage include a dependence upon proximal sensory context to establish surface relationships (see SURFACE PERCEPTION) and a relative lack of dependence upon prior experience. By establishing environmental STRUCTURE FROM VISUAL INFORMATION SOURCES, mid-level vision thus invests sensory events with some measure of meaning. A clear example of this type of visual processing is found in the phenomenon of perceptual TRANSPARENCY (Metelli 1974) and the related topic of LIGHTNESS PERCEPTION. Physiological studies of the response properties of neurons at mid-levels of the cortical hierarchy have yielded results consistent with a mid-level representation (e.g., Stoner and Albright 1992).
High-Level Vision HIGH-LEVEL VISION is a loosely defined processing stage, but one that includes a broad leap in the assignment of meaning to sensory events -- namely identification and classification on the basis of previous experience with the world. It is through this process that recognition of objects occurs (see OBJECT RECOGNITION, HUMAN NEUROPSYCHOLOGY; OBJECT RECOGNITION, ANIMAL STUDIES; and VISUAL OBJECT RECOGNITION, AI), as well as assignment of affect and semantic categorization. This stage thus constitutes a bridge between sensory processing and MEMORY. Physiological and neuropsychological studies of the primate temporal lobe have demonstrated an essential contribution of this region to object recognition (Gross 1973; Gross et al. 1985).
The processes by which information is acquired and interpreted by the brain are modifiable throughout life and on many time scales. Although plasticity of the sort that occurs during brain development and that which underlies changes in the sensitivity of mature sensory systems may arise from similar mechanisms, it is convenient to consider them separately.
The development of the mammalian nervous system is a complex, multistaged process that extends from embryogenesis through early postnatal life. This process begins with determination of the fate of precursor cells such that a subset becomes neurons. This is followed by cell division and proliferation, and by differentiation of cells into different types of neurons. The patterned brain then begins to take shape as cells migrate to destinations appropriate for their assigned functions. Finally, neurons begin to extend processes and to make synaptic connections with one another. These connections are sculpted and pruned over a lengthy postnatal period. A central tenet of modern neuroscience is that these final stages of NEURAL DEVELOPMENT correspond to specific stages of COGNITIVE DEVELOPMENT. These stages are known as "critical periods," and they are characterized by an extraordinary degree of plasticity in the formation of connections and cognitive functions.
Although critical periods for development are known to exist for a wide range of cognitive functions such as sensory processing, motor control, and language, they have been studied most intensively in the context of the mammalian visual system. These studies have included investigations of the timing, necessary conditions for, and mechanisms of (1) PERCEPTUAL DEVELOPMENT (e.g., Teller 1997), (2) formation of appropriate anatomical connections (e.g., Katz and Shatz 1996), and (3) neuronal representations of sensory stimuli (e.g., Hubel, Wiesel, and LeVay 1977). The general view that has emerged is that the newborn brain possesses a considerable degree of order, but that sensory experience is essential during critical periods to maintain that order and to fine-tune it to achieve optimal performance in adulthood. These principles obviously have profound implications for clinical practice and social policy. Efforts to further understand the cellular mechanisms of developmental plasticity, their relevance to other facets of cognitive function, the relative contributions of genes and experience, and routes of clinical intervention, are all among the most important topics for the future of cognitive neuroscience.
Mature sensory systems have limited information processing capacities. An exciting area of research in recent years has been that addressing the conditions under which processing capacity is dynamically reallocated, resulting in fluctuations in sensitivity to sensory stimuli. The characteristics of sensitivity changes are many and varied, but all serve to optimize acquisition of information in a world in which environmental features and behavioral goals are constantly in flux. The form of these changes may be broad in scope or highly stimulus-specific and task-dependent. Changes may be nearly instantaneous, or they may come about gradually through exposure to specific environmental features. Finally, sensitivity changes differ greatly in the degree to which they are influenced by stored information about the environment and the degree to which they are under voluntary control.
Studies of the visual system reveal at least three types of sensitivity changes represented by the phenomena of (1) contrast gain control, (2) attention, and (3) perceptual learning. All can be viewed as recalibration of incoming signals to compensate for changes in the environment, the fidelity of signal detection (such as that associated with normal aging or trauma to the sensory periphery), and behavioral goals.
Generally speaking, neuronal gain control is the process by which the sensitivity of a neuron (or neural system) to its inputs is dynamically controlled. In that sense, all of the forms of adult plasticity discussed below are examples of gain control, although they have different dynamics and serve different functions.
Contrast Gain Control A well-studied example of gain control is the invariance of perceptual sensitivity to the features of the visual world over an enormous range of lighting conditions. Evidence indicates that the limited dynamic range of responsivity of individual neurons in visual cortex is adjusted in an illumination-dependent manner (Shapley and Victor 1979), the consequence of which is a neuronal invariance that can account for the sensory invariance. It has been suggested that this scaling of neuronal sensitivity as a function of lighting conditions may be achieved by response "normalization," in which the output of a cortical neuron is effectively divided by the pooled activity of a large number of other cells of the same type (Carandini, Heeger, and Movshon 1997).
Attention Visual ATTENTION is, by definition, a rapidly occurring change in visual sensitivity that is selective for a specific location in space or specific stimulus features. The stimulus and mnemonic factors that influence attentional allocation have been studied for over a century (James 1890), and the underlying brain structures and events are beginning to be understood (Desimone and Duncan 1995). Much of our understanding comes from analysis of ATTENTION IN THE HUMAN BRAIN -- particularly the effects of cortical lesions, which can selectively interfere with attentional allocation (VISUAL NEGLECT), and through electrical and magnetic recording (ERP, MEG) and imaging studies -- POSITRON EMISSION TOMOGRAPHY (PET) and functional MAGNETIC RESONANCE IMAGING (fMRI). In addition, studies of ATTENTION IN THE ANIMAL BRAIN have revealed that attentional shifts are correlated with changes in the sensitivity of single neurons to sensory stimuli (Moran and Desimone 1985; Bushnell, Goldberg, and Robinson 1981; see also AUDITORY ATTENTION). Although attentional phenomena differ from contrast gain control in that they can be influenced by feedback WORKING MEMORY as well as feedforward (sensory) signals, attentional effects can also be characterized as an expansion of the dynamic range of sensitivity, but in a manner that is selective for the attended stimuli.
Perceptual Learning Both contrast gain control and visual attention are rapidly occurring and short-lived sensitivity changes. Other experiments have targeted neuronal events that parallel visual sensitivity changes occurring over a longer time scale, such as those associated with the phenomenon of perceptual learning. Perceptual learning refers to improvements in discriminability along any of a variety of sensory dimensions that come with practice. Although it has long been known that the sensitivity of the visual system is refined in this manner during critical periods of neuronal development, recent experiments have provided tantalizing evidence of improvements in the sensitivity of neurons at early stages of processing, which parallel perceptual learning in adults (Recanzone, Schreiner, and Merzenich 1993; Gilbert 1996).
The meaning of many sensations can be found solely in their symbolic and experience-dependent mapping onto actions (e.g., green = go, red = stop). These mappings are commonly many-to-one or one-to-many (a whistle and a green light can both be signals to "go"; conversely, a whistle may be either a signal to "go" or a call to attention, depending upon the context). The selection of a particular action from those possible at any point in time is thus a context-dependent transition between sensory processing and motor control. This transition is commonly termed the decision stage, and it has become a focus of recent electrophysiological studies of the cerebral cortex (e.g., Shadlen and Newsome 1996). Because of the nonunique mappings, neurons involved in making such decisions should be distinguishable from those representing sensory events by a tendency to generalize across specific features of the sensory signal. Similarly, the representation of the neuronal decision should be distinguishable from a motor control signal by generalization across specific motor actions. In addition, the strength of the neuronal decision signal should increase with duration of exposure to the sensory stimulus (integration time), in parallel with increasing decision confidence on the part of the observer. New data in support of some of these predictions suggests that this may be a valuable new paradigm for accessing the neuronal substrates of internal cognitive states, and for bridging studies of sensory or perceptual processing, memory, and motor control.
Incoming sensory information ultimately leads to action, and actions, in turn, are often initiated in order to acquire additional sensory information. Although MOTOR CONTROL systems have often been studied in relative isolation from sensory processes, this sensory-motor loop suggests that they are best viewed as different phases of a processing continuum. This integrated view, which seeks to understand how the nature of sensory representations influences movements, and vice-versa, is rapidly gaining acceptance. The oculomotor control system has become the model for the study of motor processes at behavioral and neuronal levels.
Important research topics that have emerged from consideration of the transition from sensory processing to motor control include (1) the process by which representations of space (see SPATIAL PERCEPTION) are transformed from the coordinate system of the sensory field (e.g., retinal space) to a coordinate system for action (e.g., Graziano and Gross 1998) and (2) the processes by which the neuronal links between sensation and action are modifiable (Raymond, Lisberger, and Mauk 1996), as needed to permit MOTOR LEARNING and to compensate for degenerative sensory changes or structural changes in the motor apparatus.
The brain structures involved in motor control include portions of the cerebral cortex, which are thought to contribute to fine voluntary motor control, as well as the BASAL GANGLIA and CEREBELLUM, which play important roles in motor learning; the superior colliculus, which is involved in sensorimotor integration, orienting responses, and oculomotor control; and a variety of brainstem motor nuclei, which convey motor signals to the appropriate effectors.
Studies of the neuronal mechanisms that enable information about the world to be stored and retrieved for later use have a long and rich history -- being, as they were, a central part of the agenda of the early functional localizationists -- and now lie at the core of our modern cognitive neuroscience. Indeed, memory serves as the linchpin that binds and shapes nearly every aspect of information processing by brains, including perception, decision making, motor control, emotion, and consciousness. Memory also exists in various forms, which have been classified on the basis of their relation to other cognitive functions, the degree to which they are explicitly encoded and available for use in a broad range of contexts, and their longevity. (We have already considered some forms of nonexplicit memory, such as those associated with perceptual and motor learning.) Taxonomies based upon these criteria have been reviewed in detail elsewhere (e.g., Squire, Knowlton, and Musen 1993). The phenomenological and functional differences among different forms of memory suggest the existence of a variety of different brain substrates. Localization of these substrates is a major goal of modern cognitive neuroscience. Research is also clarifying the mechanisms underlying the oft-noted role of affective or emotional responses in memory consolidation (see MEMORY STORAGE, MODULATION OF; AMYGDALA, PRIMATE), and the loss of memory that occurs with aging (see AGING, MEMORY, AND THE BRAIN).
Three current approaches (broadly defined and overlapping) to memory are among the most promising for the future of cognitive neuroscience: (1) neuropsychological and neurophysiological studies of the neuronal substrates of explicit memory in primates, (2) studies of the relationship between phenomena of synaptic facilitation or depression and behavioral manifestations of learning and memory, and (3) molecular genetic studies that enable highly selective disruption of cellular structures and events thought to be involved in learning and memory.
The current approach to this topic has its origins in the early studies of Karl Lashley and colleagues, in which the lesion method was used to infer the contributions of specific brain regions to a variety of cognitive functions, including memory. The field took a giant step forward in the 1950s with the discovery by Brenda Milner and colleagues of the devastating effects of damage to the human temporal lobe -- particularly the HIPPOCAMPUS -- on human memory formation (see MEMORY, HUMAN NEUROPSYCHOLOGY). Following that discovery, Mortimer Mishkin and colleagues began to use the lesion technique to develop an animal model of amnesia. More recently, using a similar approach, Stuart Zola, Larry Squire, and colleagues have further localized the neuronal substrates of memory consolidation in the primate temporal lobe (see MEMORY, ANIMAL STUDIES).
Electrophysiological studies of the contributions of individual cortical neurons to memory began in the 1970s with the work of Charles Gross and Joaquin Fuster. The logic behind this approach is that by examining neuronal responses of an animal engaged in a standard memory task (e.g., match-to-sample: determine whether a sample stimulus corresponds to a previously viewed cue stimulus), one can distinguish the components of the response that reflect memory from those that are sensory in nature. Subsequent electrophysiological studies by Robert Desimone and Patricia Goldman-Rakic, among others, have provided some of the strongest evidence for single-cell substrates of working memory in the primate temporal and frontal lobes. These traditional approaches to explicit memory formation in primates are now being complemented by brain imaging studies in humans.
The phenomenon of LONG-TERM POTENTIATION (LTP), originally discovered in the 1970s -- and the related phenomenon of long-term depression -- consists of physiologically measurable changes in the strength of synaptic connections between neurons. LTP is commonly produced in the laboratory by coincident activation of pre- and post-synaptic neurons, in a manner consistent with the predictions of DONALD O. HEBB (1904-1985), and it is often dependent upon activation of the postsynaptic NMDA glutamate receptor. Because a change in synaptic efficacy could, in principle, underlie behavioral manifestations of learning and memory, and because LTP is commonly seen in brain structures that have been implicated in memory formation (such as the hippocampus, cerebellum, and cerebral cortex) by other evidence, it is considered a likely mechanism for memory formation. Attempts to test that hypothesis have led to one of the most exciting new approaches to memory.
The knowledge that the NMDA receptor is responsible for many forms of LTP, in conjunction with the hypothesis that LTP underlies memory formation, led to the prediction that memory formation should be disrupted by elimination of NMDA receptors. The latter can be accomplished in mice by engineering genetic mutations that selectively knock out the NMDA receptor, although this technique has been problematic because it has been difficult to constrain the effects to specific brain regions and over specific periods of time. Matthew Wilson and Susumu Tonegawa have recently overcome these obstacles by production of a knockout in which NMDA receptors are disrupted only in a subregion of the hippocampus (the CA1 layer), and only after the brain has matured. In accordance with the NMDA-mediated synaptic plasticity hypothesis, these animals were deficient on both behavioral and physiological assays of memory formation (Tonegawa et al. 1996). Further developments along these lines will surely involve the ability to selectively disrupt action potential generation in specific cell populations, as well as genetic manipulations in other animals (such as monkeys).
One of the first cognitive functions to be characterized from a biological perspective was language. Nineteenth-century physicians, including Broca, observed the effects of damage to different brain regions and described the asymmetrical roles of the left and right hemispheres in language production and comprehension (see HEMISPHERIC SPECIALIZATION; APHASIA; LANGUAGE, NEURAL BASIS OF). Investigators since then have discovered that different aspects of language, including the PHONOLOGY, SYNTAX, and LEXICON, each rely on different and specific neural structures (see PHONOLOGY, NEURAL BASIS OF; GRAMMAR, NEURAL BASIS OF; LEXICON, NEURAL BASIS OF). Modern neuroimaging techniques, including ERPs, PET, and fMRI, have confirmed the role of the classically defined language areas and point to the contribution of several other areas as well. Such studies have also identified "modality neutral" areas that are active when language is processed through any modality: auditory, written, and even sign language (see SIGN LANGUAGE AND THE BRAIN). Studies describing the effects of lesions on language can identify neural tissue that is necessary and sufficient for processing. An important additional perspective can be obtained from neuroimaging studies of healthy neural tissue, which can reveal all the activity associated with language production and comprehension. Taken together the currently available evidence reveals a strong bias for areas within the left hemisphere to mediate language if learned early in childhood, independently of its form or modality. However, the nature of the language learned and the age of acquisition have effects on the configuration of the language systems of the brain (see BILINGUALISM AND THE BRAIN).
Developmental disorders of language (see LANGUAGE IMPAIRMENT, DEVELOPMENTAL; DYSLEXIA) can occur in isolation or in association with other disorders and can result from deficits within any of the several different skills that are central to the perception and modulation of language.
Rediscovery of the phenomena of perception and memory without awareness has renewed research and debate on issues concerning the neural basis of CONSCIOUSNESS (see CONSCIOUSNESS, NEUROBIOLOGY OF). Some patients with cortical lesions that have rendered them blind can nonetheless indicate (by nonverbal methods) accurate perception of stimuli presented to the blind portion of the visual field (see BLINDSIGHT). Similarly, some patients who report no memory for specific training events nonetheless demonstrate normal learning of those skills.
Systematic study of visual consciousness employing several neuroimaging tools within human and nonhuman primates is being conducted to determine whether consciousness emerges as a property of a large collection of interacting neurons or whether it arises as a function of unique neuronal characteristics possessed by some neurons or by an activity pattern temporarily occurring within a subset of neurons (see BINDING BY NEURAL SYNCHRONY).
Powerful insights into systems and cellular and molecular events critical in cognition and awareness, judgment and action have come from human and animal studies of SLEEP and DREAMING. Distinct neuromodulatory effects of cholenergic and aminergic systems permit the panoply of conscious cognitive processing, evaluation, and planning during waking states and decouple cognition, emotional, and mnemonic functions during sleep. Detailed knowledge of the neurobiology of sleep and dreaming presents an important opportunity for future studies of cognition and consciousness.
Closely related to questions about consciousness are issues of EMOTIONS and feelings that have, until very recently, been ignored in cognitive science. Emotions sit at the interface between incoming events and preparation to respond, however, and recent studies have placed the study of emotion more centrally in the field. Animal models have provided detailed anatomical and physiological descriptions of fear responses (Armony and LeDoux 1997) and highlight the role of the amygdala and LIMBIC SYSTEM as well as different inputs to this system (see EMOTION AND THE ANIMAL BRAIN). Studies of human patients suggest specific roles for different neural systems in the perception of potentially emotional stimuli (Adolphs et al. 1994; Hamann et al. 1996), in their appraisal, and in organizing appropriate responses to them (see EMOTION AND THE HUMAN BRAIN; PAIN). An important area for future research is to characterize the neurochemistry of emotions. The multiple physiological responses to real or imagined threats (i.e., STRESS) have been elucidated in both animal and human studies. Several of the systems most affected by stress play central roles in emotional and cognitive functions (see NEUROENDOCRINOLOGY). Early pre- and postnatal experiences play a significant role in shaping the activity of these systems and in their rate of aging. The profound role of the stress-related hormones on memory-related brain structures, including the hippocampus, and their role in regulating neural damage following strokes and seizures and in aging, make them a central object for future research in cognitive neuroscience (see AGING AND COGNITION).
A glance at the neuroscience entries for this volume reveals that we are amassing detailed knowledge of the highly specialized neural systems that mediate different and specific cognitive functions. Many questions remain unanswered, however, and the applications of new experimental techniques have often raised more questions than they have answered. But such are the expansion pains of a thriving science.
Among the major research goals of the next century will be to elucidate how these highly differentiated cognitive systems arise in ontogeny, the degree to which they are maturationally constrained, and the nature and the timing of the role of input from the environment in NEURAL DEVELOPMENT. This is an area where research has just begun. It is evident that there exist strong genetic constraints on the overall patterning of different domains within the developing nervous system. Moreover, the same class of genes specify the rough segmentation of the nervous systems of both vertebrates and invertebrates. However, the information required to specify the fine differentiation and connectivity within the cortex exceeds that available in the genome. Instead, a process of selective stabilization of transiently redundant connections permits individual differences in activity and experience to organize developing cortical systems. Some brain circuits display redundant connectivity and pruning under experience only during a limited time period in development ("critical period"). These time periods are different for different species and for different functional brain systems within a species. Other brain circuits retain the ability to change under external stimulation throughout life, and this capability, which now appears more ubiquitous and long lasting than initially imagined, is surely a substrate for adult learning, recovery of function after brain damage, and PHANTOM LIMB phenomena (see also AUDITORY PLASTICITY; NEURAL PLASTICITY). A major challenge for future generations of cognitive neuroscientists will be to characterize and account for the markedly different extents and timecourses of biological constraints and experience-dependent modifiability of the developing human brain.
Though the pursuit may be ancient, consider these the halcyon days of cognitive neuroscience. As we cross the threshold of the millenium, look closely as the last veil begins to fall. And bear in mind that if cognitive neuroscience fulfills its grand promise, later editions of this volume may contain a section on history, into which all of the nonneuro cognitive science discussion will be swept.
Adolphs, R., D. Tranel, H. Damasio, and A. Damasio. (1994). Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature 372:669-672.
Albright, T. D. (1992). Form-cue invariant motion processing in primate visual cortex. Science 255:1141-1143.
Albright, T. D. (1993). Cortical processing of visual motion. In J. Wallman and F. A. Miles, Eds., Visual Motion and its Use in the Stabilization of Gaze. Amsterdam: Elsevier.
Albright, T. D., R. Desimone, and C. G. Gross. (1984). Columnar organization of directionally selective cells in visual area MT of the macaque. Journal of Neurophysiology 51:16-31.
Armony, J. L., and J. E. LeDoux. (1997). How the brain processes emotional information. Annals of the New York Academy of Sciences 821:259-270.
Barlow, H. B. (1972). Single units and sensation: A neuron doctrine for perceptual psychology? Perception 1:371-394.
Boring, E. G. (1950). A History of Experimental Psychology, 2nd ed. R. M. Elliott, Ed. New Jersey: Prentice-Hall.
Born, R. T., and R. B. Tootell. (1993). Segregation of global and local motion processing in primate middle temporal visual area. Nature 357:497-499.
Bushnell, M. C., M. E. Goldberg, and D. L. Robinson. (1981). Behavioral enhancement of visual responses in monkey cerebral cortex. 1. Modulation in posterior parietal cortex related to selective visual attention. Journal of Neurophysiology 46(4):755-772.
Callaway, E. M. (1998). Local circuits in primary visual cortex of the macaque monkey. Annual Review of Neuroscience 21:47-74.
Carandini, M., D. J. Heeger, and J. A. Movshon. (1997). Linearity and normalization in simple cells of the macaque primary visual cortex. Journal of Neuroscience 17(21):8621-8644.
Desimone, R. (1991). Face-selective cells in the temporal cortex of monkeys. Journal of Cognitive Neuroscience 3:1-7.
Desimone, R., T. D. Albright, C. G. Gross, and C. J. Bruce. (1984). Stimulus selective properties of inferior temporal neurons in the macaque. Journal of Neuroscience 8:2051-2062.
Desimone, R., and J. Duncan. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience 18:193-222.
Dobkins, K. R., and T. D. Albright. (1995). Behavioral and neural effects of chromatic isoluminance in the primate visual motion system. Visual Neuroscience 12:321-332.
Enroth-Cugell, C., and J. G. Robson. (1966). The contrast sensitivity of retinal ganglion cells of the cat. Journal of Physiology (London) 187:517-552.
Felleman, D. J., and D. C. Van Essen. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex 1:1-47.
Gattass, R., A. P. B. Sousa, and E. Cowey. (1985). Cortical visual areas of the macaque: Possible substrates for pattern recognition mechanisms. In C. Chagas, R. Gattass, and C. Gross, Eds., Pattern Recognition Mechanisms. Vatican City: Pontifica Academia Scientiarum, pp. 1-17.
Gilbert, C. D. (1996). Learning and receptive field plasticity. Proceedings of the National Academy of Sciences USA 93:10546-10547.
Goodhill, G. J. (1997). Stimulating issues in cortical map development. Trends in Neurosciences 20:375-376.
Graziano, M. S., and C. G. Gross. (1998). Spatial maps for the control of movement. Current Opinion in Neurobiology 8:195-201.
Gross, C. G. (1973). Visual functions of inferotemporal cortex. In H. Autrum, R. Jung, W. Lowenstein, D. McKay, and H.-L. Teuber, Eds., Handbook of Sensory Physiology, vol. 7, 3B. Berlin: Springer.
Gross, C. G. (1994a). How inferior temporal cortex became a visual area. Cerebral Cortex 5:455-469.
Gross, C. G. (1994b). Hans-Lukas Teuber: A tribute. Cerebral Cortex 4:451-454.
Gross, C. G. (1998). Brain, Vision, Memory: Tales in the History of Neuroscience. Cambridge, MA: MIT Press.
Gross, C. G., D. B. Bender, and C. E. Rocha-Miranda. (1969). Visual receptive fields of neurons in inferotemporal cortex of the monkey. Science 166:1303-1306.
Gross, C. G., R. Desimone, T. D. Albright, and E. L. Schwartz. (1985). Inferior temporal cortex and pattern recognition. In C. Chagas, Ed., Study Group on Pattern Recognition Mechanisms. Vatican City: Pontifica Academia Scientiarum, pp. 179-200.
Hamann, S. B., L. Stefanacci, L. R. Squire, R. Adolphs, D. Tranel, H. Damasio, and A. Damasio. (1996). Recognizing facial emotion [letter]. Nature 379(6565): 497.
Hartline, H. K., H. G. Wagner, and E. F. MacNichol, Jr. (1952). The peripheral origin of nervous activity in the visual system. Cold Spring Harbor Symposium on Quantitative Biology 17:125-141.
Hendry, S. H., and T. Yoshioka. (1994). A neurochemically distinct third channel in the macaque dorsal lateral geniculate nucleus. Science 264(5158):575-577.
Hubel, D. H., and T. N. Wiesel. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. Journal of Physiology 160:106-154.
Hubel, D. H., and T. N. Wiesel. (1968). Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology 195:215-243.
Hubel, D. H., and T. N. Wiesel. (1977). Ferrier lecture. Functional architecture of macaque monkey visual cortex. Proceedings of the Royal Society of London, Series B, Biological Sciences 198(1130):1-59.
Hubel, D. H., T. N. Wiesel, and S. LeVay. (1977). Plasticity of ocular dominance columns in monkey striate cortex. Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences 278:377-409.
James, W. (1890). The Principles of Psychology, vol. 1. New York: Dover.
Kaplan, E., and R. M. Shapley. (1986). The primate retina contains two types of ganglion cells, with high and low contrast sensitivity. Proceedings of the National Academy of Sciences of the USA 83(8):2755-2757.
Katz, L. C., and C. J. Shatz. (1996). Synaptic activity and the construction of cortical circuits. Science 274:1133-1138.
Koffka, K. (1935). Principles of Gestalt Psychology. New York: Harcourt, Brace.
Konorski, J. (1967). Integrative Activity of the Brain. Chicago: University of Chicago Press.
Kuffler, S. W. (1953). Discharge patterns and functional organization of the mammalian retina. Journal of Neurophysiology 16:37-68.
Lettvin, J. Y., H. R. Maturana, W. S. McCulloch, and W. H. Pitts. (1959). What the frog's eye tells the frog's brain. Proceedings of the Institute of Radio Engineers 47:1940-1951.
Livingstone, M. S., and D. H. Hubel. (1984). Anatomy and physiology of a color system in the primate visual cortex. Journal of Neuroscience 4:309-356.
Livingstone, M. S., and D. H. Hubel. (1988). Segregation of form, color, movement, and depth: Anatomy, physiology, and perception. Science 240:740-749.
Logothetis, N. K., and J. D. Schall. (1989). Neuronal correlates of subjective visual perception. Science 245:761-763.
Lorento de Nó, R. (1938). Cerebral cortex: Architecture, intracortical connections, motor projections. In J. F. Fulton, Ed., Physiology of the Nervous System. New York: Oxford University Press, pp. 291-339.
Marr, D. (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. San Francisco: W. H. Freeman.
Merigan, W. H., and J. H. Maunsell. (1993). How parallel are the primate visual pathways? Annual Review of Neuroscience 16:369-402.
Metelli, F. (1974). The perception of transparency. Scientific American 230(4):90-98.
Miller, K. D. (1994). A model for the development of simple cell receptive fields and the ordered arrangement of orientation columns through activity-dependent competition between ON- and OFF-center inputs. Journal of Neuroscience 14:409-441.
Moran, J., and R. Desimone. (1985). Selective attention gates visual processing in the extrastriate cortex. Science 229(4715):782-784.
Mountcastle, V. B. (1957). Modality and topographic properties of single neurons of cat"s somatic sensory cortex. Journal of Neurophysiology 20:408-434.
Mountcastle, V. B., W. H. Talbot, I. Darian-Smith, and H. H. Kornhuber. (1967). Neural basis of the sense of flutter-vibration. Science 155(762):597-600.
Newsome, W. T., K. H. Britten, and J. A. Movshon. (1989). Neuronal correlates of a perceptual decision. Nature 341:52-54.
Perry, V. H., R. Oehler, and A. Cowey. (1984). Retinal ganglion cells that project to the dorsal lateral geniculate nucleus in the macaque monkey. Neuroscience 12(4):1101-1123.
Raymond, J. L., S. G. Lisberger, and M. D. Mauk. (1996). The cerebellum: A neuronal learning machine? Science 272:1126-1131.
Recanzone, G. H., C. E. Schreiner, and M. M. Merzenich. (1993). Plasticity in the frequency representation of primary auditory cortex following discrimination training in adult owl monkeys. Journal of Neuroscience 13:87-103.
Rolls, E. T. (1992). Neurophysiological mechanisms underlying face processing within and beyond the temporal cortical visual areas. In V. Bruce, A. Cowey, W. W. Ellis, and D. I. Perrett, Eds., Processing the Facial Image. Oxford: Clarendon Press, pp. 11-21.
Sary, G., R. Vogels, G. Kovacs, and G. A. Orban. (1995). Responses of monkey inferior temporal neurons to luminance-, motion-, and texture-defined gratings. Journal of Neurophysiology 73:1341-1354.
Schein, S. J., and R. Desimone. (1990). Spectral properties of V4 neurons in the macaque. Journal of Neuroscience 10:3369-3389.
Schwartz, E. L. (1980). Computational anatomy and functional architecture of striate cortex: A spatial mapping approach to perceptual coding. Vision Research 20:645-669.
Shadlen, M. N., and W. T. Newsome. (1996). Motion perception: Seeing and deciding. Proceedings of the National Academy of Sciences 93:628-633.
Shapley, R. (1990). Visual sensitivity and parallel retinocortical channels. Annual Review of Psychology 41:635-658.
Shapley, R. M., and J. D. Victor. (1979). Nonlinear spatial summation and the contrast gain control of cat retinal ganglion cells. Journal of Physiology (London) 290:141-161.
Squire, L. R., B. Knowlton, and G. Musen. (1993). The structure and organization of memory. Annual Review of Psychology 44:453-495.
Stoner, G. R., and T. D. Albright. (1992). Neural correlates of perceptual motion coherence. Nature 358:412-414.
Stoner, G. R., and T. D. Albright. (1993). Image segmentation cues in motion processing: Implications for modularity in vision. Journal of Cognitive Neuroscience 5:129-149.
Swindale, N. V. (1980). A model for the formation of ocular dominance stripes. Proceedings of the Royal Society of London Series B, Biological Sciences 208(1171):243-264.
Tanaka, K. (1997). Columnar organization in the inferotemporal cortex. Cerebral Cortex 12:469-498.
Teller, D. Y. (1997). First glances: The vision of infants. The Friedenwald lecture. Investigative Ophthalmology and Visual Science 38:2183-2203.
Tolhurst, D. J., J. A. Movshon, and A. F. Dean. (1983). The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision Research 23(8):775-785.
Tonegawa, S., J. Z. Tsien, T. J. McHugh, P. Huerta, K. I. Blum, and M. A. Wilson. (1996). Hippocampal CA1-region-restricted knockout of NMDAR1 gene disrupts synaptic plasticity, place fields, and spatial learning. Cold Spring Harbor Symposium on Quantitative Biology 61:225-238.
Ungerleider, L. G., and M. Mishkin. (1982). Two cortical visual systems. In D. J. Ingle, M. A. Goodale, and R. J. W. Mansfield, Eds., Analysis of Visual Behavior. Cambridge, MA: MIT Press, pp. 549-586.
Zola-Morgan, S. (1995). Localization of brain function: The legacy of Franz Joseph Gall. Annual Review of Neuroscience 18:359-383.
Churchland, P. S., and T. J. Sejnowski. (1992). The Computational Brain. Cambridge, MA: MIT Press.
Cohen, N. J., and H. Eichenbaum. (1993). Memory, Amnesia, and the Hippocampal System. Cambridge, MA: MIT Press.
Dowling, J. E. (1987). The Retina: An Approachable Part of the Brain. Cambridge, MA: Belknap Press of Harvard University Press.
Finger, S. (1994). Origins of Neuroscience: A History of Explorations into Brain Function. New York: Oxford University Press.
Gazzaniga, M. S. (1995). The Cognitive Neurosciences. Cambridge, MA: MIT Press.
Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Boston: Houghton Mifflin.
Heilman, E. M., and E. Valenstein. (1985). Clinical Neuropsychology, 2nd ed. New York: Oxford University Press.
Helmholtz, H. von. (1924). Physiological Optics. English translation by J. P. C. Southall for the Optical Society of America from the 3rd German ed., Handbuch der Physiologischen Optik (1909). Hamburg: Voss.
Kanizsa, G. (1979). Organization in Vision. New York: Praeger.
Kosslyn, S. M. (1994). Image and Brain. Cambridge, MA: MIT Press.
LeDoux, J. E. (1996). The Emotional Brain: The Mysterious Underpinnings of Emotional Life. New York: Simon & Schuster.
Milner, A. D., and M. A. Goodale. (1995). The Visual Brain in Action. New York: Oxford University Press.
Penfield, W. (1975). The Mystery of the Mind. New Jersey: Princeton University Press.
Posner, M. L. (1989). Foundations of Cognitive Science. Cambridge, MA: MIT Press.
Squire, L. R. (1987). Memory and Brain. New York: Oxford University Press.
Weiskrantz, L. (1997). Consciousness Lost and Found. New York: Oxford University Press.
Copyright © 1999 Massachusetts Institute of Technology