What we see is how we are : New paradigms in visual research

As most sensory modalities, the visual system needs to deal with very fast changes in the environment. Instead of processing all sensory stimuli, the brain is able to construct a perceptual experience by combining selected sensory input with an ongoing internal activity. Thus, the study of visual perception needs to be approached by examining not only the physical properties of stimuli, but also the brain's ongoing dynamical states onto which these perturbations are imposed. At least three different models account for this internal dynamics. One model is based on cardinal cells where the activity of few cells by itself constitutes the neuronal correlate of perception, while a second model is based on a population coding that states that the neuronal correlate of perception requires distributed activity throughout many areas of the brain. A third proposition, known as the temporal correlation hypothesis states that the distributed neuronal populations that correlate with perception, are also defined by synchronization of the activity on a millisecond time scale. This would serve to encode contextual information by defining relations between the features of visual objects. If temporal properties of neural activity are important to establish the neural mechanisms of perception, then the study of appropriate dynamical stimuli should be instrumental to determine how these systems operate. The use of natural stimuli and natural behaviors such as free viewing, which features fast changes of internal brain states as seen by motor markers, is proposed as a new experimental paradigm to study visual perception.

Back in ancient Greece, Hippocrates and later Plato, were among the first thinkers in attributing the brain its role in perception and other mental processes, but it wasn't until the beginning of the 19th century that the brain was recognized as a collection of distinct areas that participated in a cooperative way to establish the behavior observed in human and animals.In all this time and even today, brain and mind are often considered as dissociated entities, the brain being the object of medical and biological studies, while the mind the topic of philosophers and psychotherapists or cognitive scientists.Today, the scientific community has again returned to the notion of mind and brain as part of a single system: in many fields of biology and the discovery of new experimental techniques make it possible for the first time to face the task of a global understanding of the brain as a functional unit.In July 2005, Science magazine published a special number dedicated to the most important scientific questions for the next 25 years.Among the questions of greater importance were included several ones referred to the cognitive abilities of the human being such as memory and consciousness.
The scientific interest in the brain and mind is nevertheless much more than an academic exercise.In a systematic manner, the cost associated to the pathologies and ailments of the brain are becoming greater each year.A study of the European Brain Council (Andlin-Sobocki et al., 2005) carried out in 28 countries of Europe (that includes 466 million people), has shown that approximately 27% of the population suffer from some illness that involves the nervous system and particularly, the brain.The associated cost to these disorders reaches approximately 386 billion euros, equivalent to 829 euros by European inhabitant, which compares to the cost of more than 25 tunnels as the one built between England and France.In the United States, brain disorders affect approximately 100 million people each year, including mental disorders such as depression, schizophrenia or neurological disorders like cephalea (headache), Alzheimer and Parkinson.The cost of these illnesses is enormous, being estimated to top 50 thousand million dollars each year.The National Institutes of Health (NIH) invest approximately a third of their annual budget in Neuroscience research, which corresponds approximately to 5 thousand million dollars.In Chile, little is known of these statistics, nevertheless fragmented studies show that the preponderance of nervous pathologies and their associated consequences has also acquired an enormous importance in public health.Statistics delivered by the Department of Health show that in the last decade, medical consultations in psychiatry and neurosurgery have duplicated while consultations in other pathologies have diminished or have been experience little or no growth.
A NEW PARADIGM?
If the mind is the operations of the brain, how can we understand the different mental processes and explain the extensive range of behaviors in humans and other animals?A human brain has more than 10 12 neurons and 10 times more glial cells.Evidently, the complexity of this organ does make it very difficult to study if it is examined as an entire structure.A strategy that has enabled almost all our knowledge on the operations of the brain has come from an approach where we have reduced the scope of the behaviors of interest as well as the brain structures involved in such behaviors.In this manner, a sensory or motor system can be studied under the assumption that in those systems, the fundamental basis of the operation of simple behaviors can be observed and by extension, extrapolated to the rest of the brain.The sensory systems of the brain has been the most studied because they allow the experimenter to carry out controlled disruptions by external stimulation and to directly observe their consequences on the neuronal activity.In this essay, I will refer frequently to studies carried out in the visual system as a paradigmatic example of our approach to understand cerebral function.The visual system is the best-known sensory system largely because we as human beings are fundamentally visual animals.Almost a third of the cerebral cortex in primates has been associated to visual perception.Primarily I will refer to the way in which the field of neurophysiology has tried to understand visual perception and what we have learned in recent years that has changed our main scientific paradigm.Nonetheless, the ideas considered here are as applicable for vision as for other sensory or motor systems.
Let us examine what happens when we engage in a typical visual task such as viewing a visual scene.We move our eyes four or five times a second exploring a great part of our visual field.During each visual fixation, that lasts from 100 to 350 msec, the image is projected on our retinas and produces changes in the receptor neurons as well as in all the rest of the network of neurons that participate in visual perception.When our eyes move, the images also move at great velocity over the cells in the retina, in marked contrast with our visual perception, which appears to be constituted by a uniform, clear, and stable world.If we examine the ability of the retina to respond to stimuli of different physical nature, we may find that this knowledge may be at odds with our perception, since the retina is not uniform neither in receptor density nor in the response characteristics of its cells.
When we review a neuroscience textbook on visual function, we will find that typically the visual world is depicted as an image projected on the retina that in turn is transmitted by the visual pathway and subsequently projected on the primary visual cortex where the image is reconstructed.This image is then interpreted by some other area of the brain.In this view, the neuronal activity of the retina is transmitted progressively from the eye to the lateral geniculate nucleus (visual thalamus) and then conveyed toward the primary visual cortex.From there, the visual activity is transmitted successively toward other higher cortices where eventually it produces the final representation of the objects that are being observed.If we examine a current diagram of the visual system in the same textbook, we likely find that the neuronal activity is shown like a flow chart with one-way arrows originating in the eye pointing towards the "inside" of the brain.In order to make explanations about visual perception this scheme requires to determine how each one of the neurons that participate in this chained neuronal processing, respond to different visual stimuli.In these conditions, the visual stimulus has simple properties, is presented for a relatively long period and is designed to produce the maximum possible activity in order to obtain an adequate amount of data.Subsequently, the activity of many neurons are grouped, to infer different aspects of the functional organization, such as the receptive field properties, perceptual correlates or the mechanisms for codification of the stimulus.This approach has been in many aspects, successful.A significant knowledge has been obtained on how individual neurons modify their activity when they are faced with visual stimuli and often these properties correlate with the performance of the animal in perceptual tasks.
Thus, the central focus has been to determine the physical properties of the stimulus that cause changes in the discharge rate of the neurons that are being recorded, modifying some physical parameter of the stimulus such as its brightness, contrast, orientation, form, color etc.This paradigm has been quite successful because we have been able to determine in which way the cells of the retina, the thalamus and the cortex respond to a collection of simple stimuli with known properties.Yet, our predictions of how neurons respond to visual stimuli are better in the retina that in the thalamus, and similarly better in the latter nucleus than in the cerebral cortex.This occurs because to the extent that we move away from the initial sensory surface (the retina) the responses of neurons are increasingly variable.Besides the inconstancy of the responses, the stimulus needed to achieve a significant change in the discharge rate of the neurons becomes increasingly complex.In the retina, circles of light and darkness are good stimuli, in the primary visual cortex bars and gratings are better stimuli, but as higher cortical areas are examined, the visual stimulus needs to incorporate another feature such as movement or complex geometrical forms, even to become complicated objects such as faces.This would occur because the neurons of upper cerebral areas begin to combine the simplest responses of earlier neurons increasing the complexity necessary to change their activity, thus intensifying their specificity to different classes of stimuli.This specificity is translated in the proposal that perception is constituted (or codified) by the activity of one or few neurons.Overall, we have worked with the assumption that perception is a function primarily based on the physical properties of the stimulus, and therefore if we adequately characterize the way neurons respond to these different properties, we will be able to explain how every perception occurs.
Nevertheless, in the last decade new studies have uncovered evidence that challenge this paradigm.First, the notion of the receptive field as the functional unit with which the brain represents the world seems to be much less appropriate or representative of brain function than thought.The receptive field is defined as the area of the sensory surface that on stimulation by a physical stimulus, evokes changes in that neuron.A series of recent works (Bringuier et al., 1999;Jones et al., 2001;Trotter and Celebrini, 1999;Vinje and Gallant, 2000;Worgotter et al., 1998) have shown that each neuron is susceptible to change its activity and to be influenced by stimuli in very distant places compared to its traditional receptive field (now called classical).Moreover, this influence is not symmetrical or lineal.In recent years, other studies have demonstrated that even that our best predictions on the responses of visual neurons to known stimuli such as bars and gratings, fail systematically in situations of natural stimulation (Yen et al., 2004).
A second line of evidence comes from studies that show that the brain is a more complex network than initially assumed.In fact, textbook diagrams with one-way connectivity are inexact and confusing because practically all the nuclei of the brain have reciprocal connections with various other regions.This of course, also occurs in the visual system.The visual thalamus receives from the primary cortex ten times more connections than the ones that it sends toward the cortex, all the while the primary visual cortex is connected with many other adjacent cortices.It has been estimated that a given cortical column receives only 1% of its afferents from subcortical nuclei, while 99% of the remaining afferents are coming from other cortical regions (Douglas & Martin, 1998).Thus, the different areas of the cortex more that forming a sequence of processing nuclei, form a recurrent network where the neurons are mutually modified in an extended manner.What consequence has this configuration for the neurons of the sensory systems when these are exposed to different stimuli?The most important consequence is that the activity of any neuron in the network will be modulated not only by the present stimulus, but also by the formidable modulation that stems from other neurons of the network.A recent experiment carried out by Murray et al. (2002) showed this phenomenon in the activity of the visual cortex of humans (V1).Through functional magnetic resonance techniques, they exposed subjects to stimuli of different complexity but with similar local features and found that the activity in V1 and the lateral occipital complex (LOC) is modulated depending on the complexity of this stimulus.Visual perception requires the grouping of individual elements of coherent patterns that reduce the descriptive complexity of a visual scene.When the stimulus was very simple such as formed by separate lines, there was a high activity in V1 and low activity in the lateral occipital complex (an important area in perception of objects).The opposite occurred when the stimulus was a coherent geometrical form by joining the same lines as a closed figure.This suggests that the activity in the early visual areas was reduced as a direct result of the cognitive processes carried out in higher visual areas.
Consequently, we can state that our models and explanations on the neurophysiology of sensory systems are undergoing a paradigmatic change.The traditional paradigm is based in a representational model, where perception is a feed-forward, unidirectional process in which the visual cortex is but just one stage of a hierarchical system.Therefore, perception would occur because of the activation at the highest level of this hierarchy.In recent years, a new paradigm is taking shape, which resembles a constructivist proposition.Here perception is seen, as a dynamic process where visual images are constructed, not codified.In this perspective, the visual system recurrently generates a visual percept with incomplete sensory activity, but including ongoing activity of the entire neural network.This percept would correspond to the most probable interpretation based on prior experiences, of the current visual stimulus.This implies that our visual world is not a sensory "capture" where our experiences reflect the unique properties of the physical world to which we are exposed.Instead, as the title of this essay suggests, ours visual experience is something we build based on the current state and configuration of our neuronal network.This construction depends as much of the physical world to which we are exposed, as of the ongoing activity of the network of neurons that constitute our brain, and of the experience that modified the dynamics of the network with which we build our experience.Every perception is a construction, reflected as a neuronal correlate of populations of neurons specifically and recurrently active to produce the same experience or image.Approximations to this idea already had been presented in different forms, like the concept of enaction, introduced by Varela et al. (1991).Enaction is located as an intermediate proposal between objective and subjective radicalism.The idea is "to emphasize the growing conviction that the cognition is not the representation of a world "pregiven" by a mind "pregiven", but rather the enaction of a world and a mind based on a history of several actions that an agent executes in its environment" NEURONAL CORRELATES OF PERCEPTION If our visual experience is a construction of the brain, what constitutes the appropriate neural correlate of this experience?Clearly, the visual system is faced with a formidable task: the ability to recognize and to categorize complex patterns in the visual images.Within an epoch as short as 150 milliseconds, the visual system is capable of segmenting combinations of features or visual characteristics in a coherent collection of independent objects.Although the possible number of interrelations of objects inside a visual scene is of an infinite variety, the visual system can effectively operate on the combinatorial explosion of the images that it faces at any time.Nevertheless, there is limited knowledge of the actual visual perception mechanisms that account for the discrimination or construction of objects in the visual world.It has been proposed that an important stage in this process requires the establishment of relations among different visual characteristics, like orientation, form, color, etc.The universe of visual characters that constitute an object should be grouped according to some criterion and segregated from others to prevent inappropriate conjunctions.Given that the visual cortex of mammals is organized in a collection of interconnected areas (Felleman and van Essen, 1991), each one with activity characteristics of its own, it is probable that this process involves the integration of the activity of many neuronal populationsselective to some visual character-in many places of the cortex.Different models or neuronal correlates have been proposed to realize this integrative process.One model suggests that the activity of selective populations to some visual character converges hierarchically to other cells that codify visual characters of a greater order (Barlow, 1972), following the classical hierarchical view of anatomical connections discussed above.This hypothesis is known as the theory of the cardinal cell, or as baptized by Jerry Lettvin, the "grandmother" neuron.This hypothesis is schematized in figure 1A.Only one cell changes significantly its firing rate when a stimulus is present (stim1) and an alternate cell fires when the stimulus changes (stim2).Some studies have shown that the activity of certain cells seems to codify complex properties of objects as occurs with human faces (Gross et al., 1972;Fujita et al., 1992;Tanaka, 1993).Recently it has been reported that cells in the hippocampus can be activated when a famous person such as an actor appears in a photo (Quiroga et al., 2005).Nevertheless, it has been theoretically shown that a mechanism that encodes purely on hierarchical convergence suffers from a very serious caveat such as a very limited representation capacity (Malsburg, 1985;Malsburg and Singer, 1988).Such neuronal networks can thus represent, as many objects as cells they possess, assuming an appropriate  (2, 3, 5 and 1, 4, 6).selectivity.Given the infinite permutability in the always-changing visual world, the number of neurons in the visual system seems inadequate.Moreover, each represented object requires a pattern of specific connectivity, which makes the number of connections in these networks practically impossible.Given these and other limitations such as the fragility of an ultra specialized representation, other models for visual perception have been proposed that take into account the parallel architecture of the visual system.The distributed representation model refers to the notion of a distributed code or representation, where a visual object is represented by the activity of a population of neurons dispersed along many cortical areas.This hypothesis is schematized in figure 1B where many cells show changes when stimulus 1 or 2 is present, but the identity of the stimulus is coded in the combined magnitude of the neuronal firing.Population coding significantly increases the capacity of the cortex to uniquely represent a pattern (Ballard et al., 1983).This code has been demonstrated to operate in the motor cortex (Georgopolous, 1993), and hippocampus (Wilson and McNaughton, 1993), but it is not clear whether this mechanism will work in sensory cortices (Young and Yamane, 1992).This approximation has generated a lot of interest because predictions in the motor system, which are based on the activity of populations of neurons, have enabled the development of brain machine interfaces (Nicolelis, 2004).This model appears to be consistent with the anatomical organization of the visual cortex of mammals.The majority of visual patterns do evoke activity in numerous neurons distributed in a number of cortical areas, in both cerebral hemispheres.However, the notion of a distributed code is not exempt of problems.It is not clear, for example, how the relations among characters of an object in a visual image are established without ambiguity, or how multiple representations of different objects may coexist in several cortical areas without being confused.These aspects of the problem have been called the "binding" or superimposition problem.Binding relates to how the activity in a cortical region is grouped with the activity in another area, to configure a visual object.An additional caveat has to do with the need to avoid interference when multiple visual objects should be represented simultaneously.Although a number of different mechanisms have been proposed to solve these problems, two have received extensive interest: attentional selection (Treisman, 1986;Desimone et al., 1990), and temporal correlation (Milner, 1974;Malsburg, 1981Malsburg, , 1985;;Malsburg and Schneider, 1986;Singer, 1990).These two mechanisms are not mutually exclusive, but they differ in important aspects.The attentional mechanism achieves grouping and eliminates superimposition by modulating the relative activity of cell populations activated by the objects attended to (Moran y Desimone, 1985;Spitzer y Richmond, 1991;Desimone et al., 1990;Tanaka, 1993).The temporal correlation model achieves grouping and eliminates superimposition by synchronizing the activity of those cells that are evoked for the same visual object and desynchronizing the neuronal activity evoked by different objects.Temporal correlation has the advantage that it enables the representation of multiple objects without interference (Malsburg and Schneider, 1986).Figure 1C schematizes this hypothesis.Two or more stimuli can be simultaneously represented combining population code with synchronization.Consistent with the temporal correlation hypothesis, it has been shown that synchronized discharge is a general property of the activity in the visual cortex in cats and monkeys.Research performed with multiple electrodes has shown that groups of nearby cells in the primary visual cortex often discharge in synchrony during visual stimulation (Gray & Singer, 1989;Maldonado et al., 2000;Maldonado & Babul, 2007).It has also been shown that this synchrony in the responses extends beyond the limits of the columnar organization, occurring between areas of the striate and extrastriate cortex (Gray, 1999).Also consistent with theoretical predictions, it has been found that the occurrence and magnitude of the synchronous activity depends on the global and local properties of the stimulus (Kreiter et al., 1992, Engel et al., 2001).These discoveries are particularly important because they indicate that the synchronized discharge is a dynamic process influenced by the properties of the visual stimulus.A given group of cells can synchronize its activity with different populations of cells at different times, depending on the stimulation conditions.These dynamic correlations also have been shown to occur in other cortical systems (Aertsen et al., 1991;Ahissar et al., 1992;Vaadia andAertsen, 1992, Rhiele et al., 1997) and at different spatial scales (Varela et al., 2001).In this manner, if the synchrony reflects the establishment of relations among visual characters in an image, then the dynamic changes in patterns of correlation can provide a flexible combinatorial coding in the integration of distributed activity.If we extend this idea to the mind, we could say that mental states are configured through the synchronous activation of distributed populations where activation of a neuronal ensemble produces a corresponding cognitive object.These states are established largely by the intrinsic dynamics of the network and with the perturbations that the brain undergoes every time it confronts physical stimuli from the environment.Now, the challenge is to determine ways to characterize these states and to study their dynamical properties.
A MATTER OF TIMING If we are to examine the temporal properties of the neuronal discharges, then it is also reasonable to think that we have to pay attention to the temporal structure of the visual stimuli we use to study cortical function.One characteristic of the studies on the responses of neurons to visual perception is that we have used simple and repetitive stimuli.Most of the time, the stimuli evoke a large transient response followed by sustained response that lasts as long as the stimulus is present.These transient responses are typically ignored or considered stimulus artifacts.Yet is wellknown that most natural stimuli often produce very short changes in neural activity, particularly in dynamical situations such as free viewing of natural images or movies.This is relevant because natural vision is exactly the type of situation we ultimately want to explain.In free viewing conditions the average duration of visual fixations (which occur 4 or 5 times a second) is approximately between 150-250 ms in non-human and human primates (Flores et al., 2005;Maldonado et al., 2000;Schiller et al., 2004).The neuronal activity occurring in the first 20 ms of a maintained discharge in the inferotemporal cortex is informationally prominent for the discrimination of a visual stimulus (Rolls et al., 2006).In visual masking studies, designed to diminish visual detectability, this fine transience of the processes related to vision can also be observed.When a mask is projected immediately before or after a target stimulus, it significantly alters the detection and the activity of neuronal discharge in V1.This modulation of the activity is produced mainly by the suppression of the transient discharge related to the apparition (masked before stimulus) and disappearance of the stimulus (subsequent masking) (Macknik, & Livingstone, 1998).In local field potential and EEG studies, it is also observed that the synchronous activity among electrodes in different bands of frequencies relates to the detectability of a visual stimulus, enlarging and diminishing within the first 400 msec of the presentation of the stimulus (Tallon-Baudry et al., 1996;Rodriguez et al., 1999, Fries et al., 2001).Finally, studies with transcraneal magnetic stimulation (TMS) also give support to the involvement of transient processes in visual perception.In a visual identification task in which pulses of TMS were delivered to V1, it was shown that the performance in the recognition task is deteriorated only when the TMS stimulus is applied between 60 to 140 msec after the onset of the visual stimulus, with a null detectability if TMS is given between 80 to 100 msec (Amassian et al., 1989).

AN EXPERIMENTAL PROPOSITION
Because the dynamical characteristics of visual responses and the intrinsic dynamical changes of visual stimuli occurring in natural vision, I propose that a proficient approach to study visual function may be carried out while the animals or the human subjects perform tasks under more ecological conditions.This suggests studying brain function under conditions in which the brain transits from one state to another in a natural form while it carries out tasks that are common in its daily operation.In this way, changing stimuli comes at times where the brain expects or provokes them, rather than the investigator specifying the times at which the events occur.One experimental example of this proposition occurs precisely during the free viewing of natural stimuli.When we see an image or we visually explore a scene, we move our eyes three or four times per second.This sequence of ocular movements reveals that the visual system has finished processing a certain area of the image in order to move its attention to other places of the image.Clearly, this natural task reveals through motor mechanisms the instances in which the visual system changes of state.The idea then is to use the moments in which the subject or animal initiates a visual fixation in a place of the image as a marker of natural events.Thus, EEG signals in a human, or the activity of neuronal populations registered with electrodes in an animal can be analyzed with relation to the times of ocular movements.This approximation intends to enable the brain to carry out a task under dynamics not interrupted by the experimenter, but that are at the same time, capable to deliver appropriate markers to analyze and to correlate neuronal activity with behavior.In the visual system, this approximation can be carried out in several ways.For example, the neuronal activity of subjects can be compared between free visual exploration and the activity elicited during visual search or an attentional task.Obviously, the proposition of incorporating natural markers is not restricted solely to the visual system.In principle, any behavior that enables a characterization of a motor event associated to a neuronal activity is suitable to be used as marker.This may occur for example in the olfactory system, where indeed perception appears to be closely related to the inhalation and sniffing motor behavior (Verhagen et el., 2007).Many other perceptual phenomena are indeed associated to motor behaviors.This seems to be more the usual case that the exception since apparently every perception may include a motor act (Noe, 1990).Of course, this new experimental approximation is not exempt of controversy (Rust & Movhson, 2005;Felsen & Dan, 2005).One of the evident obstacles has to do with the analysis that is carried out on the neuronal signal.If the sensory-motor events occurring during a natural behavior turn up frequently as in the case of the ocular movements, the speed with which the neuronal states change requires that the analysis be done with equal or higher speed.For recordings of cerebral activity, including single unit activity carried out in animals this may be less of a problem, but in the case of EEG recordings and local field potential, no tools yet exist that enable a fast characterization of the signal.For example, frequency examination always requires an analysis window that may be longer that the time during which the brain is changing state.Hopefully in the coming years, new tools will be developed to deal with these signals and uncover the fast mechanisms that operate during normal brain function.

Figure 1 :
Figure 1: Schematic representation of three models of neural correlates of perception.(A) Hierarchical convergence.The activity of selective populations to some visual character converges hierarchically to other cells that codify visual characters of a greater order.Only one cell changes significantly its firing rate when a stimulus is present (stim1) and an alternate cell fires when the stimulus changes (stim2).(B) Population coding.Objects or stimuli are represented by the activity of a population of neurons distributed among cells and diverse cortical areas.Many cells show changes when stimulus 1 or 2 is present, but the identity of the stimulus is coded in the combined magnitude of the neuronal firing.(C) Temporal correlation hypothesis.This model achieves grouping and eliminates superimposition by synchronizing the activity of those cells that are evoked for the same visual object and desynchronizing the neuronal activity evoked by different objects.Two or more stimuli can be simultaneously represented combining population code with synchronization.Bars and circles represent the times at which cells 2, 4, and 6 fire simultaneously for stimulus 2 while cells 1, 3 and 5 do so for stim1.During another stimuli presentation (3 and 4), the same cells establish different synchronous ensembles(2, 3, 5 and 1, 4, 6).

Figure. 2 :
Figure.2: Example of data analysis performed during free viewing and eye movements monitoring.(A) Example image presented to a human subject.They scanned the pictures for up to 5 sec.In this image presentation of 5-sec duration, the white line represents the eye trajectory.(B) Vertical and horizontal eye traces recorded during the same image presentation.(C) The eye movement velocity that is used to discriminate saccades and fixation events.(D) Time dependent changes in power spectrum of the oscillatory EEG signals locked to visual fixations.Data were aligned to the onset of the fixations (time 0 on abscissa, dotted vertical line).The thin white line indicates the typical time course of a saccade-fixation sequence, the dotted part referring to the onset jitter caused by the variable duration of saccades.Here a strong, beta-band oscillation is shown locked to the onset of fixations.A global FFT analysis of the whole signal fails to see this event.