The visual system is a series of highly complex interlinking processes. There are many arrangements to these processes, including lateralized, hierarchical, and parallel processing. The system is a modular one with particular processes being confined to specific brain areas.
Lateral inhibition
Perhaps the first, basic principle underlying the organisation of visual processing is lateral inhibition. This process occurs in on-centre, off-surround receptive fields, or indeed, off-centre, on-surround receptive fields. Several basic types of visual information are coded in this way. Light intensity is one of the most important. The principle of lateral inhibition allows changes in light intensity to be detected by higher visual processes and this leads ultimately to many things such as edge detection, and object recognition. Lateral inhibition is also used in the detection of colour. There are fields with centres excited by red light and surrounds excited by green light, those with centres excited by blue light and surrounds excited by yellow light, and of course, the reverse of each. Lateral inhibition is an efficient way of coding visual information as the retinal ganglion cells (which make up receptive fields) only fire according to changes in visual information. This means that only relevant information is transmitted and energy is not wasted on processing redundant information.
Contralateral processing
Contralateral processing underlies the whole visual processing system. This principle requires that information in the left half of the visual field is processed by the right hemisphere of the brain, and information in the right half of the visual field is processed in the left hemisphere. It is important to highlight that it is not the case that information from each eye is processed in the opposite hemisphere – it is the visual field that is split instead. This contralateral processing occurs in the lateral geniculate nucleus (LGN). Here, axons from ganglion cells in the right and left halves of the retinas are split and fed to opposite hemispheres of the brain.
Hierarchical processes in striate cortex
Beyond the above, the principles governing much of the remaining visual processes are, in some form, hierarchical processes. These begin at the striate cortex, located at the back of the occipital lobe. Here the primary visual cortex (area V1) begins to extract information from the raw signals supplied via the LGN. The striate cortex itself contains a hierarchy in that it is organised on levels. Information enters the striate cortex at about its middle, at level 4C. Information is then relayed to the other levels for processing. The visual information is not yet fully and totally integrated together – that comes later. The striate cortex deals with individual inputs from ganglion cells. The visual field is “mapped”, in that distinct areas of striate cortex deal with distinct areas of the visual field. The map is distorted – 25% of the striate cortex is dedicated to processing information from the fovea, an area which makes up less than 0.1% of the visual field. This, among other factors, shows that the finest details are dealt with by this tiny area of visual field.
The striate cortex contains neural circuits which deal with the major, basic aspects of an image. These include colour (the method for detecting this, lateral inhibition, is mentioned above), and texture (there are neural circuits which are excited by specific patterns of light). The striate cortex also has neural circuits which analyse spatial frequencies. Retinal disparity is detected too, by neural circuits that increase in response if the same stimulus is in two different locations. The receptive fields in the striate are greater in size than with the receptive fields of ganglion cells, and consequently, detect bigger features. For instance, Hubel and Wiesel found that most of the cells in the striate cortex are orientation sensitive.
Parallel processing in the striate cortex
Although there are hierarchical functions going on in the striate cortex, it can be said to display traits typical of parallel processing. The striate cortex is constructed of about 2500 modules (each about 0.5mm x 0.7mm, and containing about 150,000 neurons), each receiving information from one part of the visual field. This unintegrated approach has been likened to a set of mosaic tiles. Within the modules, visual information is run through various hierarchical process, but between modules, there are parallel processes, since each stream of information is processed separately. In fact, parallel processes are occurring inside the modules as well. There are two segments in each module, which surround structures known as “blobs”. Blobs are sensitive only to colour information and ignore all other kinds of information. Outside the blobs, in the two segments, colour information is ignored and the inputs from each eye are binocularly combined and other factors are processed, such as orientation and spatial frequency. Blasdel noticed the different modules in action by replacing an appropriate section of a primate’s skull with a glass panel and injecting a voltage sensitive dye into the striate cortex. He identified the modules and their processes using analysis on a computer.
Hierarchy in the extrastriate cortex
There is more hierarchical processing beyond the striate cortex when information is passed into the extrastriate cortex, or visual association cortex. As the name suggests, these areas are involved with further visual processing and ultimately serve to make the visual information meaningful and useful by associating it with other visual information and other areas of the brain, such as memory and other sensory modalities. There are several regions of the extrastriate cortex, each of which are involved in a great many processes. The areas are named V2, V3, V3A, V4, and V5 (or more commonly, MT). Experiments using lesions in primate brains have shown what some areas are used for. For instance, Schein and Desimone (1990) found that area V4 of the primate extrastriate cortex contained neurons sensitive to specific colours. Evidence shows that processes in the extrastriate cortex are consistent with Land’s (1974) theory of colour constancy. The relevant areas compensate for the source of light by comparing the colour of each location in the visual field with the average colour of the entire scene. Area V5 (MT) has been found to be involved in the detection of movement. It takes its input from areas V2, V3, and V4, plus another input from the superior colliculus – a “rawer” stream, more direct from the eyes. This could be seen as a type of parallel process in a sense, since Rodman, Gross, and Albright (1989, 1990) found that the destruction of either of these inputs did not eliminate the perception of movement.
The dorsal and ventral visual streams: large-scale parellel visual processing
There exists a major split in the visual information as it leaves the extrastriate cortex, and this is another example of parallel processing. There are two streams of visual analysis – the ventral stream (leading to the inferior temporal cortex), and the dorsal stream (leading to the posterior parietal cortex). These two streams run different visual processes and are ultimately concerned with extracting different information. Generally, the ventral stream involves process to determine what an object is, while the dorsal stream decides where objects are located in space. It is appropriate to note here that leading into these two streams are two different systems of cells – the magnocellular system, and the parvocellular system. These have different specialisations. The magnocellular system, despite having poor spatial acuity, is especially good at detecting movement, and has a high temporal acuity. It is, however, insensitive to colour. The parvocellular system is more or less the opposite. It has great spatial acuity, poor temporal acuity, and analyses colour. These systems are essentially a kind of parallel processing. Mostly magnocellular information is fed into the dorsal stream, whereas the proportion is almost even, leading into the ventral stream, but there are strong suggestions that the emphasis is on parvocellular input.
These streams are indeed essentially different areas of visual processing entirely. They are really quite separate. For instance, people with damage to areas of one steam but not the other, are not “blind”, but merely lack one aspect of vision. For instance, catastrophic destruction in the ventral stream will result in a person being unaware of anything they see (since they cannot associate it with other parts of their brain), so they are apparently blind. They also suffer a deficit in object recognition. However, the connections from the fully functioning dorsal stream which lead to motor controls survive. These apparently blind patients can successfully navigate their way through the world, despite not “seeing” where they are going. This, and other similar phenomena, are known as blindsight.
Damage to the dorsal, but not ventral, stream will usually result in a person being able to recognise what objects they can see, but have no idea where they are. If they attempt to reach for them, their movements are totally misdirected. They will also be unable to detect movement properly. Such damage can be terribly disturbing and disoientationg for a sufferer.
More hierarchy: within the dorsal and ventral streams
Although on the scale of the brain as a whole, the ventral and dorsal streams are parallel processing, within the streams (just as within the modules of the striate cortex), there are hierarchical processes. For instance, the form of objects is compounded in the ventral stream, and to do this, it takes information which has been first run through areas V3, V3A and V4 of the extrastriate cortex, then after this, to the temporal neocortex. Recognition actually takes place in the inferior temporal cortex where the information from the extrastriate cortex is combined. There are two areas involved in this, areas TEO and TE, which can be treated like a two step hierarchical process. The TEO has a primary input from V4 and its output primarily goes to the TE (notice, this output is not exclusively to the TE, which demonstrates that the interaction of the areas of visual processing are indeed a very complex hierarchical process, with streams of information “looping back” to other areas). Lesions in monkeys in their TEO has shown that they find it impossible to differentiate between two simple 2D patterns (Mishkin et al. 1983).
Area TE is the area where the most global features are detected, and it contains huge receptive fields up to the size of the entire contralateral visual field. This area responds best to 3D objects, even if the object changes location. Tanaka (1992) found that individual cells would respond to a particular set of general features, but this did not lead him to believe that there was one cell which would respond to each specific object. Rather, that groups of these cells responding would correspond to object recognition - this is known as "population coding" of visual stimuli.
One of the most crucial parts of the ventral stream is its termination in the perirhinal cortex. This area is where object recognition occurs. There are many associations with memory and other sensory modalities here, thus achieving associations between visual stimuli and memories of object identity. Beyond this, visual processing becomes even less purely within the visual domain.
Vision and memory
The two visual processing streams eventually recombine. A function of this is to produce visual memories, such as those in episodic (or autobiographical) memory. The entorhinal cotex and hippocampus are crucial to coordinating memory functions and recieve imputs from both the ventral and dorsal streams, thus combining the "what" and "where" aspects of visual processing to store a complete visual scene in the memory.
Summary
There are so many aspects of the hierarchical processing of the brain, that they have not yet all been discovered. The visual processing system is a very complex one indeed, reflecting the complex processes that are necessary to extract all the visual information we use. On a large scale, the visual system is organised contralaterally, and on a small scale, it is often organised using the principle of lateral inhibition. Parallel processes seem to run throughout all the aspects of visual processing. This is necessary to keep the information separate and manageable for the hierarchical processes to work.
Schematic diagram of the visual system
Retina
(photoreceptors:
rods & cones)
|
Ganglion cells
P cells M cells
/ \
Parvocellular Magnocellular
stream stream
| |
(Lateral Geniculate Nucleus)
____|______________|____
| | | |
| | V1 | |
|____|______________|____|
| |
____V______________V____
| | | |
| | V2 | |
|____|______________|____|
| |
____V______________V____
| | | |
| | V3 -----|-----|
|____|___________________| |
| |
_____V_____ _____V_____
| | | |
| V4 | | MT |
V |___________| | (V5) | D
E | |___________| O
N _____V_____ | R
T | | | S
R | TEO | _____V_____ A
A |___________| | | L
L | | | MST |
| TE | |___________| S
S |___________| | T
T | | R
R _____V______ _____V_____ E
E | | | | A
A | Perirhinal | | Parietal | M
M | cortex | | areas |
|____________| |___________|
| |
_____V______ _________V______
| | | |
| Entorhinal |<-------| Parahippocampal |
| cortex | | Formation |
|____________| |_________________|
| | | |
__V__V__V__V__
| |
| Hippocampus |
|______________|