thumb|275x275px|Visual Storytelling by Suhani Gowan.

Dual coding theory is a theory of cognition that suggests that the mind processes information along two different channels; verbal and nonverbal. It was first hypothesized by Allan Paivio of the University of Western Ontario in the late 1960s. In developing this theory, Paivio used the idea that the formation of mental imagery aids learning through the picture superiority effect.

According to Paivio, there are two ways a person could expand on learned material: verbal associations and imagery. Dual coding theory postulates that both sensory imagery and verbal information is used to represent information. Imagery and verbal information are processed differently and along distinct channels in the human mind, creating separate representations for information processed in each channel. The mental codes corresponding to these representations are used to organize incoming information that can be acted upon, stored, and retrieved for subsequent use. Both imagery and verbal codes can be used when recalling information. Another limitation of the dual coding theory is that it is only valid for tests on which people are asked to focus on identifying how concepts are related. This theory has been applied to the use of multimedia presentations. Because multimedia presentations require both spatial and verbal working memory, individuals dual code information presented and are more likely to recall the information when tested at a later date. Moreover, studies that have been conducted on abstract and concrete words have also found that the participants remembered concrete words better than the abstract words.

Paivio found that participants when shown a rapid sequence of pictures as well as a rapid sequence of words and later asked to recall the words and pictures, in any order, were better at recalling images. Participants, however, more readily recalled the sequential order of the words, rather than the sequence of pictures. These results supported Paivio's hypothesis that verbal information is processed differently from visual information and that verbal information was superior to visual information when sequential order was also required for the memory task. Lee Brooks conducted an experiment that provided additional support for two systems for memory. He had participants perform either a visual task, where they had to view a picture and answer questions about the picture, or a verbal task, where they listened to a sentence and were then asked to answer questions pertaining to the sentence. To respond to the questions, participants were asked to either respond verbally, visually, or manually. Through this experiment, Brooks found that interference occurred when a visual perception was mixed with manipulation of the visual task, and verbal responses interfere with a task involving a verbal statement to be manually manipulated. This supported the idea of two codes used to mentally represent information.). It also has had implications for cognitive robotics.

Cognitive neuroscience support

Two different methods have been used to identify the regions involved in visual perception and visual imagery. First, functional magnetic resonance imaging (fMRI) is used to measure cerebral blood flow, which allows researchers to identify the amount of glucose and oxygen being consumed by a specific part of the brain, with an increase in blood flow providing a measure of brain activity. Second, an event-related potential (ERP) can be used to show the amount of electrical brain activity that is occurring due to a particular stimulus. Researchers have used both methods to determine which areas of the brain are active with different stimuli, and results have supported the dual-coding theory. Other research has been done with positron emission tomography (PET) scans and fMRI to show that participants had improved memory for spoken words and sentences when paired with an image, imagined or real. Those participants also showed an increase in brain activation that processes abstract words not easily paired with an image.

Alternative theory

Dual coding theory is not accepted by everyone. John Anderson and Gordon Bower proposed an alternative method – the propositional theory – of how knowledge is mentally represented. The propositional theory claims that mental representations are stored as propositions rather than as images. Here, proposition is defined as the meaning that underlies the relationship between concepts. The propositional theory is able to explain the basic concept of an idea without needing images or verbal information. It is able to take the complex and break it down more into the differing components of the idea or concept. This theory states that images occur as a result of other cognitive processes because knowledge is not represented in the form of images, words, or symbols. This theory is also related to A Natural Deduction Systems model. This type of model allows for both feed forward and its reverse.

The common coding theory has also been proposed as an alternative to dual coding theory. The common coding theory looks at how things we see and hear are connected to our motor actions. It claims that there is a common code that is shared between perceiving something and the respective motor action.

See also

  • Multimedia learning

References

  • Multiple Code Theory