Influence of Pictures on Word Recognition

Georgia Southern University

Georgia Southern Commons

Electronic Theses and Dissertations Jack N. Averitt College of Graduate Studies

Spring 2009

In6uence of Pictures on Word Recognition

Audrey A. Hazamy

Follow this and additional works at: https://digitalcommons.georgiasouthern.edu/etd

Recommended Citation

Hazamy, Audrey A., "In6uence of Pictures on Word Recognition" (2009).

Electronic Theses

and Dissertations

. 430.

https://digitalcommons.georgiasouthern.edu/etd/430

This thesis (open access) is brought to you for free and open access by the Jack N. Averitt College

of Graduate Studies at Georgia Southern Commons. It has been accepted for inclusion in Electronic

Theses and Dissertations by an authorized administrator of Georgia Southern Commons. For more

information, please contact [email protected].

The Influence of Pictures on Word Recognition

AUDREY A. HAZAMY

(Under the Direction of Lawrence Locker)

ABSTRACT

The picture superiority effect is a well-documented phenomenon that is defined as the superior

memory of pictorial stimuli compared to word stimuli. The predominant explanation for this

effect is that pictures are encoded more effectively than words, and are therefore able to be better

remembered. Research has demonstrated the benefit that pictures provide for remembering

words. However, little work has been to done to examine any negative and/or intrusion effects of

pictures on memory for words. The current study further explored the picture superiority effect

and facilitation effects of pictures on words as well as interference effects of pictures on word

memory. Results indicated that matching and semantically related pictures facilitated word

recognition memory. Incongruent/unmatching pictures however were of no detriment.

Implications of picture processing and pictorial influence on word processing are discussed.

INDEX WORDS: Picture superiority, Picture and word processing, Picture processing, Picture

intrusion, Picture interference, Picture facilitation, Semantic relatedness, Word processing

The Influence of Pictures on Word Recognition

AUDREY A. HAZAMY

B.A., Florida Atlantic University, 2007

A Thesis Submitted to the Graduate Faculty of Georgia Southern University in Partial

Fulfillment of the Requirements for the Degree

MASTER OF SCIENCE

STATESBORO, GEORGIA

2009

AUDREY A. HAZAMY

The Influence of Pictures on Word Recognition

AUDREY A. HAZAMY

Major Professor: Lawrence Locker

Committee: Karen Z. Naufel

John D. Murray

Electronic Version Approved:

May 2009

ACKNOWLEDGMENTS

I would like to acknowledge the advice, direction, and support of Dr. Lawrence Locker,

committee chairman. I also thank the members of my graduate committee, Dr. Karen Z. Naufel

and Dr. John D. Murray for their guidance, suggestions, and encouragement throughout this

process.

TABLE OF CONTENTS

Page

ACKNOWLEDGEMENTS...…………………………………………………………………… 5

CHAPTER

1 INTRODUCTION……………..……………………….……………………………… 7

When Pictures Facilitate Memory ……….....…………………………………..… 8

Facilitative Applications of Pictures.…….….....….…………………………….. 14

Non-facilitative and Potential Intrusion Effects of Pictures…...….……….……. 17

Semantic Effects of Pictures on Words……………….…….…………………… 21

Purpose…….……………………………………………….……………………. 26

2 EXPERIMENT 1 ………......………………………………………………………… 28

Method…….…………………………………………………………………….. 28

Results and Discussion ...…….…………………………………………………. 29

3 EXPERIMENT 2 ……......…………………………………………………………… 31

Method………….…..………………………………………………………….... 31

Results and Discussion…..…...…...…………………………………………….. 32

4 GENERAL DISCUSSION …...…...……………………………….………………… 33

REFERENCES.………..…….………………………………………………………….…...…. 40

APPENDICES

A SAMPLE STIMULI EXPERIMENT 1…...………………………….……………… 45

B SAMPLE STIMULI EXPERIMENT 2………………………………….…………... 46

CHAPTER 1

INTRODUCTION

Psychologists have expansively studied the differences between the encoding and

processing of words and pictures. Perhaps one of the most extensively investigated of these

differences is known as the picture superiority effect, which is the finding that items presented in

picture format are better remembered than those presented in word formats (Nelson, Reed, &

Walling, 1976; Stenberg, Radenborg & Hedman, 1995). Many differing perspectives have

emerged over the years as to the source of this effect. Although diverse in their approaches and

explanations, the superior processing nature of pictures is the primary tenet of these various

theories. Due to their encoding benefit, pictures are utilized in numerous instances such as

teaching sight vocabulary, and pairing pictures with words and text to provide clarity to overall

meaning. The beneficial aspects of using pictures with words and texts, and the superior nature

of picture processing are well replicated and are demonstrated throughout the literature (e.g.,

Nelson et al.; Waddill & McDaniel, 1992; Stenberg et al). A review of the literature investigating

the phenomenon provides well-documented explanations of the mechanisms behind the

facilitative nature of pictures on word encoding. However, it also reveals a paucity of research

regarding the potential intrusion of pictures on word processing and memory. Intrusion effects of

pictures would be a vital key to understanding how pictures influence word processing and how

pictures themselves are processed differently than words. Research concerning intrusion effects

of pictures on word processing will potentially allow for the more effective use of pictures in

some instances, and the ability to recognize when it may not be beneficial to implement their use.

When Pictures Facilitate Memory

Dual-coding Hypothesis. Perhaps the most well-known explanation for the picture

superiority effect is the dual-coding hypothesis (Paivio, 1969; 1971). Dual coding has been the

basis of numerous theories and debates regarding picture superiority. The hypothesis states that

pictures and words are processed differently such that pictorial information is stored dually

(visually as well as acoustically), whereas words are stored via only an acoustic/verbal store and

thus only in one form (Paivio, 1971; Pellegrino, Siegel & Dhawan, 1975; Snodgrass & McClure

1975; Nelson et al., 1976; Mintzer & Snodgrass, 1999).

To demonstrate support for the dual-coding hypothesis Jenkins, Neale and Deno (1967)

compared the recognition of pictures and words. Jenkins et al. presented participants either

pictures or words and then tested recognition memory using either the same or the alternate

format. That is, words or pictures were presented in a study condition and then recognition was

tested using either the word or the picture representation of the word. The picture-word condition

revealed highly similar results to the word-word condition, suggesting that the pictures are stored

using a verbal representation in addition to a visual store. The greatest difficulty in correctly

identifying the original stimuli was observed for participants in the word-picture condition. This

difficulty is also indicative of the idea that words are not encoded using a visual form, but rather

only via a verbal form.

Pellegrino et al. (1975) also provided support for the dual code hypothesis by examining

short-term retention of pictures and concrete words. Pellegrino et al. used auditory and visual

distraction in an attempt to interfere with the encoding and retention of a picture or presented

word. Auditory distraction was created by having participants count backward out-loud by 13.

The visual distraction required participants to perform a visual search task to find a shape within

a matrix of lines (known as the hidden figures task). Auditory distraction led to better retention

for pictures than words, presumably due to the additional visual store associated with pictures.

Visual distraction was not associated with detriment for pictures compared to words. These

results were presumably due to the acoustic store that both possessed. Picture memory could be

based on an acoustic store, thus leading to no detriment compared to words. However, the use of

both visual and acoustic distraction produced a significant decline in picture memory, below that

of the decline in participants’ word memory, providing support for the idea that pictures are

dually encoded (Pellegrino et al.).

Snodgrass and McClure (1975) explored the dual-code hypothesis by instructing

participants to manipulate the type of encoding used for words and pictures. Participants verbally

rehearsed items, or rehearsed items using visual imagery. Participants in either of the picture

recognition conditions, (i.e., verbal rehearsal or rehearsal using imagery) showed no difference in

recognition accuracy. Thus, memory for pictures was not influenced by the instructions given.

There was however a significant difference in the word conditions. Words rehearsed using the

imagery strategy were better remembered than those that were repeated using the verbal

rehearsal strategy, suggesting acoustic storage is the default (as discussed by Jenkins et al., 1967)

unless otherwise instructed. These results indicated that pictures are dual coded automatically

whereas words are only dual coded when participants are initiated to do so.

A dual coding view for vocabulary learning was provided by Sadoski (2005). He

suggested that the dual coding theory provides a dominant framework for effective vocabulary

learning in children. One study noted in Sadoski’s work by Arlin, Scott and Webster (1978,

1979), examined the effectiveness of using pictures, auditory pronunciation, or no alternate

method in conjunction with the printed word form when children are learning sight vocabulary.

Overwhelmingly, the children who were presented pictures in combination with the printed word

form acquired the sight vocabulary significantly faster than those in the other conditions. Taking

into account this and other previous research, Sadoski concluded that children provided with

pictures when learning concrete words are better able to learn the words than children who are

not provided with a picture complement. According to Sadoski, these findings support the dual

code theory that pictures are encoded both non-verbally and verbally. With the use of pictures,

the meaning of the word is expanded beyond plain text recognition and processed along more

than a single acoustic store as would be the case if words were presented without a picture.

Thus, the pairing of pictures with words helps establish multiple connections between the verbal

and non-verbal representation of a word. The dual coding effects of pictures presented with a

visual label of a word allows for better vocabulary acquisition when children are learning to

sight-read. Numerous studies such as those mentioned above have provided strong support for

Paivio’s (1969, 1971) dual-code hypothesis. However, other perspectives have also been

proposed that provide differing and alternate explanations for the picture superiority effect.

Alternative Explanations. Researchers have proposed the sensory-semantic model of

picture superiority that states that rather than being processed along dual stores, pictures are

processed along a single store, although this single store is qualitatively superior in nature to that

of word stores (Nelson et al., 1976). Essentially, pictures provide qualitatively superior sensory

information for coding than do words, and are thus easily processed and encoded into memory.

In their study, Nelson et al. hypothesized that the picture superiority effect might be reversed by

the same sensory codes that normally lead to an advantage. They demonstrated this by providing

participants a list of words or pictures then manipulated the pictures’ conceptual and schematic

similarities. For example, pictures appeared similar or different to one another in terms of

configuration and shape, making them less or more distinguishable from one another.

Participants least remembered pictures with high degrees of conceptual and schematic

similarities with other pictures on the list, even compared to word memory. The lack of

distinctiveness between pictures supposedly neutralized the superior sensory code and thus

resulted in more errors. However, when conceptual and schematic similarities between pictures

were low (i.e. pictures were easily distinguished from one another), the picture superiority effect

remained. Thus it can be concluded from this research, that rather than being encoded in two

forms as the dual-code hypothesis states, pictures may be better remembered because they

provide participants with more distinguishable and qualitatively superior perceptual encoding

information than do words.

In addition to the sensory-semantic model, Nelson and Reed (1976) also incorporated a

levels of processing (Craik & Lockhart, 1972) component into their explanation of the generally

superior memory for pictures over their verbal labels. According to Nelson and Reed, pictures

are not automatically given a verbal label, as suggested by the dual code hypothesis, but rather

the extent of verbal labeling is controlled by the learner. The learner may apply a label to a

picture if the situation calls for a label, such as in order to assist in the processing and storage of

the stimuli. However, in order to convey the meaning of a picture, a verbal label need not be

applied and thus the application of a label is not necessarily spontaneous. A level of processing

theory may be more applicable rather than a dual coding strategy, as pictures were not given

word labels by participants on all trials (Nelson & Reed, 1976). Instead, it was suggested that the

picture-superiority effect may more likely be a result of qualitative differences at the sensory

level in which pictures provide a better mnemonic representation and subsequently lead to a

deeper level of processing than that of their word labels.

Alternatively, Stenberg et al. (1995) offer a common code theory of semantic activation

in which semantic links provide pictures their superior advantage to verbal labels. According to

their theory, pictures activate semantic links better than verbal labels, implying that pictures are

superior in activating memory cues than words. Stenberg et al. provided participants pictures and

words to study in an initial task and then tested their memory using a subsequent memory task.

Not surprisingly, results revealed that accuracy and reaction times were both faster for picture

stimuli than for the words. Using cross-modal priming, in which words are primed using pictures

and vice versa, they also found that words that were primed using their picture form were better

recognized than pictures that were primed using their word form. From these results, it was

concluded that processing of pictures may prime and activate analogous semantic nodes more so

than do words. These additional semantic links activated during processing may aid retrieval

during a subsequent search during a memory test. These results support a privileged, semantic

based access model for picture superiority, in which pictures lead to superior semantic encoding.

Others have offered another comprehensive argument against the dual-code hypothesis.

Implementing the form-change paradigm (see Jenkins et al., 1967) in which pictures and words

are studied and then tested using their same or alternate form, Mintzer and Snodgrass (1999)

provided support for the distinctiveness model of picture superiority. The most prominent

component of the distinctiveness model states that pictures have highly distinctive semantic and

sensory features that make them highly distinguishable from one another and from words.

Words, however, do not have such features. Support for the theory was demonstrated in that

pictures that were studied and presented as pictures showed the best recognition memory,

supposedly due to their mnemonic superiority compared to words. A form-change cost is the

difference in recognition memory performance between items studied and tested in the same

form and items studied and tested using their alternate form. In addition to the finding that

pictures presented in the recognition task as pictures best facilitated participants' performance,

the study also found that pictures had a much more significant form-change cost to the

participant’s recognition memory than did words. That is participants’ performed worst when

pictures were presented in the recognition task as words. According to the model, this form-

change cost occurs because the pictures loose their mnemonic edge over words (Mintzer &

Snodgrass). Also, because pictures had the greatest form-change cost, results were inconsistent

with the idea that the pictures are dual encoded, as a verbal code was apparently not evoked for

the pictures. Rather, much like the theories of Nelson et al., (1976) and Stenberg et al., (1995),

Mintzer and Snodgrass suggest that the superior memory for pictures is due to their distinctive

sensory and semantic properties and not due to an additional verbal store.

Stenberg (2006) provided further insight into how conceptual and perceptual aspects of

pictures may provide pictures with their superiority over words. While conceptual factors such as

semantic meaning and perceptual aspects such as discriminating sensory details were

investigated before (as described above), Stenberg attempted to distinguish which of the two

factors played a more significant role in the superiority effect. Using orientating tasks and

encoding manipulation, participants were directed toward more conceptual or perceptual

processing of the pictures and words during a study task. Results indicated that although

perceptual form based processing played a role in picture processing, there was a large difference

between conceptual based and perceptual based processing. The results favored the notion that a

larger contribution to the picture superiority effect is that of conceptual meaning based

processing.

As these studies demonstrate, the majority of the alternatives to the dual coding

hypothesis emphasize semantic and distinctiveness explanations of the effect. However, all the

studies described above, although having slightly differing perspectives on the picture superiority

phenomenon, provide overwhelming support for the notion that pictures are better processed and

encoded relative to words.

Facilitative Applications of Pictures

Thus far, the literature has largely discussed the nature and hypotheses behind the picture

superiority effect. I will now turn to the applicability and facilitative nature of using

pictures/illustrations in conjunction with words. A great deal of research has been conducted that

explores how pictures facilitate learning and text/word comprehension. Waddill, McDaniel and

Einstein (1988) examined whether the type of text (either narrative or expository) and the type of

picture presented with the text (either detail or relational) would influence memory for the text. It

is presumed that pictures primarily serve a supplementary function in that they reinforce the

information provided via text. In their study, participants were presented either a fairy tale

(narrative) or an expository text (i.e., texts that aim to explain something such as the information

found in an encyclopedia) in conjunction with detail information (e.g., illustrations that portray a

specific detail of a proposition), relational information (e.g., illustrations that connected ideas

and causal relationships of many propositions) pictures or no pictures. In some cases,

participants were directly instructed to attend to the pictures, while in others participants were

not. The latter case allowed for incidental learning. Results indicated that when no instruction

was given to a participant to attend to the pictures, the pictures served only to reinforce the

important target information provided in the text. That is, only target information was better

remembered. Additionally, pictures only increased free recall when the picture was designed to

depict the specific type of information the text presumably invited. Expository texts are

presumed to elicit more individual-item processing (detail) and narratives are presumed to elicit

relational processing (Waddill et al.). Thus, recall of detailed information was only significantly

better for expository text and recall of relational information was better for the fairy tale

condition. However, when instruction guided participants to pay particular attention to the

pictures, they served not only to reinforce the target detail and relational information regardless

of the condition, but also directed attention to non-target information. Thus, pictures do appear to

support text processing, although the nature of the facilitation varies depending on differing

conditions.

In a follow-up study, Waddill and McDaniel (1992) again examined the pictorial

enhancement effects on detail and relational information within text. They analyzed the

facilitative effects of pairing pictures with text by providing participants expository text with

loose conceptual coherence or with explicit conceptual relationships. The use of explicit

conceptual relationships in text conveyed ideas and connections more overtly by using phrases

such as “by contrast”, “more than” etc. For both types of text, results indicated that the types of

illustrations paired with the text had an effect on the subsequent recall of the text. Recall of both

relational information and detail information was significantly better in the group provided

illustrations than those provided no illustration supplements regardless of text type, indicating

that pictures enabled further processing than would occur had no pictures been present (Waddill

& McDaniel). Likewise, in a study reported by Glenberg and Langston (1992), participants in

one group were provided procedural texts accompanied by appropriate illustrations and

participants in the other group were presented with lone text. During a speeded test, participants

had to respond "yes" or "no" in reference to questions about two steps in a procedure (e.g.,

"Would the step containing the phrase on top immediately precede the step on the bottom […]"

Glenberg & Langston). The speeded test was followed by true/false comprehension questions.

The assumption was that if participants represented the procedure in a mental manner, they

would more quickly and accurately answer questions about far pairs in a procedure (e.g., step 1

and 4) because in a mental representation of the procedure, the middle steps are strongly related

to the preceding and succeeding steps equally. Participants provided with the text and

appropriate pictures tended to represent the procedure mentally, rather than simply mentally

representing the text, as was the case in the no picture condition. The results of the study suggest

that pictures aid the comprehension of corresponding text, via building representative mental

models and drawing inferences as to what the text was about.

The use of pictures at the time of encoding of associated words that converge onto a lure

word can also reduce the later false recognition of words (Israel & Schacter, 1997). Based on the

methodology of Roediger and McDermott (1995) in which participants often falsely recognize

lure words related to words in a presentation list, Israel and Schacter introduced the use of

pictures to the task of distinguishing semantically related words and lure words in a recognition

task. Participants were presented words either in auditory form, or in both auditory and visual

form. Additionally, participants were provided either pictorial enhancement in the form of a line

drawing or no pictorial enhancement. The use of pictorial stimuli reduced the amount of false

recognition in a subsequent memory task that included lure words as well as non-lure words and

enhanced the ability of participants to correctly distinguish between old and new items in a

recognition task list. Again, studies such as this support the usefulness of using the distinctive

aspects of pictures.

Levin (1981) (as cited in Carney & Levin, 2002) perhaps provided the most

comprehensive assertion of how pictures influence text processing in his proposed “five

functions of pictures”. The first of the five functions is the simple decorational function, in which

pictures presented along side text are primarily for embellishment purposes. The second is the

representational function in which pictures simply mirror text. A representational picture literally

depicts the text it is accompanying and its purpose is to make the text more concrete to the

reader. Third, organizational illustrations provide a structural framework for the text such as

illustrations of a procedure a person would typically find in an instructional manual for how to

put something together. Organizational pictures are useful when attempting to depict a series of

steps. Interpretational pictures, such as those often found in science books clarify complicated

and difficult text (e.g., biological processes). Finally, transformational pictures are systematic

memory enhancing devices that are specifically designed to improve the recall of information

(Carney & Levin, 2002). Often referred to as mnemonic pictures, Levin (1986) also found that

transformational pictures assisted students in higher order cognitive application and applying

problem solving skills to the text in question. Transformational pictures often use illustrations of

keywords to prompt the memory of the reader, [e.g., the use of a bell to depict the name of a key

city Belleview, (Dretzke, 1993 as cited in Carney & Levin)]. In summary, the use of pictures in

conjunction with text provides participants with numerous processing and encoding advantages

over the presentation of text alone. The five functions of illustrations indicate the numerous ways

in which pictures can facilitate text comprehension as well as text recall.

Non-facilitative and Potential Intrusion Effects of Pictures

The literature discussed thus far has emphasized the superior memory for pictures over

words and the facilitative nature of pairing pictures with words and text passages. Although a

primary purpose of this study is to further explore the facilitative nature of pictures on word

memory, exploring the potential intrusive effects of pictures on word memory is also an

important objective. However, there appears to be little prior work that discusses potential

intrusion effects of pictures on the memory of words. Of the literature that has explored the

possibility of negative effects of using pictures for verbal enhancement, the most widely

discussed is the issue of using pictures in tandem with words when teaching children vocabulary.

Some research suggests that pictures aid vocabulary learning (e.g., Sadoski, 2005; Goodman,

1965) and the use of picture enhancement is a generally accepted practice in schools when

teaching children new sight words. However, as the following literature suggests, this common

practice may not be an effective one. Singer, Samuels and Spiroff (1974) reported a study in

which the use of pictures and contextual conditions supported the idea of Samuels (1967) focal

attention theory. Samuel’s focal attention theory states that the use of pictures and contextual

cues are detrimental to the learning of words in new readers as they detract from the learning of

the words graphic features (i.e. they distracted the learner from focusing attention to the word

itself). Results of the study supported this notion in that children who were presented only words

learned the words more efficiently than those who were provided pictorial or contextual cues. In

a similar study, Solman and Singh (1992) investigated the effects of using pictorial enhancement

when teaching children to read words. The children were presented four conditions: two simple

conditions in which a word was presented alone, either in small print or in large print; two

compound conditions in which pictures were presented with the word as either a line drawing

above the word, or below the word. In contrast to studies that suggest pictures effectively

facilitate the memory and learning of words (e.g., Sadoski; Arlin, Scott & Webster, 1978-1979 as

cited in Sadoski, 2005; Goodman), children had better accuracy in naming the sight words in

both the simple word conditions than in either of the compound picture conditions. In a similar

study, mentally retarded children acquired sight vocabulary faster when words were presented

without pictorial enhancement or prompts (Singh & Solman, 1990). It was argued that blocking

may be responsible for the hindrance pictures pose on written word acquisition. That is, a

previously learned association of the picture and verbal response (verbal naming of the picture)

may interfere with the learning of the written form of the word. In a sense, the old association of

the picture and verbal response blocks the new association of the picture and written word.

In a comparable study, Conley et al. (2004) also examined the use of pictures in the

acquisition, as well as the maintenance of sight words, in children learning to read. This study

compared the use of the cover, copy, and compare (CCC) method to the use of the picture

matching teaching method. The CCC method requires that the student cover a correct model of

the word on one side of a page with an index card and then write the word in the space on the

other side of the sheet. The student then uncovers the correct model and checks his/her work. In

addition, students in the CCC group were instructed to trace the word before attempting to write

it on their own. Students in the picture-matching condition were presented either a picture or a

word card by the investigator and were asked to match the card with the corresponding picture or

word card from a stack of their own. Although initial results suggested that the picture-matching

approach required fewer sessions to acquire mastery of the vocabulary, further assessment

indicated that in the CCC condition, word maintenance was ultimately higher than in the picture-

matching condition. Additionally, Conley et al. found when the pictures were subsequently

removed in the picture-matching condition, children began to misidentify the words, suggesting

that in the picture-matching conditions children were picture reading and not word reading.

Overall, the outcomes suggest that the use of methods such as the CCC method may be more

effective than picture-word methods.

Although the literature discussed in these studies identifies the problem of pairing

pictures with words for children learning to read, to the author’s knowledge, relatively little

research has focused on the possible hindrance of pairing pictures with words in adult readers.

However, pictorial intrusion effects on word processing have been investigated in the context of

asking participants to make a particular judgment about a word in the presence of a distracting

picture. Research has shown that one such judgment, the classification of words into categories,

is affected by the presence of pictures (Smith & Magee, 1980, Arieh & Algom, 2002; Lupker &

Katz, 1982). Smith and Magee conducted a study in which participants were asked to classify

pictures and words in the presence of incongruent stimuli by making “yes” or “no” responses

regarding whether the stimuli belonged to a specific category. Word categorization was found to

be significantly delayed in the presence of distracting pictures. The categorization of pictures

however was not affected by the presence of distracting words. Lupker and Katz also studied the

phenomenon. Participants made category judgments regarding word stimuli in the presence of

incongruent pictures. Participants were specifically asked to ignore the pictorial stimuli and

simply respond “yes” or “no” as to whether the word stimuli fell into the category “animal”. In a

second experiment, participants made “yes” or “no” judgments as to whether the target word was

the word “DOG”. The researchers found that the incongruent background pictures interfered

with the word judgments. Lupker and Katz, and Smith and Magee accounted for their results in

terms of automatic semantic processing theory of pictures. That is pictures are automatically

placed into semantic categories and allow faster semantic access than do words. Because the

categorization tasks require the semantic analysis of a word, and because the incongruent

pictures are placed into semantic categories faster than are the words, the pictures intrude on

word judgment. Based on the notion of automatic semantic picture processing, it can be

concluded that in some contexts pictures that are incompatible with a correct word response can

negatively influence word judgments. A similar argument made by Arieh and Algom (2002)

suggested that pictures are categorized faster than are words and that word categorization suffers

from intrusion by incongruent pictures. They also found that the influence and the amount of

interference is largely subjected to contextual variations. Contextual changes (e.g., contrast

changes between the print and background, reducing/increasing the size of pictures, and adding

an x to the prefix and suffix of a word) in the stimuli served to reduce/eliminate or enhance the

pictorial interference on word categorization. Thus, these results further complicate the issue of

how pictures intrude on words in varying types of tasks.

Semantic Effects of Pictures on Words

The semantic intrusion effect of pictures (as demonstrated by the automatic semantic

processing theory of pictures) on word judgments suggests that there is a potential for intrusion

effects of semantically related pictures on word processing. Thus, in addition to addressing the

interference of pictures on word encoding and retention, the present study will also attempt to

investigate the effect of semantic relatedness of a picture on word memory. Although research

supporting the automatic semantic processing theory examined effects of semantically related

pictures on word judgments, no known research has been conducted to directly investigate the

semantic relatedness of pictures on subsequent word memory. Research has however been

directed to studying the effects of semantically related distractors on stimuli using the picture-

word interference task. The picture-word interference task consists of presenting participants

line-drawings and then asking participants to name the picture in the presence of a distractor

word. On the whole, research overwhelmingly suggests that the use of words as distractors using

the picture-word naming task slows response time in the naming of the target pictures (Smith &

Magee, 1980). More importantly to this study, a great deal of research has also been conducted

that explores the effects of different types of words (i.e., semantically related) on picture naming

(Rosinski, 1977; Caramazza & Costa, 2000; Costa, Alario & Caramazza, 2005; Damian &

Bowers, 2003 and Mahon et al., 2007). For example, using both children and adult groups,

Rosinski demonstrated semantic interference in the picture-word interference task. Participants

were presented pictures in the context of distractor words and were asked to verbally label the

picture while ignoring the word distractor. The words were either congruent with the picture

label, a consonant-vowel-consonant trigram, a semantically related word within the same

category as the picture (e.g., the word “lion” presented on a picture of a pig) or a non-congruent

word (e.g., the word “cup” presented on the picture of a pig). The same overall effect was noted

in all age groups; semantically related words produced a larger amount of interference to picture

naming than did un-related words and non-sense trigrams, suggesting that the semantic

relatedness of the distractor plays a role in the ability to verbally name pictures in the

interference task. Numerous other researchers have also concluded, as did Rosinski, that

semantically related distractors from the same semantic category leads to pronounced

interference effects, and thus the semantic relatedness of a distractor affects the magnitude of

interference in the picture-word interference task (Costa, et al., 2005).

Along the same lines, Vitkovitch and Tyrrell (1999) discussed the effects of distractor

words on the subordinate name retrieval of a picture. The picture-name interference task often

focuses on basic level naming of a picture (e.g., dog) whereas this study focused on the

subordinate naming or more specific naming (e.g., poodle) of an object. Participants were

presented picture and word conditions in which the distractor word was semantically related,

unrelated or neutral in regard to the picture stimulus. Results revealed that when pictures were

paired with subordinate distractor words that were from the same basic-level category (e.g.,

picture of a poodle and the distractor word spaniel) or semantically related basic-level category

(e.g., picture of a poodle and the word koala) as the target picture, subordinate naming latencies

were longer than unrelated or neutral words. The results support the notion that there is a

semantic relatedness interference effect. That is, semantic relatedness of a word from the same or

related basic-level category of a target picture can produce semantic interference/competition in

subordinate picture naming (Vitkovitch & Tyrrell).

Based on the research discussed above, the interference produced by semantically related

information had been termed the semantic interference effect. More specifically, it refers to the

notion that participants are slower to name a picture of an object in the presence of a

semantically related distractor than in the presence of an unrelated one. Perhaps the most

prominent hypothesis behind the interference effect is the notion of lexical selection by

competition/conflict. The main premise behind the lexical selection by competition is that given

a picture (e.g., car) and a semantically related distractor (e.g., truck) or an unrelated distractor

(e.g., hat), the semantically related distractor activates more semantically related lexical nodes to

the target than does the unrelated distractor. The simultaneous activation of the related lexical

nodes, in addition to the activation of the target node is thought to create competition in the

retrieval process and may be the source of the interference in the picture-naming task (Costa et

al., 2005; Mahon et al., 2007, Damian & Bowers, 2003).

Given the research discussed above, the question then becomes whether the same

intrusion effects are observed in which the processing of semantically related pictures interferes

with the processing of a target word. As mentioned, the direction of the reverse effect, that of

picture interference on word naming has been less researched and defined than that of the word

intrusion effect, and thus there is debate regarding whether there is an intrusion of pictures on

word naming. Smith and Magee (1980) did note that most research thus far suggests that the

naming of words in the presence of incongruent pictures is relatively un-affected, perhaps due to

the automaticity of reading words. For instance, Rosinski, Golinkoff and Kukish (1975) found

picture naming was largely disrupted by the presence of incongruent word stimuli, whereas the

naming of words was relatively unaffected by incongruent pictorial stimuli. Because of the

relatively little interference pictures have on word naming, the semantic relatedness effect of

pictures on word naming has not been extensively uninvestigated. However, the automatic

semantic processing theory of pictures noted earlier suggests that the semantic relatedness of

pictures may have an effect on how words are processed, although the task used might be a

critical factor.

To further complicate the potential semantic effects of pictures on word memory,

research has also suggested the possibility of semantic facilitation under some conditions. Using

the picture-word interference paradigm, Costa et al. (2005) noted that distractor words that fell

into the same semantic category as a picture had in fact produced semantic interference (e.g. a

target picture of a car and a distractor word truck) as suggested by previous studies. However,

they also noted that if a distractor word was semantically related but from a different category,

(e.g. the target picture car and the distractor word bumper) semantic facilitation rather than

interference was observed. In a similar study, researchers Mahon et al. (2007) also noted

semantic facilitation effects using the picture-word interference task. Compared to unrelated

distractor words, their first set of experiments demonstrated that target pictures (e.g., bed) were

named faster in the presence of a semantically related distractor verb (e.g., sleep) than unrelated

distractor verbs (e.g., shoot). Additionally, it was noted that target naming of a picture (e.g.,

horse) was faster for within-category semantic distractors that were close in relationship to the

target (e.g., zebra) than for within-category semantic distractors that were farther in relationship

to the target (e.g., whale). That is, the closer the semantic relationship to the target, the greater

the facilitation effects and the faster the target naming. Studies such as those just discussed

suggest that semantic facilitation rather than intrusion effects in target picture naming may occur

in the picture-word interference task in the presence of semantically related distractors. Given the

task variations, the question then becomes whether semantically related pictures will have

similar effects on word memory. The potential for both intrusion and facilitative effects in the

picture-naming task have been demonstrated and suggests that the effects of semantically related

pictures on word memory could facilitate or inhibit depending on the specific conditions.

The literature reviewed has illustrated a great deal support for pictorial facilitation effects

as well as the pictorial superiority effect. Many views and theories have been proposed

concerning the superior nature of picture encoding over word encoding and research

investigating how pictures reinforce text and word processing has largely supported these views.

Although the literature provides some indication of picture intrusions on word categorization, the

overall intrusion of pictures on words is still a relatively unexplored area of research. Beyond the

scope of the potential negative effects of pictures when teaching children sight vocabulary,

research concerning picture intrusion on subsequent word memory and learning is largely absent.

The potential effects of pictures at the time of word encoding on subsequent word memory has,

to the author’s knowledge, not been investigated. Although a great deal of information has been

acquired regarding pictorial influence on word processing, there remains much to be learned

regarding the nature of the relationship between picture and word processing.

Purpose

The purpose of the current study is to explore further the processes involved in memory

for words in the presence of accompanying pictures. Due to the superior encoding nature of

pictures, it was expected that the presence of pictures when encoding words will provide either

facilitation or interference depending its relationship to the target word. Most research regarding

adults suggests that corresponding pictures facilitate word processing (e.g., Stenberg et al., 1995;

Israel & Schacter, 1997; Waddill et al., 1988). Thus, for the first experiment, it was predicted

that this facilitation effect will carry over to the memory of to-be-remembered words in a list

presented with matching pictures. However, the superior encoding nature of pictures was

expected to interfere with the encoding and memory of words in a condition in which the picture

is incongruent and unrelated to a word. Though research showing the inhibitory effects of

pictures on words is scarce, the existing research suggests that inhibitory effects may take place

in certain instances (e.g., categorization; Smith & Magee, 1980, Arieh & Algom, 2002; Lupker

& Katz, 1982). As pictures are more easily encoded and remembered than words, the presence of

a distractor picture may inhibit the effective encoding of a word into memory. Lastly, according

to the dual code hypothesis, pictures are encoded both visually and non-visually (Paivio, 1971;

Pellegrino et al., 1975; Snodgrass & McClure 1975; Nelson, et al, 1976; Mintzer & Snodgrass,

1999). Due to the additional verbal code pictures receive with the visual code, if the dual code

hypothesis is tenable, pictures that are presented in the recognition condition in their alternate

word form, may be mistakenly remembered as being presented in word form during the study

portion.

Semantic relatedness research using the picture-word interference task suggests that the

semantic relatedness of a distractor to a target stimulus may affect the encoding process. The

semantic processing theory of pictures, as demonstrated in studies such as Lupker and Katz

(1982), as well as the semantic research using the picture-word interference task illustrate the

intrusion effects that semantically related stimuli have on a target stimulus. Alternatively, other

research using the picture-word interference task also suggests the possibility of semantic

facilitation effects (Mahon et al., 2007; Costa et al., 2005). The majority of this research has

examined the influence of words on pictures. Thus, the current study will extend this research

through an examination of pictures on the processing of words. It was hypothesized that

semantically related pictures will influence the memory of target words, although it is possible

that these effects may be intrusive or facilitative. In summary, the present study served four

purposes. First was to establish whether matching pictures presented with words facilitates the

later memory of the words. Second, was to investigate whether incongruent/unrelated pictorial

stimuli will inhibit the memory of the target words. Next was to investigate whether semantically

related pictures presented in Experiment 2 will have an inhibitory or facilitative effect on word

memory. Fourth, although a less central purpose, was to establish whether the incongruent

pictures from Experiment 1 are dually encoded and later falsely recognized in their word form

during the recognition test.

CHAPTER 2

EXPERIMENT 1

Method

Participants. Participants were 48 (14 males, 34 females) Introductory Psychology

students at Georgia Southern University who participated as a requirement for course credit.

There were originally 51 participants, however, 2 participants were not included in the analysis

due to experimenter error, and 1 participant was not included due to equipment error.

Design and procedure. Each participant was presented with one of three possible lists of

120 to-be-remembered stimuli, consisting of 40 items per condition. Three different lists were

employed so that across participants all stimulus words appeared in each of the three conditions.

Participants were randomly assigned to one of the three lists. The stimuli consisted of words in

lower case letters with a picture presented above the word. Participants were instructed to devote

their attention to the words presented in the stimuli pairs and not the pictures accompanying the

words. The pictorial stimuli consisted of familiar items adapted by Rossion and Pourtois (2004)

from Snodgrass and Vanderwart’s (1980) black and white pictorial database. Rossion and

Pourtois converted the pictures to color and collected normative data on the objects and their

labels. Critical stimuli were selected at random, with the restriction that items with two word

labels (e.g., baby carriage) were not included. Fillers in the recognition task were also selected

from this database.

Three word-picture conditions were presented to participants. The first condition

consisted of 40 words with matching pictures, the second condition consisted of 40 words

presented with a neutral picture (i.e.: a row of four X’s), and the third condition words were 40

words presented with incongruent pictures (see appendix A for examples). In the incongruent

condition, the forty pictures were paired words unrelated to the picture. These 40 incongruent

picture-word pairs were used across all lists.

Stimuli were presented to participants on a computer screen at two-second intervals.

After the presentation of the 120 stimuli, participants were given a recognition task in which they

were asked to indicate whether they had seen a given word in the study portion. The recognition

task consisted of 80 filler words that the subject had never seen, the 120 critical words previously

presented, and the 40 alternate word form of the incongruent pictures that were presented in the

study task. That is, if a picture of an airplane had been presented in the study task with an

incongruent word, the word airplane was included in the recognition test. Participant results were

based on mean accuracy rates for each participant in the recognition task. Participants responded

“yes” or “no” using the keys on a keyboard “a” or “s” for left handed, and the keys “k” or “l” for

right-handed participants.

Results and Discussion

A one-way repeated-measures ANOVA was used to analyze differences among mean

accuracy rates. The picture manipulation had an effect on the recognition memory of the words,

F(2, 94) = 19.03, p < .05. Pairwise comparisons were analyzed using a Bonferroni adjustment.

Words presented with matching pictures (M = .62, SEM = .02) were better recognized than

words presented with incongruent (M =.54, SEM = .02) or neutral pictures (M =.53, SEM = .02).

There were no other differences among the conditions. Finally, the hypothesis regarding the

dual-code theory in which it was hypothesized that incongruent pictures that were presented in

their alternate form during the recognition task would have a high false positive rate was not

supported by the results. For the incongruent pictures that were presented as words, the false

positive rate was (26%), which although somewhat higher then false positives for fillers (14%),

was substantially lower (in terms of participants believing that they had previously seen the

word) than either the neutral, congruent or incongruent conditions. That the incongruent pictures

presented in word form revealed relatively low false positive rates, in tandem with the facilitative

effect of matching pictures would seem to provide support for frameworks such as the sensory-

semantic model (Nelson et al, 1975) and common code theory (Stenberg et al., 1995) rather than

a dual-code theory of picture superiority (Paivio, 1969,1971).

The result regarding the matching picture condition is consistent with the well-

documented facilitative nature of pictures on memory. Participants recognized words that were

presented in the matching word-picture condition more accurately than those presented in any of

the other conditions. In contrast, there was no significant difference between the recognition of

words in the incongruent or neutral conditions. Although pictures are presumed to be more easily

encoded and stored in memory as demonstrated throughout the literature, the presence of the

irrelevant pictures did not interfere with the encoding of the words. This finding suggests that

although pictures may be processed more efficiently or easily in some cases, they do not take

primacy over words in an automatic fashion in a task in which verbal memory is emphasized.

The extent to which this finding extends to picture word relationships beyond simply completely

incongruent or exact matches was explored in Experiment 2.

CHAPTER 3

EXPERIMENT 2

Method

Participants. Participants were 50 (14 males, 36 females) Introductory Psychology

students at Georgia Southern University who participated as a requirement for course credit.

There were 51 original participants, however, one participant was not included due to equipment

error.

Design and procedure. As in experiment 1, each participant was presented with one of

three lists consisting of 120 to-be-remembered stimuli, 40 items per condition. The pictorial

stimuli were again selected from Rossion and Pourtois (2004). The three conditions for

experiment 2 were as follows: semantically related, unrelated/incongruent and neutral (i.e., a row

of four X’s) picture-word pairs (see appendix B for examples). The stimuli for the semantically

related condition consisted of a picture and a semantically related word. The associative norms

were taken from the Edinburgh Associative Thesaurus (Kiss, Armstrong, Milroy, & Piper, 1973;

http://www.eat.rl.ac.uk) and The University of South Florida word association, rhyme, and word

fragment norms at http://www.usf.edu/FreeAssociation/. The incongruent condition remained as

in Experiment 1. That is 40 words paired with 40 unrelated pictures. As in Experiment 1, these

were constant across lists. Participants were presented the picture-word and then engaged in a

subsequent recognition task consisting of the 40 semantically related, 40 neutral and 40

incongruent words as well as 120 filler words that did not appear in the picture-word phase of the

study. As in Experiment 1, results were based on the accuracy of the recognition task.

Participants responded by pressing appropriate buttons on the computer keyboard.

Results and Discussion

Two missing data points were not included in the analysis due to lack of participant

response. Results were analyzed using a one-way repeated-measures ANOVA. The picture

condition had an effect on the recognition memory of the words, F(2, 98) = 6.90 p < .05.

Pairwise comparisons were analyzed using a Bonferroni adjustment. Words presented with

semantically related pictures (M = .53, SEM = .02) were better recognized than words presented

with incongruent (M =.49, SEM = .02) or neutral pictures (M=.47, SEM = .02). There were no

other differences among the conditions.

Results indicate that the facilitative nature of pictures is not restricted to exact matches;

pictures that are semantically related to a word also facilitated word memory. The lack of

significant difference between the incongruent/unmatching and neutral conditions replicated the

findings of Experiment 1. Implications of these findings are discussed.

CHAPTER 4

GENERAL DISCUSSION

The purpose of this study was to further explore the effects of pictorial stimuli on word

recognition memory. As evidenced by the results of both experiments, even in the presence of

conflicting stimuli, it appears that participants were able to benefit from relevant pictorial

information while ignoring disadvantageous or irrelevant information. Congruent with research

that has shown pictures to be more easily and efficiently encoded and better remembered (Paivio,

1969, 1971; Snodgrass &McClure, 1975; Nelson et al., 1976; Stenberg et al., 1995; Stenberg,

2006) these results indicate that participants were able to advantageously use pictures in

matching and semantically related conditions to facilitate memory. However, this suggests that

picture superiority does not translate into primacy such that memory is automatically influenced

by pictorial information when that information contradicts the verbal information. The ability to

make use of the pictures in a discriminating fashion, suggests that participants may have

implemented a primarily verbal based memory, selectively supporting this memory when

possible with the visual information.

One possible explanation for the lack of interference by incongruent pictorial information

may be explained in terms of the ability of adults to be able to selectively divert and withhold

attention relative to the stimuli. Research has suggested that underachieving children are more

distracted and exhibit more detrimental effects than overachiever children by distracting stimuli

(Baker & Madell, 1965 and Silverman, Davids, & Andrews, 1963 as cited in Samuels, 1967).

Given that participants were instructed in the present study to focus on lexical memory, and as

adults have fully developed reading skills, it is possible that the adult college population

examined in the present study were reasonably adept at diverting attention from irrelevant

stimuli. The findings suggest that, if task conditions emphasize verbal processing, this is not

superceded even by the rich information contained in distractor pictures.

That this might be a function to some degree of selective attention, is illustrated by the

finding that participants were able to use the matching pictures to their benefit even in the

context of a list that contained mismatched and neutral stimuli. This result in combination with

the finding that the incongruent pictures were not detrimental to memory performance, even

when presented in their alternate word forms, suggests that participants strategically utilized

pictorial information rather than ignoring the pictorial information all together. It could be

argued, however, that pictorial information may not be beneficial when it is semantically related,

but distinct as it would activate a related, but different concept than the target word. The extent

to which semantically related pictorial information positively or negatively affects verbal

memory was examined in Experiment 2. The finding that semantically related pictures were also

beneficial to the memory of words, while again there appeared to be no evidence of interference

by unrelated pictures, supports the above mentioned notion that participants utilized pictorial

information strategically to support verbal memory.

One means by which participants may be able to selectively extract beneficial

information from pictures is if pictures are analyzed primarily at the semantic level (see also

Stenberg et al., 1995). The notion that pictures emphasize semantic processing and are analyzed

primarily at a conceptual level is substantiated by research concerning the picture-word

interference task finding that words interfere with the labeling of pictures, however the naming

of words is unaffected by the presence of incongruent pictures (Smith & Magee, 1980; Rosinski

et al., 1975). Additionally, research also indicates that pictures interfere with the categorization

of words, a semantically based task, whereas picture categorization is relatively immune to the

presence of irrelevant words (Smith & Magee; Lupker & Katz, 1982; Arieh & Algom, 2002).

Thus, taken as a whole, the research lends credence to the assumption that pictures are processed

more rapidly in terms of semantic information but more slowly activate verbal (e.g.,

orthographic-phonological) information. The emphasis of words however is on the orthographic-

phonological aspects with access to semantic information less rapidly available (Smith &

Magee).

These results can also be accounted for within a spreading activation model such as

network models of semantic memory (Collins & Quillian, 1969; Collins & Loftus, 1975). If

pictures do lead to superior semantic processing (Stenberg et al., 1995), congruent pictures might

activate nodes representing word’s meanings to a greater extent, leading to better memory. In the

case of semantically related, but verbally distinct pictures, it could be argued that even though

the semantically related pictures were not an exact match to the target words, they may have

activated analogous nodes within the semantic network of the target, which in turn supported

formation of the verbal level.

If the semantic level activation does indeed support verbal level information, it may be

useful to also consider an explanation within interactive distributed models of word processing in

order to better elucidate the relationship between word and picture processing (e.g., Seidenberg

& McClelland, 1989; Seidenberg, 2005). A distributed model is comprised of separate neuron-

like units representing the different levels of a word: orthographic (spelling), phonological

(pronunciation) and the semantic (meaning). The processing of words involves the activation of

units connected by weights in which the patterns of activation correspond to the different levels

of the word (e.g., the orthography). A main contention of distributed models is the interactive

nature of the unit representations. All of the computational levels of a word are fully

interconnected and act in parallel with one another (see Seidenberg and Seidenberg &

McClelland for further explanation of the models). In a model in which input information such as

a word activates orthographic, phonological and semantic units of processing, memory would

presumably be a function of the orthographic and phonological as well as semantic information

associated with an input word. Pictorial information may be utilized as a form of contextual

information such that participants are able to extract the semantic information from the picture

while disregarding other information that might be activated (the word form of the picture). This

enhanced semantic information (from the picture) would then support the verbal and visual

levels of processing via interactive support from the semantic level (e.g., Pexman and Lupker,

1999; Seidenberg & McClelland, 1989). The notion that the current findings are primarily a

semantic-level effect of pictures is supported by the finding from Experiment 2 in which distinct

pictures that are semantically related to the target word facilitated memory. Also, that pictures

presented in word form did not lead to excessively high false positive rates in the picture-word

condition suggests that semantic information was used to direct attention to relevant pictures (as

participants did not mistakenly recognize the word-level forms of the pictures). Thus, picture-

word processing in this context may be a function of selective attention based on semantic

attributes of contextual pictorial information.

Finally, the results provide some insight into theories of picture superiority, at least in

regard to how pictures are processed. In terms of the dual-code hypothesis, results did not

support the notion that processing of pictures involves an automatic verbal component. As noted,

had pictures been automatically stored dually (visually as well as verbally) as projected by the

dual-code hypothesis (Paivio, 1969, 1971), one would expect a high false positive rate during the

recognition task in which the pictures were subsequently presented as their verbal label.

Although the scope of this study was not to establish which of the many theories of picture

superiority is most accurate, results do most likely exclude a dual-coding view. Results are

instead supportive of theories of semantic based pictorial advantage (Stenberg et al., 1995;

Stenberg, 2006). That is, the basis of picture superiority has been argued to be superiority

primarily in conceptual processing. Stenberg (2006) argues the superior conceptual (semantic)

processing of pictures is a stronger contribution to the effect than perceptual and sensory

properties as emphasized in other theories (Nelson et al., 1976; Mintzer & Snodgrass, 1999).

Arguments for a semantic based advantage for pictorial information assumes pictures more

efficiently access semantic representations than words. Deeper processing at the semantic level

presumably leads to better retention, and thus the picture superiority effect emerges. A similar

argument has been made here in that picture effects were facilitative, but not intrusive as they

were processed primarily in terms of meaning without activating or emphasizing an

accompanying verbal code. It should be noted, however, that the results do not rule out the

possibility that dual-coding may arise in some contexts or tasks in which a dual code might be

beneficial. However, it would appear this is not necessarily automatic, but rather supports the

assertion made by Nelson and Reed (1976) that whether or not a verbal label is applied to a

picture is under control of the individual. That is, the application of a verbal label to a picture

may be a function of the perceived utility for the task. If the verbal label of the picture can

directly relate to the information at hand, the labeling of the picture will most likely be

implemented. However, if the utilization of the verbal label is not beneficial to the participants

(as exampled in the non-matching conditions), a label will not be applied.

Although the pictures did not interfere with word recognition in this particular study,

further research should explore picture-word effects within other contexts. As discussed

previously, much of the research regarding intrusion effects of pictures on words has been

completed within the parameters of children learning sight vocabulary (Singh & Solman, 1990;

Solman & Singh 1992; Conley et al., 2004). Although matching pictures have been shown to be

beneficial to adults, as was also supported by the results of this study, the use of corresponding

pictorial supplements may not be beneficial when provided to children (Singh & Solman;

Solman & Singh; Conley et al.; Samuels, 1967). This differential benefit of using pictorial

supplements between adults and children may be because adults possess better-developed word

processing abilities relative to children. Children may also not have developed the attentional

mechanisms sufficient to selectively extract relevant information from some pictures while

disregarding others. Thus, pictures shown to children in conjunction with words are not

reinforcing the word-level representation but rather distracting from it (Singh & Solman; Solman

& Singh; Samuels). In adults, the written and verbal associations of the word are established and

the processing of words is largely automatic (Guttentag & Haith, 1979). Consequently, pictures

reinforce the target word due to their rich sensory and semantic qualities, but only in the case in

which the pictorial stimuli are beneficial.

However, pictures may not necessarily be beneficial to adults in all conditions. Similar

intrusive effects such as those seen in children learning their native language may be seen in

children and adults learning the vocabulary of a second language. As second languages are often

taught using the same method of pairing vocabulary with pictures and one is establishing new

and sometimes unfamiliar orthographic-phonologic connections, intrusion effects of pictures

may be very likely.

In regard to limitations of the current study, participants were directly instructed to attend

to the words and told they were to be subsequently tested on the words. A different pattern may

have been observed if instructions were changed such as to emphasize the importance of pictures

or to present a surprise memory task rather than inform participants that words would be the

focus of testing. Additionally, in regard to semantic relatedness, different types of relationships

were not examined such as differences between associative relationships verses semantic

relationships (e.g., see Locker, Simpson, & Yates, 2003). Lastly, task conditions and contextual

variations (e.g., contrast changes between the print and background, size of picture and size of

font) may play a large role in determining the potential intrusion effects of pictures on word

processing (Arieh & Algom, 2002).

Generally, the results of this study are indicative of an ability of participants to

focus attention primarily on word-level information (and thus an orthographic-phonologic

primacy) and strategically utilize pictures only when it is beneficial (i.e., when the aim is verbal

memory performance). Although pictures have highly superior encoding properties (Nelson et

al., 1976; Mintzer & Snodgrass, 1999; Nelson & Reed, 1976; Stenberg et al., 1995; Stenberg,

2006), the advantages of pictures do not detract from the encoding of words in conditions in

which the pictures are incongruent and thus conflict with the target. Pictures, were however

beneficial in matching and related conditions. I have argued in the present study that this benefit

may be primarily semantic in nature. The present results do lend support for the already wide use

of pictorial stimuli to reinforce and facilitate learning and comprehension of words and text in

adult populations. However, potential intrusion effects of pictures on word processing requires

further investigation in different experimental contexts.

REFERENCES

Arieh, Y., & Algom, D. (2002). Processing picture--word stimuli: The contingent nature of

picture and of word superiority. Journal of Experimental Psychology: Learning, Memory,

and Cognition, 28, 221-232.

Carney, R. N., & Levin, J. R. (2002). Pictorial illustrations still improve students' learning from

text. Educational Psychology Review, 14, 5-26.

Caramazza, A., & Costa, A. (2000). The semantic interference effect in the picture-word

interference paradigm: does response set matter? Cognition, 75, B51-B64.

Collins, A. M., & Loftus, E. F. (1975). A Spreading-Activation Theory of Semantic Processing.

Psychological Review, 82, 407-428.

Collins, A. M., & Quillian, M. R. (1969). Retrieval Time from Semantic Memory. Journal of

Verbal Learning and Verbal Behavior, 8, 240-247.

Conley, C. M., Derby, K. M., Roberts-Gwinn, M., Weber, K. P., & McLaughlin, T. F. (2004).

An analysis of initial acquisition and maintenance of sight words following picture

matching and copy, cover, and compare teaching methods. Journal of Applied Behavior

Analysis, 37, 339-350.

Costa, A., Alario, F.-X., & Caramazza, A. (2005). On the categorical nature of the semantic

interference effect in the picture-word interference paradigm. Psychonomic Bulletin &

Review, 12, 125-131.

Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing. A framework for memory

research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.

Damian, M. L., and Bowers, J. S. (2003). Locus of semantic interference in picture-word

interference tasks. Psychonomic Bulletin & Review, 10, 111-117.

Glenberg, A. M., & Langston, W. E. (1992). Comprehension of illustrated text: Pictures help to

build mental models. Journal of Memory and Language, 31, 129-151.

Goodman, K. S. (1965). A linguistic study of cues and miscues in reading. Elementary English,

42, 639-643.

Guttentag, R. E., & Haith, M. M. (1979). A developmental study of automatic word processing

in a picture classification task. Child Development, 50, 894-896.

Israel, L., & Schacter, D. L. (1997). Pictorial encoding reduces false recognition of semantic

associates. Psychonomic Bulletin & Review, 4, 577-581.

Jenkins, J. R., Neale, D. C., & Deno, S. L. (1967). Differential memory For Picture and word

stimuli. Journal of Educational Psychology, 58, 303-307.

Kiss, G. R., Armstrong, C., Milroy, R., & Piper, J. (1973) An associative thesaurus of English

and its computer analysis. In Aitken, A.J., Bailey, R.W. and Hamilton-Smith, N. (Eds.),

The Computer and Literary Studies. Edinburgh: University Press.

Kiss, G.R., Armstrong, G., & Milroy, R., The Associative Thesaurus of English. Edinburgh

(1972).

Levin, J. R. (1986). Four cognitive principles of learning-strategy instruction. Educational

Psychologist, 21, 3-17.

Locker, L., Simpson, G. B., & Yates, M. (2003). Semantic Neighborhood Effects on the

Recognition of Ambiguous Words. Memory & Cognition, 31, 505-515

Lupker, S. J., & Katz, A. N. (1982). Can automatic picture processing influence word

judgments? Journal of Experimental Psychology: Learning, Memory, and Cognition,

8, 418-434.

Mahon, B.Z., Costa, A., Peterson, R., Vargas, K.A., & Caramazza, A. (2007). Lexical selection

is not by competition: A reinterpretation of semantic interference and facilitation

effects in the picture-word interference paradigm. Journal of Experimental Psychology,

33, 503-535.

Mintzer, M. Z., & Snodgrass, J. G. (1999). The picture superiority effect: Support for the

distinctiveness model. American Journal of Psychology, 112, 113-146.

Nelson, D. L., & Reed, V. S. (1976). On the nature of pictorial encoding: A levels-of-processing

analysis. Journal of Experimental Psychology: Human Learning and Memory, 2, 49-57.

Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (1998). The University of South Florida word

association, rhyme, and word fragment norms. http://www.usf.edu/FreeAssociation/.

Nelson, D. L., Reed, V. S., & Walling, J. R. (1976). Pictorial superiority effect. Journal of

Experimental Psychology: Human Learning and Memory, 2, 523-528.

Paivio, A. (1969). Mental imagery in associative learning and memory. Psychological Review,

76, 241-263.

Paivio, A. (1971). Imagery and verbal processes . Oxford, England: Holt, Rinehart & Winston

xi, pp. 596.

Pellegrino, J. W., Siegel, A. W., & Dhawan, M. (1976). Short-term retention of pictures and

words as a function of type of distraction and length of delay interval. Memory &

Cognition, 4, 11-15.

Pexman, P. M., & Lupker, S. J. (1999). Ambiguity and Visual Word Recognition: Can Feedback

Explain Both Homophone and Polysemy Effects? Canadian Journal of Experimental

Psychology, 53, 323-334.

Roediger, H. L., & McDermott, K. B. (1995). Creating false memories: Remembering words not

presented in lists. Journal of Experimental Psychology: Learning, Memory, and

Cognition, 21, 803-814.

Rosinski, R. R. (1977). Picture-word interference is semantically based. Child Development,

48, 643-647.

Rosinski, R. R., Golinkoff, R. M., & Kukish, K. S (1975). Automatic Semantic Processing in a

Picture-Word Interference Task. Child Development, 46, 247-253.

Rossion, B., & Pourtois, G. (2004). Revisiting Snodgrass and Vanderwart's object set: The role

of surface detail in basic-level object recognition. Perception, 33, 217-236.

Sadoski, M. (2005). A dual coding view of vocabulary learning. Reading & Writing Quarterly:

Overcoming Learning Difficulties, 21, 221-238.

Samuels, S. J. (1967). Attentional process in reading: The effect of pictures on the acquisition of

reading responses. Journal of Educational Psychology, 58, 337-342.

Seidenberg, M. S., (2005). Connectionist Models of Word Reading. Current Directions in

Psychological Science, 14, 238-242.

Seidenberg, M. S., & McClelland, J. L., (1989). A Distributed, Developmental Model of Word

Recognition and Naming. Psychological Review. 96, 523-568.

Singer, H., Samuels, S. J., & Spiroff, J. (1974). The effect of pictures and contextual conditions

on learning responses to printed words. Reading Research Quarterly, 9, 555-567.

Singh, N. N., & Solman, R. T. (1990). A stimulus control analysis of the picture-word problem

in children who are mentally retarded: The blocking effect. Journal of Applied Behavior

Analysis, 23, 525-532.

Smith, M. C., & Magee, L. E. (1980). Tracing the time course of picture-word processing.

Journal of Experimental Psychology: General, 109, 373-392.

Snodgrass, J. G., & McClure, P. (1975). Storage and retrieval properties of dual codes for

pictures and words in recognition memory. Journal of Experimental Psychology: Human

Learning and Memory, 1, 521-529.

Solman, R. T., Singh, N. N., & Kehoe, E. J. (1992). Pictures block the learning of sightwords.

Educational Psychology, 12, 143-153.

Stenberg, G. (2006). Conceptual and perceptual factors in the picture superiority effect.

European Journal of Cognitive Psychology, 18, 813-847.

Stenberg, G., Radeborg, K., & Hedman, L. R. (1995). The picture superiority effect in a cross-

modality recognition task. Memory & Cognition, 23, 425-441.

Vitkovitch, M., & Tyrrell, L. (1999). The effects of distractor words on naming pictures at the

subordinate level. The Quarterly Journal of Experimental Psychology A: Human

Experimental Psychology, 52, 905-926.

Waddill, P. J., & McDaniel, M. A. (1992). Pictorial enhancement of text memory: Limitations

imposed by picture type and comprehension skill. Memory & Cognition, 20, 472-482.

Waddill, P. J., McDaniel, M. A., & Einstein, G. O. (1988). Illustrations as adjuncts to prose: A

text-appropriate processing approach. Journal of Educational Psychology, 80, 457-

464.

APPENDIX A

SAMPLE STIMULI EXPERIMENT 1

Matching Stimuli

Neutral Stimuli

Incongruent/unmatching Stimuli

APPENDIX B

SAMPLE STIMULI EXPERIMENT 2

Semantically Related Stimuli

Neutral Stimuli

Incongruent/unmatching Stimuli