Modality congruence, multiple resource theory and intermedia broadcast comparisons: an...

Relevant research literature and theoretical bases on the effects of message congruence among multiple stimulus modalities are reviewed to demonstrate the importance of modality congruence to the subject of comparisons between TV and radio broadcast media, in part in response to a comment article

A recent comment (Leigh 1991) on research that concerns intermedia comparisons outlined how Multiple Resource Theory (MRT) by Wickens (1984) can be used to understand more fully the conflicting findings in past research and to provide a theoretical framework for guiding future research efforts. The article was stimulated in response to Buchholz and Smith (1991) which appeared in this journal. MRT is concerned with the manner in which stimulus properties of tasks, the priorities or strategies adopted by an individual, and the processing operations performed on each task may influence overall and individual task performance. Interference among tasks will tend to result if common processing resources are required for successful completion of both tasks, depending in part on the difficulty of the tasks. Leigh (1991) maintains that in a broadcast advertising setting, the primary sources of possible interference are the stimulus information (e.g., limited message congruence among audio and video modalities, message complexity), the individual (e.g., motivation to process brand information), and the broader exposure environment (e.g., competing distractions, other activities being performed) (pp. 72-73). Likely effects of modality congruence and individual motivation were detailed in relation to TV and radio media.

In reviewing the research on intermedia comparisons, Leigh (1991) noted that the between-study variability in results could, for the most part, be attributable to across-study differences in the extent to which the audio and video components of the TV test commercials provided highly-related information. Smith and Buchholz (1991) [S&B] took issue with this "congruency approach" and offered an "involvement" interpretation of MRT and the studies investigating broadcast comparisons. The tone and perspective conveyed in their comment article are perplexing in that both modality congruence and involvement/motivation to process brand information weighed heavily in the ideas and propositions offered by Leigh (1991). And as Garner (1970) noted, |too often has the nature of the [stimulus] input been ignored with the consequence of incorrect assessment of the nature of information processing at worst, or an inadequate picture at best' (p. 350). Both are clearly necessary considerations. Moreover, MacInnis and Jaworski (1989) depict ad and exposure environment factors as affecting the individual's ability or opportunity to process information, both of which are posited to moderate the relationship between motivation and processing.

Perhaps S&B -- and other readers as well -- were skeptical of the importance of message congruency, given that space constraints at the time precluded presentation of relevant literature. As a result, the primary purpose of the present work is to provide related theoretical bases and discuss the research literature germane to the issue of message congruency among modalities to demonstrate its importance to the study of intermedia comparisons. A second purpose is to provide additional discussion on the conduct of research on the topic of broadcast media comparisons. Before these topics are addressed, attention is directed to commenting on several claims made in S&B.

Prefatory Comments and Reactions

Multiple Resource Theory

S&B depicted MRT as being a well-developed and research-based theory of how various tasks may require separate or common resources for performance. One gets the impression from reading their description that the theory is far more exact and substantiated than does Wickens (1984, p. 88, 91). It is indeed a useful heuristic tool for considering the likely impacts of various types of information, but it is not without holes and conflicting findings.

Two statements were made by S&B regarding MRT that are not apparently correct, based on Wickens (1984). First, they mentioned |research indicates that perceptual encoding and central processing use common resources while response processing uses separate resources' (p. 2); however, Wickens (1984, p. 88) cites the classic Brooks (1968) article that calls this assumption into question. Second, S&B (p. 2) note that research suggests perceptual encoding of audio and visual stimuli rely on separate resources (and thus would not be subject to interference). Wickens (1984, p. 87) cites several studies that clearly demonstrate this premise is not entirely tenable; considerable cross-task interference has been observed across auditory and visual encoding conditions that should have utilized separate resources. This discrepancy is particularly important because a number of subsequent statements and interpretations made by S&B are based on this questionable premise. Given that audio and visual encoding tasks can and do interfere on occasion, the issue of which, and under what circumstances, stimulus properties, task conditions, and individual factors serve to limit or accentuate such interference is an extremely important consideration, as was maintained by Leigh (1991).

Finally, S&B (p. 3) credit Wickens (1984) with stating that more attentional resources will be allocated to ad processing as opposed to other goals, if the consumer is highly involved compared to uninvolved. In all likelihood this was an inadvertent slip-up in that Wickens never made reference to either involvement or ad processing in his paper, and there are a number of advertising authors, some of whom S&B cited, that could easily have been used instead.

Other Remarks

In developing their involvement response to Leigh (1991), S&B used several conventions and approaches that tended to distort the substance of Leigh (1991) and, to a lesser extent, Edell and Keller (1989). First, S&B generally made reference to Leigh's ideas as involving congruent versus incongruent audio-visual stimuli; in fact, Leigh (1991) did not use "incongruent" and instead used less extreme terminology. It is certainly parsimonious to utilize the extremes of a continuum; however, advertisers are unlikely to intentionally develop a commercial in which the audio and video portions have nothing to do with one another. Second, in several instances, statements Leigh (1991) made were transformed to seem deterministic as opposed to probabilistic and speculative, which they invariably are (e.g., changed "might" or "probably would" to "would". Moreover, even though Edell and Keller (1989, p. 159-160) acknowledged that their ad manipulations may not have been strong enough from an inference standpoint to produce the hypothesized effects, S&B (p.4) keyed on the significant (p<.05) pretest results that Edell and Keller reported to downplay the possible influence of modality congruency. Furthermore, S&B focused on selected results of that study and did not address the fact that TV produced not only higher levels of brand and brand claim recall compared to radio, but TV also had significantly higher levels of cognitive response on all measures, including brand and total evaluative thoughts, and a very low level of recall of audio elements. Taken together, the additional results are not supportive of the conclusions they reached that visual and auditory encoding tasks rely on separate resources.

Modality Congruence: Theoretical Considerations and Research Support

Demonstration that message congruence among audio and visual modalities is an important basis for understanding differences between TV and radio will first entail the description of the manner in which stimuli of various types are processed, followed by a discussion of the important role of organization, including modality congruence, to that processing. Research support for the premises are then given.

Perceptual Processing

At the core of perceptual processing is the concept of the schema, or cognitive structure. Neisser (1976) defines a schema as being integrally involved in what he conceptualizes as the perceptual cycle:

A schema is that portion of the entire perceptual

cycle which is internal to the perceiver, modifiable

by experience, and somehow specific to what is

perceived. The schema accepts information as it

becomes available at sensory surfaces and is

changed by that information; it directs movements

and exploratory activities that make more information

available, by which it is further modified.

(p. 54)

This definition is congruent with Norman's (1976, p. 41) ideas that processing may at times be conceptually-driven top-down), with expectations based on a current schema guiding what is perceived, and at other times be data-driven (bottom-up) by the stimulus information. The task of perception is to gather sensory information together and interpret it in terms of a coherent framework, which may be done in certain instances through developing a new schema, in others by selecting an appropriate one from memory, and in others by reorganizing the schema currently operative. The problem is to determine the appropriate one to use (Norman 1976, p.73).

There are several important characteristics and implications of schemata that are relevant to processing of stimuli: 1. Stimulus information must be related to a developing or active schema if it is to be interpreted correctly and coherently. Other stimulus information will be ignored or will lead to incorrect or meaningless outcomes (Neisser 1976, p. 55; Mandler 1985, p. 38). 2. Schemata are general perceptual tools and are not modality-specific: "To attend to an event means to seek and accept every sort of information about it, regardless of modality, and to integrate all the information as it becomes available' (Neisser 1976, p. 29). 3. The activities directed by two schemata at one time can conflict or even be entirely incompatible; selective attention typically results, whereby one schema and its associated informational stimuli and activities on these stimuli are given attention and not the other. It is easy to focus on one thing and screen out others due to selective focusing, but it is quite difficult to focus on two different things at once, even when both are simple in content (Neisser 1976, p. 102). 4. Interference effects tend to occur when one tries to use the same perceptual schema for two incompatible purposes (Neisser 1976, p. 100). Intersensory bias, which is a phenomenon that results when the stimulus information to different sensory modalities is discrepant (Welch and Warren 1980), is one such instance. The reason bias results is due to the perceiver's attempt to use the same schema for both modalities, based on his or her assumption that the sensory information relates to a single, unitary event (i.e., the assumption of unity).

Points 3 and 4 are particularly relevant to television advertising, with both audio and video providing stimulus input. To process this information, either two different schemata and associated perceptual cycles must be concurrently employed, or only one modality becomes an active part of a perceptual cycle and the other modality is not monitored, or the two sources of information must be merged into one schema, potentially leading to intersensory bias.

The Role of Organization

A schema is an organized structure that is used as a guide for acquiring new information and for assembling the results. If the stimuli already possess a meaningful organization or correlated structure, the information is perceived faster and better (Garner 1974), processed to a deeper level (Craik and Lockart 1972; Norman 1976), and thus exhibits a higher level of recall (Mandler 1967) and to a lesser extent, recognition compared to unorganized material (Mandler 1980). If the stimuli do not already have a discernable organization, the perceiver will attempt to formulate one to facilitate perceptual processing and learning (Mandler 1967). However, interference among stimuli (i.e., orthogonal dimensions) may emanate from such activity (Garner 1974). Mnemonic tools, interactive imagery, and grouping are exemplary of organizational vehicles that could reside in the stimulus input or be applied by the perceiver. Similarly, congruence among modalities is a form of stimulus organization.

Two studies (Bellezza, Cheesman and Reddy 1977; Jacoby, Bartz and Evans 1978) are relevant to the issue of whether involvement or organization is a more important consideration in processing. The studies found that high involvement semantic processing (compared to low involvement sensory processing) and an organization strategy (Bellezza, Cheesman and Reddy 1977) or stimulus organization Jacoby, Bartz and Evans 1978) contribute to higher recall levels. Thus it appears based on these two studies that both are important. Supportive of this premise are the studies comparing experts and novices at the game of chess in reproducing given layouts of game pieces; experts are able to reproduce an organized layout much better than novices, whereas there are no differences if randomly-placed pieces characterized the setup (Neisser 1976, pp. 179-182). Due to their extensive experience, experts will attempt to engage in high-involvement semantic processing of a layout and will pick up information quickly and effectively if their wealth of schemata based on experience can be used. Novices are far more likely to simply focus on the sensory details due to their not having associated schemata in memory to be of assistance.

Message Congruence Among Modalities

Based on the research results of the effects of stimulus organization, it should be expected that highly-related congruent audio and video stimuli will exhibit comparable positive effects on processing and memory, compared to the cases where the audio and video provide weakly-related or even different information. A number of research studies have been conducted and conceptual articles written on the subject, primarily in the cognitive psychology area, that provide findings and theories which can be used to assess the merit of this premise and offer explanations and delimitations likely operative.

Several prefactory remarks need to be made before delving into the subject. First, the idea of message congruence among modalities refers to the extent to which two sensory channels communicate the same information, as in watching (visual) and listening (auditory) to a person speak, or in an announcer describing either the product name, attributes or actions shown in the video portion of a TV commercial. Low levels of congruence would be exhibited for instance through the use of music as the audio track in a commercial that does not enhance the association with the brand shown visually. Second, it is important that modality congruence not be confused with the concept of claim or theme congruency/incongruency investigated by Heckler and Childers (1992) and others; claim congruency is concerned with the congruence between one's expectations or schemata based on prior experience and external information (e.g., provided by audio or video), and thus is not concerned with the role of congruence on resource allocation among sensory inputs. Third, several different terms have been used in the relevant literature that are virtually synonymous with that of congruence -- redundancy (Drew and Grimes 1987; Gamer 1974), coordinated (Yee, Hunt and Pellegrino 1991), and correlated (Garner 1974). It appears that congruence has been used more frequently, but perhaps a more appropriate term can ultimately be offered for the advertising context. Finally, a number of relevant studies have investigated modality-congruence matters. In the interest of conserving space, only recent or significant articles on a separate topic will be addressed, which will allow the reader to consult those articles to locate older works already in print.

Several studies have been conducted on the perception of speech that concerned modality congruency. It has long been known that in a noisy environment, perception of the speaker improves dramatically if the listener can see the speaker (Neisser 1976, p. 160). In a study of adolescents, Dodd (1977) examined the extent to which they could correctly perceive spoken words under five different conditions -- the auditory and visual information were synchronous, out of synchrony, auditory input alone, visual input alone, or the auditory information was different from the visual information seen. Results indicated that across all measures of correct and incorrect perception, the auditory-visual congruent condition produced superior perception to the out of synchrony and single-channel conditions, which were in turn superior to the incongruent information condition.

Neisser (1976, p. 161) maintains that auditory information is more informative about speech events to a receiver than is visual information. Easton and Basala (1982) examined whether discrepant auditory information would affect an observer's ability to lipread a speaker. They found that the visual information did not significantly affect recognition of auditory speech; however, discrepant auditory information substantially interfered with recognition of visual speech. The extent of the auditory bias was related to the magnitude of the discrepancy between the visual and auditory input and the compellingness of the two modalities as originating from a single source (e.g., the same person is heard and seen versus a male is heard and a female is seen). The study provides support for our tendency to emphasize auditory input for speech perception, in spite of a normal visual dominance in human beings, and to use visual stimulation as supportive information. In addition, it supports the idea that perceivers will be subject to intersensory bias if the auditory and visual stimulation can be assumed to be unitary.

Massaro and Cohen (1983) examined the extent to which congruent or discrepant visual information affects correct recognition of auditory information. Even though experimental subjects were instructed to base their decision on what they heard, visual information significantly contributed to identification of the auditory information. They also found that the visual information has a greater impact if the auditory information is relatively ambiguous, thereby implying that the less ambiguous modality is relied upon more heavily. The results are consistent with a fuzzy logical model of perception (FLMP), which carries a critical assumption that auditory and visual information are evaluated separately and then combined to achieve perceptual recognition.

Green et al. (1991) also focused on the provision of discrepant information by the auditory and visual modalities and examined a phenomenon known as the McGurk effect, which is where discrepant phonetic information presented by the auditory and visual modalities is combined by perceivers into a new, but incorrect percept, rather than each modality's information being viewed as distinct and separate. Green et al. found that the McGurk effect held irrespective of whether gender-congruent or gender-incongruent presenters of the auditory and visual channels were used. These findings suggest that the assumption of unity (i.e., compellingness) is not a necessary prerequisite for a perceiver integrating auditory and visual information.

Melara and Marks (1990) investigated the effect of congruency among linguistic and nonlinguistic stimulus elements that were either both presented visually, both auditorially, or having one visual and the other auditory. Semantic processing was believed to be necessary for successful task performance. Results across the six experiments clearly demonstrate that congruence among the two types of stimuli leads to much faster and effective processing than either orthogonal dimensions, which resulted in interference, or, to a lesser extent, negatively-correlated dimensions. These results are consistent with Green et al. (1991) and the notion that with uncorrelated modalities, two different schemata must be used for sensory perception initially, which are then combined into a new percept.

Yee, Hunt and Pellegrino (1991) tested various aspects of Wicken's (1984) MRT by examining whether the ability to deal with information from different sources in a coordinated manner differs from the ability to deal with information from each source alone. Four experiments examined coordination of a verbal task with a perceptual visual-spatial or auditory-spatial task. Results of the experiments consistently demonstrated that even though it is more difficult to execute a perceptual-verbal coordinating task than to do either separately, the congruence afforded through a coordinating task tended to reduce overall task difficulty as the difficulty of the two separate tasks increased; they refer to this phenomenon as the compression effect. Based on Wickens (1984), two tasks involving verbal and spatial codes should rely on different resources and not exhibit compression.

As a refinement of Wickens (1984), Yee, Hunt and Pellegrino (1991) propose a model of coordination involving three stages that accounts for compression. In the first stage, internal representations of each type of stimulus information are formed using codes appropriate for that channel (stage is not highly demanding of common resources unless the same code is used). In the second stage, the two representations are converted into a common metric of verbal propositional representation (stage is demanding of common resources). In the third stage, the two propositional representations are compared and a response is selected. The model does not require that stage one is completed on both sources of stimulus information before moving on to the second; however, common verbal resources are required for conversion of spatial information to a verbal propositional representation, and such a situation could slow stage one processing of verbal information. The model suggests different outcomes of coordination or integration depending on overlap among modalities. It was formed to deal with stimulus inputs involving different codes. In all likelihood there will be a need for similar frameworks to account for other areas of MRT that need refinement.

Apart from Edell and Keller (1989), only two studies (Bither and Wright 1973; Drew and Grimes 1987) explicitly investigated effects of message congruence across modalities of television communications. Several other studies have examined message congruence in a print ad setting, but the discussion here will be limited to televised comparisons. Bither and Wright (1973) employed four experimental ad treatments -- (1) minimal visual distraction, where the audio and visuals were congruent, (2) mild visual distraction, where visual scenes were irrelevant to the message, but only mildly diverting, (3) strong visual distraction, where the visual scenes were irrelevant and highly diverting, and (4) audio distraction, where a person with a foreign accent was used, but it was congruent with the visuals. The results indicated that free recall of message content for the minimal distraction treatment was greater than the mild distraction and audio distraction treatments, which were in turn much greater than the strong visual distraction treatment. Essentially the reverse pattern was found for the reported difficulty in reception of the message.

Drew and Grimes (1987) focused on TV news stories and investigated the effects of redundancy among the audio and video channels on recognition (and not on recall as they claim). Fourteen news stories were shown to respondents in one of five experimental conditions -- high redundancy, where all fourteen stories had redundant audio and video; medium, where half had redundant material and in the other half the visuals did not relate directly to the story; low, where none of the stories had redundant inputs; audio only and video only. Across the redundancy groups there was a linear positive relationship between redundancy and audio recognition. However, for video recognition, the high redundancy group had a lower level of correct recognition than the other redundancy groups. Story understanding results were essentially the same as those for audio recognition. These results are comparable to those of Easton and Basala (1982) and of Massaro and Cohen (1983). If congruent/redundant information is provided in both modalities, viewers will pay attention to the modality that is most informative. However, when the modalities are not congruent, viewers tend to emphasize video at the expense of audio. Results for the audio-only and video-only treatments tended to be comparable to the low redundancy groups in terms of story understanding and respective audio and video recognition.

Taken together, the findings of the studies reviewed provide a strong degree of support for the important role which modality congruence has in the concurrent processing of information from auditory and visual stimuli. The psychology studies reveal that, even though the basic framework and general tenets of MRT may be reasonable, the specific resource-related conditions contributing to interference do not adhere to MRT across the board.

Several of the articles have advanced important refinements and extensions to MRT, as in Massaro and Cohen's (1983) FLMP and the coordinated model by Yee, Hunt and Pellegrino (1991); both of these models specify how even stimulus input of different codes and from two different modalities may be subject to interference due to the necessary conversion and integration process that must ensue. In the case of combining information across modalities of a normal TV commercial, however, both the visual and auditory stimulus information will typically involve initial encoding using a verbal code and thus be demanding of common resources throughout the first two stages of the perceptual process. Nevertheless, some commercials are predominantly spatial visually and verbal auditorially, e.g., a commercial for a sports car that depicts the car being driven on a winding road at a high rate of speed as the announcer conveys important attributes and benefits. This type of commercial corresponds directly to the refinements of MRT. Other commercials involve spatial and verbal codes in both modalities; these types of commercials are clearly outside the bounds of existing theory or research knowledge. What is known is that modality congruence greatly facilitates the processing task for the viewer and effectively limits the extent to which interference will likely occur.

In addition to the findings regarding effects of modality congruence, some research studies also revealed that individuals will tend to focus on one modality more heavily than another, as in auditory information being the primary input for the perception of speech and visual information serving a supportive role. In making predictions concerning the processing of stimulus information from several modalities, it is also imperative that consideration be given to the relative appropriateness of each particular modality for the task at hand (Welch and Warren 1980). Such consideration helps to predict effects resulting from a greater focus on the audio or video.

For application in broadcast advertising, a major limiting facet of the theorizing and the research studies conducted to date is that they tend to restrict focus, implicitly or explicitly, to discrete decision tasks. None of them addresses what will occur if, during exposure and processing of a dynamic audio-video stimulus presentation, something triggers elaborative thinking activities which interfere with the encoding of subsequent portions of the message. Does this elaborative activity only impact those aspects that are based on a verbal code? Can subsequent elements be recognized and perhaps recalled? These and other issues merit investigation.

Suggestions for Research

The research questions and issues offered by S&B were for the most part reasonable, although some of the specific research questions are premature. This type of research should be postponed until a number of the initial research questions concerned with modality effects are addressed and a body of knowledge accumulated on these effects. It was recommended that strong-inference manipulations of modality (TV versus radio) be initially employed using content-redundant and congruent messages, and multiple ad replicates across TV and radio to investigate the general substance of MRT (Leigh 1991). Once the general effects have been established, efforts should then move to examining factors that lead to interference and bias among modalities.

One recommendation offered by Leigh (1991, p. 74) is for future research to incorporate the full complement of memory-related response measures -- unaided and aided recall, and recognition of specific and general ad elements -- and also elicit cognitive response data when studies are conducted in the future. S&B (p.6) questioned the need to assess both recall and recognition in a study, given that several studies had produced consistent results on both sets of measures. Lynch and Srull (1982) outlined several reasons for collecting both recall and recognition data. They supplied a number of references that support the premise that any independent variable that affects recall but not recognition outcomes thus has an influence on the retrieval stage of processing (p. 21). Both modality congruence and motivation to process brand information are factors that fall into this category given that organization is more important for recall than recognition (Lynch and Srull 1982; Mandler 1980), as is motivation to process brand information (Greenwald and Leavitt 1984).

Leigh (1991) recommended collecting cognitive response data for the purpose of evaluating differences among broadcast media and pointed out that there are several different ways in which an individual may elaborate on the content of an ad message apart from elaboration by making personal connections, including such integrative activities Greenwald and Leavitt (1984, p. 588) outlined as associating new information with knowledge already stored and actively supporting or counterarguing with a message. S&B (p.7) questioned whether a "middle level" evaluative process that is "only middle level" is comparable to the highest level of elaborative activity involving personal constructive processes. The issue is not "which is better?" Any elaborative activity signifies that a message generated thinking on the part of the receiver (albeit possibly negative), and such thinking is good from the standpoint of one element of advertising effectiveness. The reason for recommending that all facets of cognitive response be assessed is quite simple: both motivation to process brand information (MacInnis and Jaworski 1989) and audience involvement levels (Greenwald and Leavitt 1984) are hierarchical constructs that pertain to how an audience member deals with a message and the factors contributing to that processing. Across an entire audience, some individuals will not progress beyond the lowest levels; others may reach more midrange levels; and still others may reach the highest levels. It is important, then, to employ measures of cognitive activity that capture various forms of elaborative activity along the continuum. Furthermore, cognitive response measures are based on retrospective and reconstructive retrieval of cognitive activity during exposure (Neisser 1976), and, as such, are subject to individual differences in articulation, understanding of the task, and expectations regarding the researcher's preference for certain types of information. Whereas one person might report responding with personal-connection elaborations, another person might think the researcher wants to know about ad-related support arguments or counterarguments.

In conclusion, it is hoped that in the near future, a number of advertising researchers, new and seasoned alike, will elect to respond to the challenge afforded by research on intermedia broadcast comparisons. The potential contributions are enormous.

References

Bellezza, Francis S., Fred L. Cheesman II, and B. Goverdhan Reddy (1977), "Organization and Semantic Elaboration in Free Recall," Journal of Experimental Psychology: Human Learning and Memory, 3 (September), 539-550. Bither, Stewart W. and Peter L. Wright (1973), "The Self-Confidence -- Advertising Response Relationship: A Function of Situational Distraction," Journal of Marketing Research, 10 (May), 146-152. Brooks, Lee R. (1968), "Spatial and Verbal Components of the Act of Recall,' Canadian Journal of Psychology, 22 (October), 349-368. Buchholz, Laura M. and Robert E. Smith (1991), "The Role of Consumer Involvement in Determining Cognitive Response to Broadcast Advertising," Journal of Advertising, 20 (1), 4-17. Craik, Fergus I. M. and Robert S. Lockhart (1972), "Levels of Processing: A Framework for Memory Research," Journal of Verbal Learning and Verbal Behavior, 11 (December), 671-684. Dodd, Barbara (1977), "The Role of Vision in the Perception of Speech," Perception, 6 (1), 31-40. Drew, Dan G. and Thomas Grimes (1987), "Audio-Visual Redundancy and TV News Recall,' Communication Research, 14 (August), 452-461. Easton, Randolph D. and Marylu Basala (1982), "Perceptual Dominance During Lipreading," Perception & Psychophysics, 32 (December), 562-570. Gamer, W. R. (1970), "The Stimulus in Information Processing," American Psychologist, 25 (April), 350-358. Garner, Wendell R. (1974), The Processing of Information and Structure, Potomac, MD: Erlbaum. Green, Kerry P., Patricia K. Kuhl, Andrew N. Meltzoff and Erica B. Stevens (1991), "Integrating Speech Information Across Talkers, Gender, and Sensory Modality: Female Faces and Male Voices in the McGurk Effect," Perception Psychophysics, 50 (December), 524-536. Greenwald, Anthony G. and Clark Leavitt (1984), "Audience Involvement in Advertising: Four Levels," Journal of Consumer Research, 11 (June), 581-592. Heckler, Susan E. and Terry L. Childers (1992), The Role of Expectancy and Relevancy in Memory for Verbal and Visual Information: What is Incongruency?" Journal of Consumer Research, 18 (March), 475-492. Jacoby, Larry L., Wayne H. Bartz and James D. Evans (1978), "A Functional Approach to Levels of Processing," Journal of Experimental Psychology: Human Learning and Memory, 4 (July), 331-346. Leigh, James H. (1991), "Information Processing Differences Among Broadcast Media: Review and Suggestions for Research," Journal of Advertising, 20 (June), 71-75. Lynch, John G., Jr. and Thomas K. Srull (1982), "Memory and Attentional Factors in Consumer Choice: Concepts and Research Methods," Journal of Consumer Research, 9 (June), 18-37. MacInnis, Deborah J. and Bernard J. Jaworski (1989), |Information Processing from Advertisements: Toward an Integrative Framework," Journal of Marketing, 53 (October), 1-23. Mandler, George (1967), |Organization and Memory," in The Psychology of Learning and Motivation, Kenneth W. Spence and Janet Taylor Spence, eds., New York: Academic Press, 327-372. Mandler, George (1980), "Recognizing: The Judgment of Previous Occurrence," Psychological Review, 87 (May), 252-271. --(1985), Cognitive Psychology: An Essay in Cognitive Science, Hillsdale, NJ: Erlbaum. Massaro, Dominic and Michael M. Cohen (1983), |Evaluation and Integration of Visual and Auditory Information in Speech Perception," Journal of Experimental Psychology: Human Perception and Performance, 9 (October), 753-771. Melara, Robert D. and Lawrence E. Marks (1990), "Processes Underlying Dimensional Interactions: Correspondences Between Linguistic and Nonlinguistic Dimensions," Memory & Cognition, 18 (September), 477-495. Neisser, Ulric (1976), Cognition and Reality. Principles and Implications of Cognitive Psychology, San Francisco: W. H. Freeman. Norman, Donald A. (1976), Memory and Attention: An Introduction to Human Information Processing, Second Edition, New York: Wiley. Smith, Robert E. and Laura M. Buchholz (1991), "Multiple Resource Theory and Consumer Processing of Broadcast Advertising: An Involvement Perspective," Journal of Advertising, 20 (September), 1-7. Welch, Robert B. and David H. Warren (1980), "Immediate Perceptual Response to Intersensory Discrepancy," Psychological Bulletin, 88 (November), 638-667. Wickens, Christopher D. (1984), "Processing Resources in Attention," in Varities of Attention, Raja Parasuraman and D. R. Davies, eds., New York: Academic Press, 63-102. Yee, Penny L., Earl Hunt, and James W. Pellegrino (1991), "Coordinating Cognitive Information: Task Effects and Individual Differences in Integrating Information from Several Sources," Cognitive Psychology, 23 (October), 615-680.