The Effects Of Facial Cues On Sentence Comprehension For Monolingual And Bilingual Listeners: An ERP Study
Open Access
- Area of Honors:
- Psychology
- Degree:
- Bachelor of Arts
- Document Type:
- Thesis
- Thesis Supervisors:
- Janet Van Hell, Thesis Supervisor
- Kenneth Levy, Honors Advisor
- Keywords:
- Foreign-accented speech
- Monolingual listeners
- Bilingual listeners
- Facial Cues
- Abstract:
- In our growing global society, cross-cultural interactions are increasing which implies that many people are communicating through a common language but with differing accents. Recent research indicates that foreign-accented speech, relative to native-accented speech, affects the neural correlates of sentence comprehension for semantic and grammatical information (Grey &Van Hell, 2017). A question that arises is whether additional cues to speaker identity, such as facial information, change the neural effects found for foreign-accented speech. In a recent sociolinguistic study, McGowan (2015) found that when the face of the speaker coincides with the nature of the accented speech, listener performance on foreign-accented sentence transcription in noise improves. Additionally, recent research utilizing the fMRI technique found that face cues facilitate bilinguals’ picture naming when the socio-cultural identity of the face was congruent with the language participants had to name the pictures in (Li, Yang, Scherf, & Li, 2013). These findings suggest that nonlinguistic cues, and faces in particular, can influence language processing. The current study extends prior work by Grey and Van Hell (2017) on accented speaker identity by adding an additional non-linguistic cue to identity: the faces of the two speakers. Using the event-related potential (ERP) technique, the current project investigated the ERP correlates of semantic and syntactic processing during foreign- and native-accented sentence comprehension in monolingual native listeners and bilingual non-native listeners. Through measurement of the P600 response to pronoun errors and the correct equivalents, grammar processing was assessed. The N400 response was analyzed to index sensitivity to semantic anomalous relative to semantically correct sentences. The study was conducted with two groups of listeners: adult non-native Spanish-English bilinguals tested in Granada, Spain, and adult native English monolinguals tested in Central Pennsylvania. Both groups heard sentences that contained errors in grammar (English subject pronouns) or semantic meaning, or were matched correct sentences. These sentences were spoken by a non-native Chinese-accented English speaker and a native American English-accented speaker. Listeners were not informed about accentedness of the two speakers, but were presented with an Asian face and a Caucasian face at the beginning of the comprehension task. The addition of a face cue to speaker identity influenced the pattern of results for the neural correlates of grammatical and semantic processing. For the English monolingual listeners, pronoun errors elicited a P600 and semantic errors elicited an N400 in the native English-accented condition. In the non-native Chinese-English accented condition, there were no effects for pronoun processing, and an N400 with a delayed onset for semantics. For the Spanish-English bilingual listeners, who listened to these sentences in their second language English, there were no significant effects for pronoun mismatches in either accent condition, suggesting the bilinguals were not sensitive to the grammatical errors in their second language. For semantics, there was an N400-like response to semantic errors for both accent conditions. These patterns suggest that facial information of speaker identity combined with foreign- and native-accented speech has differential effects on monolingual listeners’ and bilingual listeners’ sentence processing.