Hand in Hand With Speech – Gestures Complementing Vocal Communication
Jenny Galvao

Based on research carried out by Emmanuel Biau of the Universitat Pompeu Fabra and associates, so called “beat gestures” such as batoning the hand up and down, have the ability to assist in tuning in brain activity especially at relevant moments during natural speech.

The current research suggests that speech and paralinguistic information may be integrated differently than we once assumed.

“We hypothesized that if spontaneous beats have an effect on the processing of speech, our results may demonstrate an increase of synchronization in the theta frequency activity at the corresponding word onset. If this modulation of theta activity is effectively due to the presence of a preceding reliable visual cue, the increase of synchronization may be observed before the word onset occurs, in order to facilitate its processing,” say the researchers.

In this study, Spanish speakers were selected as participants, and their task was to view speakers uttering word pairs. In one condition beat gesture were used and the other, no gesture was used. The subjects also had audio and visual information available, or only audio information. Each participant got both audio only and audio with visual treatment conditions, but the order in which they got the information was just adjusted (e.g., audio only first, and audio visual second or vice versa). The participants listened, and then filled out a memory questionnaire. EEG signals were recorded.

The results showed that the participants had a high rate of correct responses, indicating that they paid attention. In fact, correct response rates were higher when the audio only information was presented second as opposed to first. Essentially, beat gestures effectively modulated brain oscillatory activity, which can effect early processing of acoustic features; the sight of gestures first led to this effect.

“Our results suggest that gestures might be anchors modulating early stages of speech processing, potentially through attentional mechanisms. Furthermore, this mechanism suggests the generality of low-frequency oscillations as the basis for integration of very different kinds of audio-visual communicative information,” the researchers explain.

The beats were often perceived as communicative gestures rather than just random movements; the gestures are perceived as salient and add meaning to the verbal communication. Nonverbal communication, even when as subtle as a simple gesture, can aid in the communication, comprehension and retention process.

Linguistic relevance is attributed to these beat gestures, and this is just one example of the many ways our nonverbal communication aids us everyday.

Jenny Galvao_smallAbout the Author: Jenny Galvao is an undergraduate student at the University of Guelph studying psychology.

 

 

 

Resource

Biau, Emmanuel; Mireia Torralba; Lluis Fuentemilla; Ruth de Diego Balaguer; and Salvador Soto-Faraco. Speaker’s hand gestures modulate speech perception through phase resetting of ongoing neural oscillations. Cortex Journal. 2015. 1-10. doi: http://dx.doi.org/10.1016/j.cortex.2014.11.018