Abstract
Previous studies of face-to-face (f2f) communication have suggested that speakers rely heavily on a variety of multi-modal cues to make real-time predictions about upcoming words in rapid turn-taking. To understand how computer-mediated communication (CMC) differs from f2f communication in terms of the prediction mechanism, this study assessed how the loss of multi-modal cues would affect word predictability in turn-taking. Participants watched videos, listened to audio, or read a transcript of f2f conversations. Across these three conditions, they predicted the same set of omitted words with different levels of predictability and semantic relatedness to other words in the context. Results showed that words of higher predictability were more accurately predicted regardless of CMC types. Higher response accuracy but longer response time were observed in conditions with richer cues, and for participants with more positive and less negative self-emotions. Meanwhile, semantic relatedness did not affect predictability. These results confirmed the key role of prediction in language processing and conversation smoothness, especially its importance in CMC.
Introduction
Effective communication depends on the predictability of conversational dynamics, enabling smooth turn-taking and facilitating the overall flow of communication. Studies on face-to-face (f2f) communication have highlighted that speakers rely on a range of verbal and non-verbal cues to make real-time predictions about upcoming words. However, computer-mediated conversation (CMC) introduces distinctive challenges stemming from the variations in the availability and effectiveness of these cues across diverse communication channels, encompassing video, audio, and text- only. In CMC, the absence or limited presence of these cues impairs the prediction process and frequently engenders disruption in coherence and fluidity in speech flow.
More notably, turn-taking fundamentally depends on cooperative information sharing and common ground. The degree of interpersonal alignment is subject to the level of intentionality, the specific communicative goals and tasks at hand, attention and affective states and emotion. These sociopsychological aspects are intertwined to affect the prediction process, thus influencing overall speech comprehension in conversational turns.
Despite extensive research on the implications of predictability for turn-taking in f2f communication, a comprehensive understanding of the nuanced variations of predictability across different communication channels is limited. The inherent disparities regarding the availability and effectiveness of verbal and non-verbal cues in modalities characteristic of CMC engender a gap in knowledge regarding the prediction and grounding mechanisms in these contexts.
As such, this study aims to fill this gap and address the open topic of how the predictability of incoming material impacts the dynamic of turn-taking in CMC as opposed to f2f communication. Additionally, this research aims to shed light on the intricate relationship between predictability and turn-taking interruptions in CMC.
To achieve these objectives, we compared predictability in video, audio, and text formats in a behavioral study in which participants were asked to complete a word prediction task with a prerecorded conversation. Accuracy, semantic relatedness, and time of responses were measured.
Our results showed that higher response accuracy, but longer response time was observed in conditions with richer cues. Semantic relatedness or attention did not affect predictability. Participants with more positive emotions showed slower responses and those with less negative emotions showed lower response accuracy. We further discussed implications of these findings in language processing and conversation smoothness, and how they can inform the processing mechanisms in CMC with varying availability of multi-modal cues.
Conclusion
In this study, we explored the effect of three CMC types (V, A, T) on word predictability. Our findings indicated that there were more accurate responses for words with higher sequential predictability in all CMC types. Higher RA but longer RT were observed in conditions with richer cues, and for participants with less negative emotions and higher attention level. Semantic relatedness did not affect predictability. These results confirmed the key role of prediction in language processing and conversation smoothness, especially its importance in CMC.
