Effects of Real-time Transcription on Non-native Speaker’s Comprehension in Computer-mediated Communications

By Ying-xin Pan, Dan-ning Jiang, Michael Picheny, Yong Qin

Summary:

In this paper the performance of a non-native speaker's comprehension of communication with the help of real-time transcription was studied. The authors hypothesized that with the help of real-time transcription a user could better understand audio and audio/video communications if a real-time transcription was provided. They also looked to see if providing a transcription history instead of just 'line-by-line' would be more beneficial. The three questions they looked to answer in their study were:

1. Does real-time transcription help non-native speakers improve comprehension in multilingual communication utilizing audio and video conferencing systems?
2. How do users perceive real-time transcription in terms of usefulness, preferences and willingness to use such a feature if provided?
3. How do users allocate their cognitive resources when presented with multiple information sources?

They described their experiment as a 2x3 design, in that they tested two modalities ie audio and audio+video, and three transcription settings ie no transcription, streamed transcription, and stream transcription with history. They had 48 non-native English speaking students which they split up to evenly test their 2x3 design. For the communication examples they used small 3-5min English language clips followed by a timed question and answer session rounded off with a questionnaire.

The authors of this paper were looking to measure three qualities of the experience of the users, these being performance on the questions, confidence in their answers, user experience with the system, and cognitive resource allocation. The results of their study showed that transcription, 'had a significant main effect on both performance (F (2, 92) =11.28, p<.01) and confidence (F (2, 92) =13.69, p<.01)'. They also noted that providing a transcription history didn't really improve performance of the user. They also noted that performance and confidence on transcription was not affected by modality. The authors lastly stated that for possible future work they would like to investigate automated speech recognition '(with the associated imperfections) as a practical alternative to human transcriptionists'.

Discussion:
This paper pretty much demonstrated what I already thought was common knowledge. I think it is widely believed that if a user is listening to a non-native language communication that transcripted text would help their understanding. I mean, isn't that what subtitles are (although subtitles generally aren't real-time transcripted)? I think their study did have some small success to it; I didn't know that adding video to audio would not really aide the understanding and comprehension. I also thought it was interesting that even given the option to scroll transcription history the users did not really use it, the streaming transcription was sufficient.