Multisemiotic and multimodal corpus analysis: an application to audio description
The aim of this paper is to investigate a method to apply the traditional mechanisms of corpus linguistic analysis to multimodal corpus linguistics (Baldry & Thibault (2004). Specifically, the first part of this paper will be concerned with the transformation that corpus linguistics has undergone over the past decades in the research and analysis of audio description, as well as discuss the compatibility of corpus-based methods with the research questions posed in text linguistics and its different subfields (coherence, cohesion…).
In the second part of the paper, we will propose a way to integrate the analysis of the different recurrent semiotic patterns that are present in an audio described film. This attempt includes, on the one hand, the formulation and isolation of appropriate accessible visual units of analysis in order to be able to annotate them and, on the other, to link these units to the so called language-only-texts, naturally related by the audio describer in the audio description script.
Although our collected corpus includes language-only-text as the very basis of tagging techniques and methodology, we relate this linguistic information to the grammar of the visual semiosis codified in film scenes, as well as the other semiotic structure of the film, namely its narratology.
Following Salway (2007) and Ädel & Reppen (2008) we do believe that there is a way to consider the possibility of tagging a multimodal corpus and extracting different types of information.
Researchers concerned with discourse phenomena have always been searching for solutions to include the context and co-text interpretation in their analysis. Audio description provides an excellent tool to start with it.
Projects involved: TRACCE (Code: SEJ2006-01829/PSIC) and AMATRA (Code: T07-SEJ-2660) research projects (funded by the Spanish Science and Education Ministry and the Andalusian Government, respectively) analyze the specific resource of the audio description script in a corpus of 200 scripts from AD films.
References
Ädel, Annelie and Randi Reppen (2008): Corpora and Discourse. The challenges of different settings. Amsterdam: John Benjamins.
Alwood, Jens. (2008) “Multimodal corpora”. In Lüdeling, Anke & Merja Kytö (eds.), Corpus Linguistics. An international Handbook. Berlin, New York: Walter de Gruyter, 207-225
McNamara, Danielle.S., & the CSEP lab (2003). “Coh-Metrix: Automated cohesion and coherence scores to predict text readability and facilitate comprehension.” Annual project report submitted to the Institute of Education Sciencesto predict text readability and facilitate comprehension. In http://csep.psyc.memphis.edu/mcnamara/vita.htm
Nivre, Joakim. (2008) “Treebanks”. In Lüdeling, Anke & Merja Kytö (eds.), Corpus Linguistics. An international Handbook. Berlin, New York: Walter de Gruyter, 225-241
Salway, Andrew. (2007), ‘A Corpus-based Analysis of Audio Description’, in Díaz Cintas, Orero and Remail (eds.), Media for All: Subtitling for the Deaf, Audio Description and Sign Language, Amsterdam – New York, Rodopi, pp. 151-174 Salway, A. (2007), ‘A Corpus-based Analysis of Audio Description’, in Díaz Cintas, Orero and Remail (eds.), Media for All: Subtitling for the Deaf, Audio Description and Sign Language, Amsterdam, New York: Rodopi, 151-174
Taboada, Maite., Kimberly Voll and Julian Brooke (2008) “Extracting Sentiment as a Function of Discourse Structure and Topicality”. School of Computing Science Technical Report 2008-20.
|
Catalína JIMÉNEZ HURTADO
cjimenez@ugr.es
Claudia SEIBEL
Gala RODRÍGUEZ POSADAS
University of Granada, Spain
Catalína JIMÉNEZ HURTADO is a lecturer in linguistics and translation at the Department of translation and interpreting at the Universidad de Granada, Spain. She is interested in the relation between lexical semantics and translation, and in the modulation of both fields in the development of different types of meaning representation and controlled languages. Besides she is interested in the connection between her former research in textlinguistics and terminology management with the field of local grammars, multimodal discourse analysis, and multimodal corpus linguistics applied to the description of audiodescription. She directs a research and development team called TRACCE, which aims to promote accessibility in the media for the blind, the deaf and the hard-of-hearing.
Claudia SEIBEL is a senior lecturer at the Department of Translation and Interpreting at the University of Granada (Spain). She is interested in terminology management, and the relation between technical-scientific translation and different types of meaning representation. Further interests include controlled language and the application of audio description to local grammar. She is member of the research and development team TRACCE, which aims to promote accessibility in the media for the blind, the deaf and the hard-of-hearing.
Gala RODRÍGUEZ POSADAS graduated in 2005 from the University of Granada (UGR) with a degree in Translation and Interpreting and a final-year dissertation about cohesion in AD directed by Dr. Catalina Jiménez Hurtado. In 2006 she completed a postgraduate course in SDHoH and AD, and has since worked as a freelance audio describer for ONCE, the Spanish National Organization for Blind People. In 2007 she received her Diploma in Doctoral Studies. After four years of collaboration with the TRACCE project, financed by the Spanish Science and Technology Ministry, she received a placement to develop her PhD through a pre-doctoral grant with AMATRA in 2008, a project financed by the Andalusian Government.
|
|