I am thrilled to announce that Kay O’Halloran an expert on multimodal discourse analysis has given me an interview. She recently spoke at the 2009 Congress of the Humanities and Social Sciences in Ottawa as a featured speaker (invited by the Canadian Association for the Study of Discourse and Writing). Kay is an Associate Professor in the Department of English Language and Literature at the National University of Singapore and she is the Director of the Multimodal Analysis Lab. (more details about Kay in future installments)
Before going with the introduction and the interview, I want to explain why I think this work is important. (Forgive me if I gush?) We have so much media coming at us at any one time and it is increasingly being ‘mashed up’, remixed, reused, and repurposed. How important is text going to be when we have icons and videos and audio materials to choose from? Take for example, the bubble charts on Andrew Maynard’s 2020 blog which are a means of representing science Twitters. How do you interpret the information? Could they be used for in-depth analysis? (I commented earlier about the bubble charts on June 23 and 24, 2009 and Maynard’s post is here. You might also want to check out the comments where Maynard explains few things that puzzled me.)
As Kay points out in her responses to my questions, we have more to interpret than just a new type of chart or data visualization.
1. I was quite intrigued by the title of your talk (A Multimodal Approach to Discourse Studies: A paradigm with new research questions, agendas and directions for the digital age) at the 2009 Congress for the Humanities and Social Sciences held in Ottawa, Canada this May. Could you briefly describe a multimodal approach for people who aren’t necessarily in the field of education?
Traditionally, language has been studied in isolation, largely due to an emphasis on the study of printed linguistic texts and existing technologies such as print media, telephone and radio where language was the primary resource which was used. However, various forms of images, animations and videos form the basis for sharing information in the digital age, and thus it has become necessary to move beyond the study of language to understand contemporary communicative practices. In a sense, the study of language alone was never really sufficient because analysing what people wrote or said missed significant choices such as typography, layout and the images which appeared in the written texts, and the intonation, actions and gestures which accompanied spoken language. In addition, disciplinary knowledge (e.g. mathematics, science and social science disciplines) involves mathematical symbolism and various kinds of images, in addition to language. Therefore, researchers in language studies and education are moving beyond the study of language to multimodal approaches in order to investigate how linguistic choices combine with choices from other meaning-making resources.
Basically multimodal research explores the various roles which language, visual images, movement, gesture, sound, music and other resources play, and the ways those resources integrate across modalities (visual, auditory, tactile, olfactory etc) to create meaning in artefacts and events which form and transform culture. For example, the focus may be written texts, day-to-day interactions, internet sites, videos and films and 3-D objects and sites. In fact, one can think of knowledge and culture as specific choices from meaning-making resources which combine and unfold in patterns which are familiar to members of groups and communities.
Moreover, there is now explicit acknowledgement in educational research that disciplinary knowledge is multimodal and that literacy extends beyond language.
The shift to multimodal research has taken place as a result of digital media which not only serves as the object of study, but also because digital media technologies offer new research tools to study multimodal texts. Such technologies have become available and affordable, and increasingly they are being utilised by multimodal researchers in order to make complex multimodal analysis possible. Lastly, scientists and engineers are increasingly looking to social scientists to solve important problems involving multimodal phenomena, for example, data analysis, search and retrieval and human computer interface design. Computer scientists and social sciences face similar problems in today’s world of digital media, and interdisciplinary collaboration is the promise of the future in what has become the age of information.
Have a nice weekend. There’ll be more of the interview next week, including a bibliography that Kay very kindly provided.
Although I’m not familiar with the terminology Dr. O’Halloran uses, I would argue that the concepts she discusses have been around for a long time – predating digital technologies/communication by decades if not longer. For instance, in my field, medical and science communication (not to mention business and advertising), our tendency to take in information “holistically” has been used in everything from drug ads to public service announcements.
Deliberate use of colour(s) in the text as well as graphs and images, particularly when describing statistics and data, would skillfully be manipulated to drive home some point – usually, the objective being to “sell” the position or drug or what-have-you. Parsing the data from a study, e.g., whether in a drug company ad or a journal article, one would find that the graph would have been transposed (in other words, the line would indeed be going up or down, which to the casual observer would imply the desired outcome, but on close inspection the information was presented in reverse. Or, the numbers would be broken down into such small units that the “improvement” was illusory, e.g., going from 1.2 to 1.3 etc. Or, the scale would be off. Other times, the units would be different. For instance, I recently discovered when researched “HRT” for a book I recently wrote (Estrogen Errors) that the graph commonly used in medical texts to describe the balance of estrogen and progesterone during a normal cycle were expressed in different units (one pico, the other nano). Were the amounts accurately portrayed, progesterone, at its peak, would be over 1000x more than estrogen. But our cultural focus was estrogen=female, the graph merely confirmed what people already believed. So, while I would agree that interdisciplinary research is sorely needed, and that one should look beyond language/text, to restrict oneself to the present day or to consider this somehow unique to the digital age is limiting. Having said that, I would agree that the extent and amount of information has certainly expanded exponentially in recent years – making such research doubly important.
Hi Inkbat! Thanks for the comments and your points are well taken. There is a general tendency to talk about the digital media as somehow divorcing us from all the media of the past and previously unheard of critical thinking skills. That said, my sense of Dr. O’Halloran’s work is that it encompasses and integrates far more than has traditionally been the subject of study. Text (literature) has often been studied in isolation from the visual image (art history) which is studied in isolation from the moving image (film studies).
hey, frogheart! 🙂
Very true. Today’s blog was interesting – as you say, this is one of the few instances where “interdisciplinary” actually seems to signify something and consists of more than just tacking on an extra subject or discipline to create some kind of misshapen mass that you then proudly declare to be “trans” or “cross” or “inter” disciplinary. Dr. O’Halloran’s work (even though I don’t actually understand what it is at this point) does seem to be a genuine effort to break through and blend different modes of disciplinary thought. I concur with your comment on the breadth of disciplines as well – it isn’t that much of a stretch to go from one sub specialty of say chemistry to another …
Is it any wonder so much of our world view is so reductionist when the academy itself and intellectual thought has become so fragmented?!