Tag Archives: Kay O’Halloran

Science’s exquisite corpse and other interesting science communication developments

2 Replies

The ‘exquisite corpse’ is a game that surrealists started playing in the earlyish part of the 20th century, according to the wikipedia essay here. I first came across the game in a poetry context. I was part of an online poetry organization and someone suggested (as I recall) that we start an exquisite corpse project on our website. Nothing much of came of it but I’ve always found the phrase quite intriguing. The idea is that a group of people play with words or images individually then put the pieces together to construct a final work.

Andrew Maynard’s 2020 Science blog has been featuring an art/science exquisite corpse project by Tim Jones. Billed as an experiment in science engagement, Jones and his colleagues (at the Imperial College) have created videos of two  members of the public, a science communicator, and a scientist talking about a drawing they’ve each created that expresses what they each think is important abou science.  What you’ll see are the interviews, the pictures that the people drew, and an exquisite corpse of science, if you go here.

Tim Jones has now invited more people to participate for the biggest art/science project in history (maybe) to create a bigger exquisite corpse of science. If you’re interested go here to Tim Jones’s site or you can read about it here at 2020 Science.

I came across a way for scientists to publish workflows and experiment plans  at myExperiment.

BBC4 has been conducting an experiment of their own, visualising radio. In this case, it’s a science show that’s cast over the internet. They’ve blogged about the project here.

All of this makes me think back to the interview that Kay O’Halloran (July 3, 6, and 7, 2009 postings) gave me on multimodal discourse analysis and Andrew Maynard’s bubble charts (June 24 and 29, 2009). It’s exciting to explore these new and rediscovered techniques and to think about how we perceive the information being conveyed to us.

One last bit, there’s been an announcement from Lord Drayson, UK’s Science and Innovation Minister and Chair of Ministerial Group on Nanotechnologies that the government is seeking advice for a national nanotechnology strategy. From the announcement on Nanowerks News,

Industry, academia and consumer groups were invited to use a new website to help develop the strategy, building on and consolidating the existing research and consultations that have already taken place. The website will gather views on core issues including research, regulation, innovation and commercialisation, measurement and standards and information as well as on the anticipated impact of nanotechnologies on a wide range of sectors. The aim of the strategy is to describe the actions necessary to ensure that the UK obtains maximum economic, environmental and societal benefit from nanotechnologies while keeping the risks properly managed.

The rest of the announcement is  here and the project website is here.  (NOTE: Consumer groups will have their own website although members of the public are welcome the new website is really intended for academia, industry, and NGOs.)

Happy weekend!

The other side of the multimodal discourse coin

1 Reply

Bill Thompson has an article, Giving life a shape, on BBC News which touches tangentially on approaching the world in a multimodal fashion. He takes a kind of digital approach i.e.when he uses the word technology he actually means digital technology and his examples come from social networking, Second Life, social gaming and other activities mediated through the Internet and computers. From the article,

… because in working through the creative potential of new technologies artists of all types are helping us to find new ways to think about these tools and working out how to integrate them into our wider cultural and commercial practice.

They are helping us to explore the latest chapter in the ongoing conversation between human psychology and the capabilities of modern technology, something which will matter more and more as the network becomes pervasive and digital devices penetrate every area of our lives.

Different modalities (audio files, graphics files, animation (Second Life), and others are referred to indirectly in the course of Thompson’s article, which is why I’ve picked up on it. In  light of the Kay O’Halloran interviews (on this website blog July 3, 6, and 7, 2009) Thompson’s description of how “artists help find us new ways to think about things” reveals the other side of the multimodal discourse coin.

While O’Halloran and her colleagues develop a framework for analyzing and understanding multimodal discourse, it’s artists (I define that word broadly) who enact and explore that discourse through their work.

One quibble, I think Thompson’s definition could be broadened so that technology  includes nanotechnology, biotechnology, synthetic biology and other emerging technologies. Now back to Thompson and a comment that works no matter how you define technology,

One problem in talking about this is that relatively few people understand the underlying technology sufficiently well to be comfortable with it. We have few stories that talk about technology and few workable metaphors or analogies that let us convey complex technological issues in ways that people really grasp.

Metaphors came up in the O’Halloran interview (July 6, 2009 posting) too and I got this in the comments (from inkbat),

I was struck by the point on metaphor. When you come right down to it, isn’t it sad that so many of our concepts are the result of some designer or advertiser or whoever deciding to create some kind of shortcut for us .. which would work if it was just in the one instance but then it takes on a life of its own and suddenly we no longer think of the heart AS IF it is a pump but as though it IS a pump. Or the brain as a computer. …

Unfortunately as inkbat points out, we forget we’ve created a metaphor and we treat it ‘as if it were so’ to results that can be disastrous. Still, I think that creating metaphors and then having to ‘break’ or ‘see through’ them ultimately discarding the old metaphor and developing a new one is part of the human condition.

Back to my nanotech ways tomorrow.

Kay O’Halloran interview on multimodal discourse: Part 3 of 3

Leave a reply

Thanks to Kay O’Halloran for kindly giving me this interview and here’s the last part which also includes a bibliography.

3. I notice that you have a project examining PowerPoint in the classroom and in corporate settings which you are conducting for the Australian Research Council. Could you explain a little bit about the project?

The project ‘Towards a Social Theory of Semiotic Technology: Exploring PowerPoint’s Design and its Use Higher Education and Corporate Settings awarded by the Australian Research Council (ARC) (Discovery Grant No. DP09889939) is a collaborative project between Chief Investigator Professor Theo van Leeuwen (Dean for Faculty of Arts and Social Sciences, University of Technology, Sydney), Dr Emilia Djonov (Post-doctoral Fellow, University of Technology, Sydney) and myself. The following description of the project is drawn from our research proposal.

PowerPoint has become the dominant technology for designing and delivering presentations, particularly in education and business settings where success often depends on skills in the use of the application. Powerpoint is the subject of much debate and it creates strong reactions, both positive and negative. It’s either praised for increasing presenters’ confidence and eloquence (e.g. Gold 2002) or it’s condemned for limiting users’ ability to present complex ideas through an over-simplification of information presented in bullet points, linear slide-by-slide formats and illegible graphics (e.g. Tufte 2003).

From the multimodal perspective, Powerpoint is a semiotic technology which has a range of options (i.e. grammar) from which presenters make selections with regards to the linguistic text, images, animations and sounds. There are default themes which the presenter may choose as well. These choices integrate in multimodal presentations which are recontextualised by the speaker during the presentation. Most studies of Powerpoint adopt a different approach, however, by either exploring lecturers’ and students’ perceptions of PowerPoint to support learning, or alternatively they are experimental studies which investigate the effects of PowerPoint versus transparency-supported lectures on learning.

Our project adopts a multimodal approach to (a) conceptualise the grammar of Powerpoint through the study of its systems of meaning; (b) analyse and compare the choices which are made in higher education and corporate settings; and (c) investigate how these choices are contextualised in presentations. In this way, we will explore how the design of PowerPoint supports or hinders the achievement of the various goals of the presenters. At the moment, there are no studies which investigate differences in the use of Powerpoint across educational and corporate settings, and furthermore, there is no evidence for arguments that PowerPoint cannot support the representation of knowledge in technical disciplines such as engineering (Tufte, 2003) or the rich narrative and interpretative skills required for social science disciplines (Adams, 2006), nor is there evidence that PowerPoint has introduced corporate rhetoric into educational practices (Turkle, 2004). In addition, the study will provide guidelines for evaluating and improving the design and use of PowerPoint and other similar presentation software.

Bibliography

Adams, C. (2006). PowerPoint, habits of mind, and classroom culture. Journal of Curriculum Studies, 38(4), 389 – 411.

Gold, R. (2002). Reading PowerPoint. In N. J. Allen (Ed.), Working with words and images: New steps in an old dance. (pp. 256-270). Westport, Connecticut: Ablex.

Tufte, E. R. (2003). The cognitive style of PowerPoint (2nd edition). Cheshire, Connecticut: Graphics Press.

Turkle, S. (2004). The fellowship of the microchip: global technologies as evocative objects. In M. Suárez-Orozco & D.B. Qin-Hilliard (Eds.), Globalization: Culture and Education in the New Millennium (pp. 97-113). Berkeley, CA: University of California Press.

Kay O’Halloran interview on multimodal discourse: Part 2 of 3

4 Replies

Before going on to the second part of her interview, here’s a little more about Kay O’Halloran. She has a Ph.D. in Communication Studies from Murdoch University (Australia), a B.Sc. in Mathematics and a Dip. Ed. and B.Ed. (First Class Honours) from the University of Western Australia.

The Multimodal Analysis Lab of which she is the Director brings together researchers from engineering, the performing arts, medicine, computer science, arts and social sciences, architecture, and science working together in an interdisciplinary environment. (This is the first instance where I’ve seen the word interdisciplinary and can wholeheartedly agree with its use. As I have found, interdisciplinary can mean that an organic chemist is having to collaborate with an inorganic chemist or an historian is working with an anthropologist. I understand that there are leaps between, for example, history and anthropology but by comparison with engineering and the performing arts, the leap just isn’t that big.)

There’s more on Kay O’Halloran’s page here and more on the Multimodal Analysis Lab here.

2. Could you describe the research  questions, agendas and directions that are most compelling to you at this  time?

Multimodal research involves new questions and problems such as:

– What are the functionalities of the resources (e.g. language versus image)?

– How do choices combine to make meaning in artefacts and events?

– What types of reconstruals take place within and across semiotic artefacts and events and what type of metaphors consequently arise?

– How is digital meaning expanding our meaning-making potential?

The most compelling agendas and directions in multimodal research include developing new approaches to annotating, analysing, modeling, and interpreting semiotic patterns using digital media technologies, particularly in dynamic contexts (e.g. videos, film, website browsing, online learning materials). The development of new practices for multimodal analysis (e.g. multimodal corpus approaches) means we can investigate social cultural patterns and trends and the nature of knowledge and contemporary life in the age of digital media, together with its limitations. Surely new media offers us the potential for new research paradigms and making new types of meanings which will lead us to new ways of thinking about the world. Also, multimodal approaches offer the promise of new paradigms for educational research where classroom and pedagogical practices and disciplinary knowledge can be investigated in their entirety. Multimodal research opens up a new exciting world, one which is being eagerly embraced by academic researchers and postgraduate students as the way forward (in my experience at least).

Kay O’Halloran interview on multimodal discourse: Part 1 of 3

3 Replies

I am thrilled to announce that Kay O’Halloran an expert on multimodal discourse analysis has given me an interview. She recently spoke at the 2009 Congress of the Humanities and Social Sciences in Ottawa as a featured speaker (invited by the Canadian Association for the Study of Discourse and Writing). Kay is an Associate Professor in the Department of English Language and Literature at the National University of Singapore and she is the Director of the Multimodal Analysis Lab. (more details about Kay in future installments)

Before going with the introduction and the interview, I want to explain why I think this work is important. (Forgive me if I gush?) We have so much media coming at us at any one time and it is increasingly being ‘mashed up’, remixed, reused, and repurposed. How important is text going to be when we have icons and videos and audio materials to choose from? Take for example, the bubble charts on Andrew Maynard’s 2020 blog which are a means of representing science Twitters. How do you interpret the information? Could they be used for in-depth analysis? (I commented earlier about the bubble charts on June 23 and 24, 2009 and Maynard’s post is here. You might also want to check out the comments where Maynard explains few things that puzzled me.)

As Kay points out in her responses to my questions, we have more to interpret than just a new type of chart or data visualization.

1. I was quite intrigued by the title of your talk (A Multimodal Approach to Discourse Studies: A paradigm with new research questions, agendas and directions for the digital age) at the 2009 Congress for the Humanities and Social Sciences held in Ottawa, Canada this May. Could you briefly describe a multimodal approach for people who aren’t necessarily in the field of education?

Traditionally, language has been studied in isolation, largely due to an emphasis on the study of printed linguistic texts and existing technologies such as print media, telephone and radio where language was the primary resource which was used. However, various forms of images, animations and videos form the basis for sharing information in the digital age, and thus it has become necessary to move beyond the study of language to understand contemporary communicative practices. In a sense, the study of language alone was never really sufficient because analysing what people wrote or said missed significant choices such as typography, layout and the images which appeared in the written texts, and the intonation, actions and gestures which accompanied spoken language. In addition, disciplinary knowledge (e.g. mathematics, science and social science disciplines) involves mathematical symbolism and various kinds of images, in addition to language. Therefore, researchers in language studies and education are moving beyond the study of language to multimodal approaches in order to investigate how linguistic choices combine with choices from other meaning-making resources.

Basically multimodal research explores the various roles which language, visual images, movement, gesture, sound, music and other resources play, and the ways those resources integrate across modalities (visual, auditory, tactile, olfactory etc) to create meaning in artefacts and events which form and transform culture. For example, the focus may be written texts, day-to-day interactions, internet sites, videos and films and 3-D objects and sites. In fact, one can think of knowledge and culture as specific choices from meaning-making resources which combine and unfold in patterns which are familiar to members of groups and communities.

Moreover, there is now explicit acknowledgement in educational research that disciplinary knowledge is multimodal and that literacy extends beyond language.

The shift to multimodal research has taken place as a result of digital media which not only serves as the object of study, but also because digital media technologies offer new research tools to study multimodal texts. Such technologies have become available and affordable, and increasingly they are being utilised by multimodal researchers in order to make complex multimodal analysis possible. Lastly, scientists and engineers are increasingly looking to social scientists to solve important problems involving multimodal phenomena, for example, data analysis, search and retrieval and human computer interface design. Computer scientists and social sciences face similar problems in today’s world of digital media, and interdisciplinary collaboration is the promise of the future in what has become the age of information.

Have a nice weekend. There’ll be more of the interview next week, including a bibliography that Kay very kindly provided.

Sensing, nanotechnology and multimodal discourse analysis

1 Reply

Michael Berger has an interesting article on carbon nanotubes and how the act of observing them may cause damage. It’s part of the Nanowerk Spotlight series here,

A few days ago we ran a Nanowerk Spotlight (“Nanotechnology structuring of materials with atomic precision”) on a nanostructuring technique that uses an extremely narrow electron beam to knock individual carbon atoms from carbon nanotubes with atomic precision, a technique that could potentially be used to change the properties of the nanotubes. In contrast to this deliberately created defect, researchers are concerned about unintentional defects created by electron beams during examination of carbon nanomaterials with transmission electron microscopes like a high-resolution transmission electron microscope (HRTEM)

The concern is that that electrons in the beam will accidentally knock an atom out of place. It was believed that slowing the beam to 80 kV would address the problem but new research suggests that’s not the case.

If you go to Nanowerk to read more about this, you’ll find some images of what’s going on at the nanoscale. The images you see are not pictures per se. They are visual representations based on data that is being sensed at the nanoscale. The microscopes used to gather the data are not optical. As I understand it, these microscopes are haptic as the sensing is done by touch, not by sight. (If someone knows differently, please do correct me.) Scientists even have a term for interpreting this data, blobology.

I’ve been reading up on these things and it’s gotten me to thinking about how we understand and interpret not just the macroworld that our senses let us explore but the micro/nano/pico/xxx scale worlds which we cannot sense directly. In that light, the work that Kay O’Halloran, an associate professor in English Language and Literature and the Director of the Multimodal Analysis Lab at the National University of Singapore, is doing in the area of multimodal discourse analysis looks promising. From her article in Visual Communication, vol. 7 (4),

Mathematics and science, for example, produce a new space of interpretance through mixed-mode semiosis, i.e. the use of language, visual imagery, and mathematical symbolism to create a new world view which extends beyond the possible using language. (p. 454)

Nanotechnology metaphors and understanding visual data

Leave a reply

I found a typesetting metaphor today in a media release titled, ‘Molecular typesetting — proofreading without a proofreader‘.  The number of publishing, writing, and reading metaphors associated with nanotechnology has always startled me.  As for the article, it is about how proteins are built with a minimal number of errors in a process that researchers compare to typesetting. If you want to read more, you can go here to Nanowerk News.

I looked at Andrew Maynard’s 2020 Science blog and found a posting that presents some visual data about science twittering. He has three spheres made of bubbles or smaller spheres representing the number of followers that science twittering attracts. He’s done this before and I’m still not sure how to interpret the data and I mean that from two perspectives. I don’t understand the visual data being presented very well (Maynard does provide an explanation in a screencast) and while I find the whole Twitter scene interesting I’m waiting to see if it becomes something more substantive (which seems to be Maynard’s stance as well).

With regard to visual data, I think this will become increasingly important and it was one of the reasons I was so interested in Kay O’Halloran’s talk at the 2009 Congress of Humanities and Social Sciences about mathematics and using visual data to communicate about it. Unfortunately, the organizers were not able to arrange a webcast but I’ll  see if I can dig up so more information about what she’s doing.

As for the Twitter phenomenon, it seems interesting to me that MySpace has just downsized itself (more here) as I can recall when it was as a big trend as Twitter is now. I’m not sure what conclusions can be drawn from the popularity of any social networking phenomenon. I think it is clear that people are interested in each other (and sometimes for the oddest of reasons) as for anything else I need more data.

One brief note, I had occasion to email Andrew Maynard last week and during the exhange I asked him why the Project on Emerging Nanotechnologies is having fewer events. (I figured their Chief Science Advisor would know why.) He says there is a reorganization taking place.

Science and multimodal media approaches

Leave a reply

There’s an interesting article on an experiment being conducted at Fortune magazine. For anyone who’s not aware, the publishing industry is in a serious quandary and many publishers are struggling for survival. This explains why Fortune magazine has a multimodal media version of its print cover story available on the web. From the article by Andrew Vanacore on the Physorg.com site here,

Dispensing advice on finding a job during a recession, the piece had a soundtrack, a troupe of improv actors from Chicago and about 4,000 fewer words than your average magazine feature. Instead of scrolling through a column of text, readers (if the term can be applied) flipped through nine pages that told the story with a mix of text, photo-illustrations, interactive graphics and video clips.

I like that bit about “readers (if the term can be applied)” because I’ve been coming to the conclusion that with less and less text (think Twitter) that we may be returning to a more oral society as opposed to our still literate-dominant society. I’ve been thinking about this since some time in the early 1990’s when a communications professor (Paul Heyer) at Simon Fraser University first made the suggestion to us in class.

Following on this idea that we will be less and less text oriented, the work that Kay O’Halloran is doing at her Mulimodal Lab (situated at the National University of Singapore) casts an interesting light on where this all may be going with regard to science communication.  An associate professor in the Dept. of English Language and Literature, O’Halloran is speaking tomorrow (in Ottawa, Canada) at the 2009 Congress of Humanities and Social Science about reading, mathematics, and digital media. I hope there will be a webcast of her talk available afterwards (I suggested it to the folks from the Canadian Association for the Study of Discourse and Writing (CASDW) who are sponsoring her talk. If there is a webcast, I’ll post a link.

Meanwhile, for those of us not lucky enough to be there, from the programme,

To understand digital texts we need theories that study more than words alone. This talk will show how images, mathematical and scientific symbols, gestures, actions, music, and sound can all be studied along with words using examples from the classroom, digital media, and mathematics.

I believe that more and more of our communication, science and otherwise, is moving in a multimodal direction. It seems so obvious to me that it surprises me that it’s not commonly accepted wisdom.

Later this week, I will have more about science funding and I have notice of another sythetic biology event coming up at the Project on Emerging Nanotechnologie.