Multimodality of scientific discourse

Multimodality, as used in the systemic functional approach to multimodal discourse analysis (SF-MDA), refers to the combined use of more than one semiotic mode in making meaning (Cheng & Gilbert, 2009). A semiotic mode is a “regularised set of resources for meaning- making…” (Kress, 2003, p. 1). Examples of these meaning-making resources are images, gesture, music, speech, and sound effects. Bock (2016) expands on this by pointing out that any material, either drawn from nature (such as feather, wood, or metal), or from cultural history (such as word, music or associated 3-dimensional object), can be a semiotic mode, provided it reflects regularities and follows an agreed-upon convention. For this study, the pedagogic approach chosen for exploration recognises the multimodality of scientific discourse.

Knowledge of individual semiotic modes and meaning-making resources in terms of their grammar and the role they play in communication might support effectively using multimodality in the pedagogy of science topics. The SF-MDA approach considers the affordances offered by many semiotic modes in terms of their ability to work in a

34 complementary way to make meaning (O’Halloran, 2008). Lemke (1998) posits that the pedagogy of science is inherently multimodal. Cheng and Gilbert (2009) add that scientific meanings are commonly made by the “joint co-deployment” of two or more semiotic modes within one message (p. 56). This multimodal nature of science is realised in that its concepts

“… are not defined by the common denominator of their representations, but by the sum, the union of meanings implied by all these representations … It is the nature of scientific concepts that they are semiotically multimodal…”(Lemke, 1998, p. 110-111). Furthermore, the multimodal nature of science concepts also requires outcomes and processes of science pedagogy to be multimodal (Lemke, 1998). This denotes that teaching, assessment tasks, and learners’ responses in science should involve the use of more than one semiotic mode.

Cheng and Gilbert (2009) assert that different semiotic modes in science have different affordances, and address different specialised tasks. The coordinated use of these affordances and specialised tasks of different semiotic modes results in the combined effect – the meanings that science teachers and textbook authors intend (Kress, Jewitt, Ogborn &

Tsatsarelis, 2001). Each meaning made by a certain semiotic mode can interact with and contribute to the meanings made by other semiotic modes (Kress et al., 2001). These different semiotic modes may carry similar meanings, or make meanings in a complementary way.

Moreover, multimodality is multiplicative because it results in semantic expansion – the multiplication of meaning – due to each semiotic mode having a different semantic orientation (Lemke, 1998). Examples of this are evident in the complementary use of language and visual semiotic modes. Language has a typographical standpoint (concerned with legibility), while visuals have a topographical standpoint (concerned with visibility) of knowledge (Martin, 1992). Thus, a combination of these two communication modes results in more meanings being made than either mode alone.

Notwithstanding affordances offered by different semiotic modes, it has been noticed that some semiotic modes are more strongly recognised than others (Kress & van Leeuwen, 2006). For this reason, certain modes have been used frequently in both social and cultural works, at the expense of other modes that also have potential for making meaning. Language (both spoken and written) is more strongly recognised and used at the expense of other modes. In particular, its use has been consistently dominant over the use of the visual mode (Kress & van Leeuwen, 1996). After this dominance was realised, it was proposed that both verbal and visual semiotic modes should be used equally in order to stop privileging language over the visual mode, and ignoring the affordance offered by the visual mode (Siegel, 2006).

35 However, Royce (2002) asserts that making sense of knowledge necessitates an understanding of the meaning-making potential of the combined visual and verbal semiotic modes. He recommends that intersemiotic resources should be explored for pedagogic use.

In chemistry, the use of the verbal semiotic mode alone is no longer regarded to be as important or effective as when used together with the visual semiotic mode (Nugroho, 2009).

Unsworth (2006) argues that when used alone, the verbal semiotic mode is an obstacle to making meaning due to two challenges: lexical (word related) difficulty, and grammatical (language rule) complexity. The lexical difficulty of chemistry is caused by high lexical density (many content words per clause or sentence) (Clay, 1971). This means many learners are not familiar with content words, as they are not used often in their everyday language.

Grammatical complexity involves using language that may not be lexically dense, but grammatically complicated, making it a challenge to meaning-making (Clay, 1971).

Another challenge with chemistry language involves it being a lingua-chemica (Gilbert &

Treagust, 2009). Lingua-chemica refers to the specialised language of chemistry, with its chemical terms and conventions only being known to and used by chemists. Gilbert and Treagust (2009) argue that chemistry students should be viewed as similar to students learning in a second language, who are expected to both learn the second language and use this same language simultaneously. To exacerbate this challenge, some students learn chemistry in a language of learning and teaching (LoLT) that is not their first language – a process that further impedes their making sense of chemistry topics (Sliwka, 2003). This is true in Namibia, where the majority of students learn content subjects, including Physical Science, in English, even though it is not their mother tongue (Namupala, 2013).

The challenge of using language alone in chemistry pedagogy for topics such as chemical bonding might be addressed by combining the verbal chemistry language with a range of visual representations (Gilbert & Treagust, 2009). This combined use of the verbal mode and visual mode helps to depict aspects of chemical models, while at the same time minimising the challenge of chemical language (Gilbert & Treagust, 2009). The range of visual representation includes graphs, diagrams, photographs, and charts. Since Pozzer and Roth (2003) posit that visual and verbal modes work complementarily in making meanings, it would make sense for science teachers to use this complementarity in a coordinated way when teaching chemistry topics such as chemical bonding. Most often, teachers rely on their talk for explaining chemistry topics because of its feasibility; not considering the affordances

36 of the visual mode, which may require more careful lesson planning. However, this over- reliance on language is problematic, as it might cause confusion and misunderstanding for learners (Taber, 2001). The complementarity of the visual and verbal modes will now be discussed.