43 Vygotsky (1997) explains that an object can be both a tool and a sign, depending on the role it plays at a particular time. He refers to a shovel as a tool if it is used to dig a hole, as it changes the shape of the ground. However, when the shovel is placed at the door, it becomes a sign, because it serves to remind the person that he needs to dig a hole.
The study of signs (semiotics) and the role they play in communication was initiated by an American pragmatist, Charles Sanders Pierce, in 1958 (Chandler, 2004). Pierce (1958) revealing the significant role played by signs in communication has impacted significantly on the way teaching and learning take place. The notion of semiotics and the use of signs in contemporary activities, including teaching and learning, have become highly influential, to the extent that ignoring them may result in inefficiency and ineffectiveness (Chandler, 2004).
According to Pierce (1958), there are three types of signs in general communication:
symbolic, iconic, and indexical. Symbolic signs are those in which the signifier does not resemble what is signified. Examples of these signs include those used in language:
punctuation, words, phrases, numbers, and sentences. For example, the word apple is a symbolic sign, as it does not resemble an apple in any form. Iconic signs are those in which the signifier resembles the signified by possessing some similar qualities (Pierce, 1958).
These signs show a natural and physical relation to the signified, and are easy to understand because they do not require one to learn them. For example, a picture of an apple is an iconic sign because it imitates actual features of an apple. Indexical signs are those in which the signifier does not resemble the signified but has connotative features that communicate a particular message to the person (Pierce, 1958). Examples of these are signs representing thunder, or smoke. However, Pierce (1958) suggests that to enable effective knowledge construction, both arbitrary and physical features need consideration, suggesting that employing a range of signs types might be useful.
Based on aforementioned explanations, all three types of signs were considered during the teaching intervention explored in this study. This was done because the visual and the verbal mode used during the intervention employ these signs for communication. The facts that tools mediate learning (Vygotsky, 1978), and that various types of signs may work complementarily to enable effective communication (Pierce, 1958), have provided strong grounds for considering tools and signs in the intervention.
44 Many recent and current schools of thought on the use of language and other semiotic modes in communication employ insights that are derivatives of Halliday’s (1978) Systemic Functional Linguistics (SFL). Earlier, Pierce (1958) defined semiotic resources as any form of activity, conduct, or process that involves signs, and their relation to meaning production.
Later, Saussure (1966) defined semiotics as “a science that studies the life of signs within the society” (p. 16). Subsequently, various views on semiotics that drew from Saussure’s (1966) definition emerged. These include the definitions of semiotics by Sebeok (1994) as “the antique doctrine of signs” (p. 5), and by Danesi and Santeramo (1992) as “the general science of signs and meanings” (p. vii). According to SFL, a semiotic is a system of signs that enables meanings to be made (Halliday, 1994). However, the only semiotic mode that was originally focused on as enabling meaning-making is language (Halliday, 1978).
Consequently, language became regarded as the principal mode of communication, resulting in it being well studied and analysed in terms of its grammar and meanings (Royce, 1998).
Halliday (1978) claims that the nature of language as a semiotic system is defined by its multiple strata, where the most central stratum is the lexicogrammar. The choice of the term
‘lexicogrammar’ is influenced by the fact that language learning and learning through language require combined knowledge of its grammar and vocabulary (Halliday, 1994).
Halliday (1994) explains that understanding the lexicogrammar is crucial, as it enables meaning construction from a sentence or a phrase.
The initial SFL view of semiotics was adapted from being “the general study of signs” to “the study of sign systems… the study of meaning in its most general sense” (Halliday, 1985, p. 3- 4). The adapted description of semiotics contrasts with the notion that language (written and spoken) is the only semiotic system for meaning-making. Apart from written/spoken language, Halliday (1985) points out that there are other semiotic modes. Drawing from Kress and Hodge (1988), language is viewed as an ideology, suggesting that communication principles that are applicable to it may be applicable to other semiotic modes as well. This was later restated by O’Toole (1995), who describes language as offering “a powerful and flexible model for the study of other semiotic modes” (p. 150). He describes language as a paradigm, which can be applied to other systems of meaning-making. Kress and Hodge (1988) posit that communication in social settings must be stretched to accommodate all semiotic modes that have the potential to make meanings. These ideas need consideration for studies that are related to learners’ sense-making of certain concepts via non-linguistic semiotic modes.
45 The strength of SFL is that understanding semiotics, including those that are non-linguistic, can be achieved by reviewing the three metafunctions. These metafunctions are reviewed on the role they play in communication and making meaning, as understanding them has the potential to explicate how other semiotic modes may be used effectively as teaching media.
Since language is merely one of many systems for making meanings (Halliday, 1985), its principles may be applicable to other semiotic modes based on their metafunctions. Table 7 shows the terminology related to the metafunctions of semiotics.
Table 7. The terminology related to the metafunctions of semiotics (Adapted from Royce, 2007)
SFL Subsequent studies
Author Halliday (1978) (Systemic Functional Linguistics)
O’Toole (1994) (Language of Displayed Arts)
Kress and van Leeuwen (1996) (Reading
images/Grammar of Visual
Design)
Royce (2007) (Intersemiotic complementarity)
Metafunctions Ideational Representational Representational Ideational Interpersonal Modal Interpersonal Interpersonal Textual Compositional Compositional Compositional
According to Halliday (1985), the metafunctions of SFL operate simultaneously in making meanings in every language. He defines the ideational metafunction of SFL as a meaning- making resource that involves the representation of experience. This includes representing our experience of the world around and inside us. The interpersonal metafunction of SFL involves making meaning through action (Halliday, 1985). This involves the speaker performing a particular action from which the listener can formulate a meaning. The textual metafunction of SFL entails making meanings through relation to context (Halliday, 1985).
These interpretations underwent adaptations in systemic functional multimodal discourse analysis (SF-MDA) to accommodate their viability in other systems of making meanings.
Thus, the subsequent metafunctions and their interpretations differ slightly from those used by Halliday (1985).
46 Discussing analysis of the metafunctions of visual-verbal intersemiotic complementarity may warrant a reminder of why the combination of verbal and visual modes is preferable to the linguistic mode alone. Royce (1998) describes visual-verbal intersemiotic complementarity as resulting in semantic expansion. This means that it conveys expanded forms of meanings, from which users choose to make their own meanings when guided by the context. The combined visual-verbal semiotic modes results in semantic expansion because language and visual images have distinct orientations (Lemke, 2000). Language presents a typological view (types and symbols) of reality to listeners or readers through symbols, participants, and circumstances. The visual semiotic mode, on the other hand, presents a topological view (spaces and shapes) of reality, where knowledge formulation is guided by the degree of image display, such as the position and relative size of its component parts (Lemke, 2000).
Lemke (2000) suggests that moving between typographical and topographical forms results in new organisational levels that provide space for new interpretations.
Accessing understanding of the intersemiosis of the visual and verbal semiotic modes (discussed earlier in this chapter) has the potential to reveal the synergy (expanded meaning) of intersemiotic complementarity, as described by Royce (1998). The ideational metafunction of visual-verbal intersemiotic complementarity is defined as meanings in both visual and verbal modes being lexico-semantically related (Royce, 1998; 2007). This suggests that words in the verbal mode and their equivalent items in the visual mode link to enable stronger meaning-making by the listener or viewer. This is usually realised through the sense relations of repetition, synonymy, antonymy, meronymy, hyponymy, and collocation. The interpersonal meaning-making resource involves meanings made in both visual and verbal modes through two sense relations: mood and modality (Royce, 1998; 2002; 2007). The textual (compositional) meaning-making resource involves making meanings from the texts and diagrams in a page; for example, texts and diagrams in a flipchart (Royce, 1998). This is achievable through the compositional sense relations such as information value, salience, visual framing, visual synonymy, and potential reading paths. Of these three meaning-making resources of intersemiotic complementarity, the ideational metafunction is the only one that is considered in this study, due to it involving experiential meaning.
Royce (1998) uses the concept of verbal to refer to written work, while this study uses it to refer to both spoken and written language, as teaching chemical bonding may not be confined to either of these alone. Chandler (2007) posits that written words are spoken words represented symbolically. Royce (1998) postulates that devising visual-verbal intersemiotic
47 complementarity is enabled by first considering the sense relations. In this study, this accounts for the ideational metafunction. The ideational sense relations of visual-verbal intersemiotic complementarity are specific items used to relate the experienced knowledge and the logical content, or the subject matter expressed, in both visual and verbal modes (Royce, 2007).
The sense relation of repetition is defined as the reiteration of identical experiential meaning (Royce, 1998). In visual-verbal intersemiotic complementarity, this happens when a lexical item repeats the meaning represented in the visual message element, or vice versa. This has the potential for making multiplicative meaning. Royce (2007) adds that repetition can also arise through the use of lexical items that are products of inflexion or derivation. For example, the inflexion of the word ‘oxidise’ leads to ‘oxidises’, and a derivation from the word ‘oxidiser’ can be ‘oxidant’.
If a case occurs where a similar experiential meaning is made in verbal and visual semiotic modes, a sense relation of synonymy is said to have occurred (Royce, 1998). This sense relation can be observed when the meaning in the lexical item is similar to (but not the same as) the meaning in the visual element, or vice versa. For example, an arrow drawn from a sodium atom’s electron to a chlorine atom has a meaning similar to the phrase ‘electron transferred from a sodium atom to a chlorine atom’. Even though the sense relations of repetition and synonymy are different from each other, I noticed that they are similar enough for their application in the coordinated visual-verbal modes to overlap, and thus complicate the analysis. Due to difficulty clearly distinguishing between these two sense relations in visual and verbal modes, the word ‘similarity’ was chosen in this study as an overarching term replacing the use of ‘repetition’ and ‘synonymy’.
If two semiotic modes make the opposite experiential meanings, the particular sense relation involved is antonymy (Royce, 1998). This sense relation may be applied in cases where learners are likely to realise opposite meanings conveyed by either the visual or verbal mode.
For example, an arrow indicating an electron being lost from a sodium atom has the opposite meaning as the lexical phrase ‘electron gained’ by a chlorine atom. The use of this sense relation is clearly useful for antonymous pairs such as ‘gain’ and ‘lose’. However, the possibility for this sense relation to confuse the learners due to the opposite meanings it makes was foreseen, despite the suggestion by Royce (1998) that it may be useful. Efforts to avoid the learning difficulty that might be linked to this case in the study involved selecting
48 knowledge items where this sense relation may be applied effectively and carefully coordinating the visual and verbal items of the knowledge taught.
The sense relation involved when the meaning is generated through the relationship between the general class of something and its sub-classes (termed hyponym and co-hyponyms respectively) is referred to as hyponymy (Royce, 1998). Either of the visual or the verbal semiotic modes can be presented as a general class of something to enable meaning to be made in the other semiotic mode. For example, if the Bohr diagram of the atom of an element shows two visible electrons in the outer shell (the diagram depicts the element as the hyponym), the teacher might state that this element belongs to group 2 of the periodic table (the teacher talk refers to group 2 of the periodic table as the co-hyponym). This in turn allows for more meaning to be made, such as that the element is a metal, it loses two electrons during bonding, and it has a valency of 2.
Royce (1998) defines meronymy as the cohesive relationship between the whole of something (termed super-ordinate) and its constituent part(s) (termed meronyms and co- meronyms). The Bohr diagram of a sodium atom can be considered as a whole visual message, with meronyms of lexical items such as nucleus, protons, neutrons, shells, and electrons. For the meaning to be carried across both modes, the lexical item ‘sodium atom’
has to be used as the whole verbal message, concurrently with its visual parts, such as the drawings of shells, protons, and electrons, and vice versa.
Halliday (1994) uses the word collocation, to refer to lexical items that have a tendency to co- occur. In intersemiotic terms, Royce (2007) defines collocation as when the item represented in one semiotic mode collocates, or has a tendency to be associated with, an item represented in another semiotic mode. This means that representing the message in one semiotic mode can have an expectancy relationship to the item in another mode. For example, a visible arrow pointing from a sodium atom to a chlorine atom may collocate with lexical items such as ‘sodium cation’ and ‘chlorine anion’.
The review of visual-verbal sense relations of the ideational metafunction has illustrated that visual-verbal intersemiotic complementarity can be applied in teaching chemical bonding.
This review has focused mainly on Royce’s (2007) concept of intersemiotic complementarity, and Halliday’s (1994) Systemic Functional Linguistics, which consider making of meaning by semiotic modes.