About
Multi-modal literacy supports the ability to create meaning through the combination of different modes including written, spoken, visual, audial, spatial, and gestural means. Each mode provides a level of meaning and the combination of mulitple modes creates an even fuller understanding as it engages a variety of senses. Picture books can be read with just the text or just the pictures but the combination of both provide a more complete understanding of the story through a multi-modal experience.
Examples of multi-modal works:
Overview and Definitions of different modes from the Department of Education and Training from the State Government of Victoria, Australia:
Written meaning
Conveyed through written language via handwriting, the printed page, and the screen. Choices of words, phrases, and sentences are organised through linguistic grammar conventions, register (where language is varied according to context), and genre (knowledge of how a text type is organised and staged to meet a specific purpose). See: Writing and Reading and Viewing
In bilingual or multilingual texts, written meaning may be conveyed through different scripts and laid out differently, whether typed or handwritten. EAL/D learners may also write words from their home languages using English letters (transliteration).
Spoken (oral) meaning
Conveyed through spoken language via live or recorded speech and can be monologic or dialogic. Choice of words, phrases, and sentences are organised through linguistic grammar conventions, register, and genre. Composing oral meaning includes choices around mood, emotion, emphasis, fluency, speed, volume, tempo, pitch, rhythm, pronunciation, intonation, and dialect. EAL/D learners may make additional choices around the use of home languages to create mood or emphasise meaning. See: Speaking and listening pedagogic resources.
Visual meaning
Conveyed through choices of visual resources and includes both still image and moving images. Images may include diverse cultural connotations, symbolism and portray different people, cultures and practices. Visual resources include: framing, vectors, symbols, perspective, gaze, point of view, colour, texture, line, shape, casting, saliency, distance, angles, form, power, involvement/detachment, contrast, lighting, naturalistic/non-naturalistic, camera movement, and subject movement. See Visual literacy metalanguage.
Audio meaning
Conveyed through sound, including choices of music representing different cultures, ambient sounds, noises, alerts, silence, natural/unnatural sounds, and use of volume, beat, tempo, pitch, and rhythm. Lyrics in a song may also include multiple languages.
Spatial meaning
Conveyed through design of spaces, using choices of spatial resources including: scale, proximity, boundaries, direction, layout, and organisation of objects in the space. Space extends from design of the page in a book, a page in a graphic novel or comic, a webpage on the screen, framing of shots in moving image, to the design of a room, architecture, streetscapes, and landscapes.
Gestural meaning
Conveyed through choices of body movement; facial expression, eye movements and gaze, demeanour, gait, dance, acting, action sequences. It also includes use of rhythm, speed, stillness and angles, including ‘timing, frequency, ceremony and ritual’ (Cope and Kalantzis, 2009. p. 362). Gestures and body language may have diverse cultural connotations.
Multimodal Resources:
Literacy and Media Lab: The Literacy and Media Lab is a collaboration between Lafayette Elementary and the School of Education at the University of Colorado-Boulder. We host after-school programs each semester, facilitated by CU students (and current K-12 teachers) pursuing their MA in Literacy Studies. This website features lessons they planned and implemented, as well as media produced by Lafayette Elementary students.
Integrated Storytelling and STEM through Multimodal Composing: Powerpoint presentation by Bridget Dalton at University of Colorado providing an overview of using STEM tools to create multi-modal elements in storytelling in conjunction with Build a Better Book.