[Air-L] Relationships between text and media objects

[Anders Fagerjord]

Hi, Daniel

These relations keep cropping up in literature under different names. I suggested using the terms ‘consonance’, ‘dissonance', ‘polyphony’, and ‘accompaniment'  in a 2010 chapter dealing with web documentaries. I have later learned that I was far from the first to give these relations a name, but I don’t think there is any established consensus.

Fagerjord, Anders. “Multimodal Polyphony: Analysis of a Flash Documentary”. Inside Multimodal Composition Ed. Andrew Morrison. New York: Hampton Press, 2010. (Preprint: http://fagerjord.no/downloads/polyphony_preprint.pdf)

Best,

—anders

--
Anders Fagerjord, dr.art.
Associate professor

Department of Media and Communication, University of Oslo

Norwegian Media Technology Lab, Gjøvik University College


P.O. Box 1093 Blindern
N-0317 OSLO
Norway

http://www.media.uio.no   http://fagerjord.no






2. aug. 2014 kl. 02:16 skrev Daniel Jung <jung at uib.no<mailto:jung at uib.no>>:

Hi all,

I have a question, or a request for literature.

Short version: Is there a general taxonomy, or model, of text/image-relationships for web design?

Long version: In his excellent book "Information Design : An Introduction" (John Benjamins 2002), p. 40, Rune Pettersson postulates four relationships between text and media objects, such as images or film.

- redundant (similar information conveyed, such as sub-titles for TV)
- relevant (supplementing information)
- irrelevant (pictures and text and probably sound in TV programmes dealing with different things)
- contradictory (disastrous in information design, but possibly beneficial for persuasion)

This book describes a general framework and not specific design areas. The preface states explicitly that modern web design is not included.

I have been looking for a kind of general taxonomy of relationships between text and images in web design, but haven't found anything really satisfactory. I have found some novel-, or work-specific articles. One of the most interesting articles is Thomas Wartenberg, 'Wordy Pictures: Theorizing the Relationship between Image and Text in Comics' in Meskin's "The Art of Comics: A Philosophical Approach" (2011). In a critique of Scott McCloud, he claims four functions of text in comics: thought or speech; narration; pictorial element; sonic event. But this is too specifically targetted on comics.

I would like to see a model like Pettersson's, but more fleshed out, newer, and, above all, applied to (or coming from) web design. Does anyone have a pointer for me?

Thank you very much!
- Daniel

_______________________________________________
The Air-L at listserv.aoir.org<mailto:Air-L at listserv.aoir.org> mailing list
is provided by the Association of Internet Researchers http://aoir.org
Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org

Join the Association of Internet Researchers:
http://www.aoir.org/