Over the coming months I’ll be turning the best bits from my PhD thesis into chapters for a book. I guess I should have done this years ago, but now there’s stuff that seems relevant to many recent conversations I’ve been party to around augmented space and our attempts to get to grips with transformations in the experience of the urban and built environment.
Title: Media Transformations. To get the ball rolling, here’s the abstract, unedited.
Multimodal theory has developed out of social semiotics and can be seen as a response to the rise in the use of new technologies for the creation, distribution and consumption of media texts and the need to find new ways of describing and explaining their role in representation and communication. Its development is historical. It is a response to change over time. The incorporation of the visual into social semiotics marks a key moment in the development of multimodal theory.
Visual literacy is discussed in relation to changes in modes of representation and a critique of this concept is provided. This is conducted in relation to how the visual modality has been integrated into social semiotics as a platform for research into multimodal communication more generally.
Framing is developed along three main lines of enquiry (semiotic, cognitive and affective) as alternative ways of accounting for some of these shifts in communication and each are presented in the form of case studies. Framing and its close relationship with composition in media texts is discussed and this understanding, one that emphasise proximity as a multimodal principle, is applied to the visual design of content, the realisation of context through the provision visual cues, and later to embodiment and urban space. The three case studies, the application of framing to a range of media texts, the critical judgements made about the role visual in contemporary theory and the application of these concepts to multimodality are presented as part of an intellectual journey.