The basic starting point of this essay is to ask: what is audiovisual thinking? The rise of audiovisual media culture has prompted educators to develop strategies for media literacy but at the same time university level education seems to the most part ignore audiovisual expression as means of communication and publication of scientific knowledge. In that perspective "media literacy" is a limited project focusing on pedagogy mainly for youth in helping to create "critical autonomy" as a "defence mechanism". As Elizabeth Thoman founder of Center for Media Literacy[2] puts it: "goal must be to help people become competent, critical and literate in all media forms so that they control the interpretation of what they see and hear rather than the interpretation control them".[3]
Thus the concept of literacy here emphasizes literacy as an act of reading and interpretation. I believe however that an essential part of literacy is also the skill of writing. What makes true media literacy is not just ability to read but includes also the ability "write" media.
In this essay I will limit my considerations to the visual side of audio‑visuality and conduct an experiment of using image as a carrier to conceptual ideas. The example image comes from advertising. I will study the image as an example for expanded conceptual interpretations. The basic idea is to project experimental interpretations on the image. Next step as "a reverse strategy" would be to create images that are experimental visualisations of concepts. In other words these examples of interpretative practices could further be applied to creative use of images as part of conceptual writing where the aim is to "write" with images.
If there is such a thing as audiovisual thinking or visual thinking, it should be possible to conduct philosophy with images. I am not proposing a style of thinking devoid of verbal expression. This would not be necessary since the context of this experiment is digital multimedia, which makes possible composition of images, texts, sounds, video and animation (to name the most basic media elements).
My research on this subject started with questions raised by Gregory Ulmer in his book Teletheory. The basic question for Ulmer in his book was: how to make an academic audiovisual essay?
He is proposing that university humanities should start producing theory using audiovisual media. To give this proposal more historical authority he uses analogy to the invention of the printing press:
"During Renaissance, humanists led the educational reforms associated with the rise of literacy and the new technology of the press. Humanists today are no less responsible for developing the educational potential of the new technologies of memory and communication. Following the lessons of our Renaissance counterparts, this responsibility is two-fold: first to translate into the "vulgate" (audio‑visual writing in the formats of film and video [..]) the principal works of the disciplines of knowledge; and second, to develop new genres that will serve educators in the electronic era as well as did the literary essay in the Gutenburg era." (Ulmer 1989, viii‑ix)
Just as Ulmer himself has done in his later writings (Ulmer 1994), it is worth repeating that what he calls "vulgate" obviously refers to digital multimedia as well as "traditional" audiovisual media (tv, cinema). The first responsibility that he presents seems quite obvious but the second responsibility "to develop new genres" needs more clarification. Ulmer's proposal for a new genre is what he calls "Mystory". Before I discuss "Mystory"-genre I would like to point out some background points on media culture and content production.
As a motivation for making "audiovisual theory" Ulmer points out the obvious fact that television is the major "educational" institution of our mass‑media culture. If we could communicate the insights that universities and science are producing into an audiovisual format, it would be a considerable educational service to the culture at large. Somehow this hasn't quite (just yet) happened. If I go to the video rental store and look at shelves, I can see different genres of movies: action, comedy, romance, musical etc. but for some reason I don't see videos in the "genre" of sociology, philosophy, mathematics etc. This only shows that the communication "revolution" (or evolution) in the branches of academic disciplines hasn't really even started yet. One might argue that there would not be much market for these "genres" but since there is market for these genres in the printed book format, I believe there could be a market in audiovisual format as well. Do we have to assume that book is inherently better for theory and science? Is it more about cultural inertia that works in academic institutions? Or is it just a question of more expensive production costs? It is hard to see that any commercial enterprise would lead the way in this matter so the key player to take initiative should be university. The problem might be that for this to happen there major changes are needed in the way disciplines define their methods and ultimately themselves.
I believe Ulmer presents couple of hints on how to proceed with this challenge and how to extract or apply theory in a creative way, in other words how to bring practice and theory together and more specifically in this context: how to bring together and into interplay university disciplines and creative challenges of new media production.
At the moment I will limit myself more to the point of view of new media education. Could there be something in "mystory" genre that could be applied in new media education? As I see it, one of the most essential challenges in new media education is to discover ways to bring together theory and practice. Another challenge in new media education is to invent pedagogy that supports creative "content production". It seems that Ulmer in his Mystory genre gives some hints on how to approach these questions.
While teaching interactive media in various media schools since 1998, especially in University of Art and Design (Media Lab) it has been my perception that in new media education there is more emphasis on mastering technological aspects than there is on the content. This has been true in my own courses and workshops as well and this study is also motivated by the attempt to make shifts in that. Obviously it is much more straight forward to teach technological competence than to "teach" creative content production. I don't believe that it is possible to "teach" content production in the very traditional sense of the word "teaching".
Nevertheless I am attempting to sketch preliminary ideas for creative learning frameworks of pedagogical practice. In the context of education I prefer using a term of "mastery of content" instead of "content production". While "content production" refers to the actual process of creating new media content, the role of education should be first of all to provide means to develop something what I would call "mastery of content". By this I refer the ability to express ideas, ability to create original audiovisual expression, to compose, to write, to screen‑write fictional, documentary or rhetorical narratives, to master audiovisual and interactive rhetoric and expressive possibilities. I feel that the real challenge in teaching interactive new media is in the question of how to bring together and in the balance mastery of technology and mastery of content. My research on Ulmer's ideas is also research on possible pedagogical strategies to be tested for these challenges.
In his book "Teletheory" one of the basic questions for Ulmer is the question of how to make an academic audiovisual essay?
His idea is that when one starts to translate theory (that traditionally works in phonetic alphabets) into audiovisual expression we should invent a new writing genre that can manage the rhetorical dimensions of this new discourse since it contain richer possibilities of expression integrating features of oral, literate and "video" discourses.
He proposes mystory as a genre of academic discourse that could function across all of these media ‑ voice, print and video (Ulmer, 1). In most elementary terms mystory is a collage and intermixing of three different kinds of discourses: personal, popular and expert discourse. In this process he suggests "sampling" as the mode of operation, using a metaphor coming from popular music. (Ulmer, 13)
Mystory is also about popularizing science and making this process creative by using personal and popular texts and images as vehicles of expression. One might certainly object that clearly mere mixing of these ingradients does not yet make process creative. I am not taking Ulmer's idea as a receipe for creativity. But I believe that the idea of mystory is an interesting challenge and in principle could also work.
Popular discourse could be said to be the new "alphabet" of audiovisual culture. It contains a wealth of communicative signs and expressions that are recognized immediately. (I would argue that advertisement is probably the most universal popular culture. Even if one decides not to watch tv or to listen radio or go to movies, you can hardly escape advertisement as billboards fill the streets and walls of urban landscape.)
Ulmer argues that this mystory genre as "collage" of genres should include personal discourse as well. I think it is important to understand that Ulmer is primarily trying to invent a pedagogy where the first goal is really not to make science but to facilitate invention and creativity. He points out that some case studies in scientific invention demonstrate how personal anecdotes are vital in the creative process and that their repression in the final scientific essays does a disservice to pedagogy (Ulmer 1989, 32).
In the new media content production industry, in all genres, the basic most important resource is creativity: I would argue that the most important condition for creativity is motivation. I believe that there can be motivation only if "one knows what one wants" and has a sense of purpose and goal in the work. In the pace of information overload and constant software upgrading, focus on this kind of self‑knowledge can often be a secondary preference as the process of learning the craft of expression unfolds. Therefore, I believe that in all creative processes there should be some ways to facilitate or support that very intimate process of self‑discovery, which makes the whole business of art and new media education so very challenging. Personal discourse, diary, autobiography is one of the most basic ways to support this process of self‑discovery. Furthermore if one is able to express the relevance of insights in the human science for the personal experience, it makes a significant difference in the rhetorical effectiveness of scientific discourse. This is particularly so if one is popularizing human sciences.
Ulmer argues that in this new media environment there is emerging something he calls "electronic cognition". According to him it works like mystory. At this point I would like to present a little example of what this "new" (?) mode of cognition could mean in practice. The basic starting point in Ulmer's project is to make theory with audiovisual signs. In my example I attempt to analyse an image as a carrier of theory. (We can say that the image is a carrier or vehicle of theory when it is metaphor or an allegory for theory, or a stimulating tool for thinking).
The example of this analysis is a marketing content. It is an extract from advertisement published in Helsingin Sanomat (major finnish newspaper)
This advertisement advertises a cultural festival in Helsinki last year. The caption of the image says: "Become who you are". In the first glance the message seems to be that art (in this cultural festival) would help man to get a glimpse of something new in himself. One would interpret that the shadow represents this "new aspect". This new aspect was hidden before the "confrontation with art".
This slogan combined with this image already contains a certain visual "theory of identity". I will experiment with this image testing further its ability to work as an allegory for theoretical ideas. I will take some theoretical hypotheses on the relationship between media and identity and some fragments from psychology of C.G. Jung.
We can say that media content (a narrative for example) when it is high quality can give us new perspectives and thus somehow contribute to our self‑understanding (and this is precisely what the add is trying to hint). On the other extreme contemporary media content can create (fake) fabrications, synthetic identities and roles that are designed only for creating consumers in the market. In this image above we can see that the shadow is not "true" to the original. So one can also interpret this image as counteracting the very message it is trying to convey. If we attached a slogan "become who you are not" the image would support that slogan even better than the "Become who you are". As this makes it perfect example of the potential ambiguity of images it makes it a carrier of the idea that the role of media for our attempts to construct our identity is also ambiguous.)
In media culture most basic example for this kind of identity fabrication could be an advertisement that "says" subliminally or less subliminally that if you buy specific clothes you are going to be "a cool, sexy, attractive, stylish" person. As media culture theorist Douglas Kellner is saying, media representations provide the basic building blocks of our identity.
This is our theoretical issue in our experiment: the role of media contributing to the ways people construct their identities.
We see the shadow in a frame. The frame could be said to stand for art or for the culture festivals as a frame. For my purposes it represents the frame of media technology. As we saw there can be at least two seemingly contradicting proposals: media providing new perspectives for our self‑understanding or media creating fake fabrications that only contribute to a chronic identity crisis.
But as the text says: "Become who you are", it is actually suggesting even more. The artistic experiences that this culture festival is offering, are not only going to give some new perspectives of your self‑understanding, they are is also going to transform you into the original reality of your identity. We should thus assume that there has to be some more original identity within ourselves that we should restore. We have now three interpretations:
1. (Art &) Media giving new perspectives to our self‑understanding
2. Media fabricating synthetic counterfeit identities[4], or
3. Art as transformation back to the original Real identity.
As images tell more than thousand words we can have all of these interpretations. Images can have contradictory interpretations. Certainly I am now also projecting these interpretations to the image.[5] Since I am using the image as a pictorial vehicle for these propositions it is more than just interpreting the image, it is about writing with the image and many of these ideas were already suggested by the advertiser as well.
As we are looking at the picture, it seems to say to us: by looking at good art we are transformed back into our original essence. But in the process of this transformation there is obviously a crisis. There is an inner conflict. Our shadow is not behaving as it should. It takes liberties. It seems to be less disciplined than our physical body. The text says: "Become who you are." This suggests that there is something more to ourselves than meets the surface. And it seems that this shadow is referring to that "something more". It is kind of representing the "real self" of this man, or is it?
Psychologist C.G. Jung has a theory of psyche where "shadow" is the name for our unconscious aspect. "The shadow is a moral problem that challenges the whole ego‑personality, for no‑one can become conscious of the shadow without considerable moral effort" (Jung 1998, 91)
Jung says that it is not easy for a person to recognize the shadow: "it is composed for the most part of repressed desires and uncivilized impulses, morally inferior motives, childish fantasies and resentments etc. all those things about oneself one is not proud of."[6]
We approach a sensitive question as to how can this "shadow" represent our real self if we are to believe that our real self should be free from all that "one is not proud of". In this case the image with its slogan seems to be fatally misleading.
Alternatively we can speculate that for us to know our real self we have to confront the "shadow" side of our being. With that proposition the image and the slogan seem to be in harmony.
As I am attempting to conduct this experiment to write with this image, it becomes more and more clear that this image is as much a stimulant for all of these speculations as it can be made a pictorial carrier for these speculations. There is a constant feedback loop between the interpretation and "appropriation" of the image. As one proceeds in this kind of analysis, it is easy to recognize how rich and multifaceted connotative world of interpretations can be generated from a reading an image. This mode of analysis merges into "a writing and thinking with the image" and I dare to suggest that this points to a creative potential not so fully exploited in our culture, which has to do with the interplay of theory and visual expression.
I hope that with this example I have succeeded to demonstrate that in new media education and new media content production and virtually in all audiovisual culture the way we use images as part of our thinking could be considerably expanded.
Is it possible to philosophize with images? I believe so. I believe that research on how to use images together with verbal language (especially the "verbalisations" of theoretical disciplines) is one of the most vital needs of the hour for the evolution in media culture. The institutional problem seems to be that the academic human science is rather slow to embrace audiovisual expression and the schools that teach audiovisual media still tend to have a bias for teaching technical skills. But for there to emerge audiovisual culture rich in content there should be more vivid interplay between these skills.
The experiment above is one elementary example of a way to start this work.
How does this relate to Ulmer's mystory genre? My hypothesis is that for us to take up the challenge of mystory-genre the first step would be to look at interrelationships between concepts and images. I am not so much applying autobiographical elements in this text. To make mystory in the sense Ulmer is suggesting the next step would be to graft autobiographical text into the discourse. But it is not so easy to shift into autobiographical anecdotes from a scolarly style. Scolarly style has embedded assumptions that the personal should be repressed.
For popularizing theory however, I don't think one has to buy the Ulmer's recipe as such in order to make theory more engaging. In my understanding the essense of "mystory"‑genre is recontextualizing appropriation, metaphor and allegory. As the experiment shows the connotations of images can turn the image into metaphor of these connotations and this should be done by "appropriation", by sampling or relocating the image into the new context defined by the connotations of the image.
Jung, C.G. (1998) The Essential Jung Selected Writings. Fontana Press, London.
Kellner, Douglas (1995) Mediaculture: Cultural Studies, Identity and Politics Between the Modern and the Postmodern. Routledge.
Ulmer, Gregory (1989) Teletheory ‑ Grammatology in the Age of Video. : Routledge,
New York & London.
Ulmer, Gregory (1994) Heuretics ‑ Grammatology in the Age of Video. Johns Hopkins Univeristy Press.
[1] This is a reworked version of a presentation I gave in "EUROPRIX Scholars Conference 2002" originally with a title: "Mystory ‑ formula for creative content production in interactive media?" (EUROPRIX Scholars Conference 2002
"Understanding the Future of European e‑Content Industries"
Tampere, Finland, November 14‑15 2002
http://www.europrix.org/europrix/academy/scholars/)
Since 2002 I noticed that it is necessary to outline the basic starting point which has to do with audiovisual media in general before attempting to analyse possibilities that interactivity would provide. The subject of this text at hand is thus limited to those basic issues reflecting the use of image as conceptual vehicle. For a broader account on the possibilities of interactive media refer to my presentation "Human-Sign Interaction - Anatomy of Interactive Audiovisual Machine and Its Expressive Possibilities" a paper presented in Cross-overs in Audiovisual Arts and Interactive Media - June 7-9, 2004, University of Art and Design Helsinki
[2] http://www.medialit.org/
[3] Elizabeth Thoman: "Skills & Strategies for Media Education http://195.37.181.70/nubb/lern_mat_1/proj_med/literatur/artikel/thoman.htm
[4] "About‑Face" is a good example of a site that seeks to expose what I would call "synthetic counterfeit identities". (http://www.about‑face.org) Usa today netsite writes: "About‑Face aims to combat negative and distorted images of women. And its Gallery of Offenders names names." http://www.usatoday.com/tech/2001‑05‑07‑hotsites.htm
[5] All of the following expressions "appropriating image", "projecting interpretations to the image", "re‑contextualizing the image", "using the image as a carrier of theory", "using image as metaphor or allegory for theory" refer to the same kind of operation: situating the image in the context of connotative interpretation or theoretical proposition that "resonates" with the image and thus re‑places and reconfigures the "immediate" interpretation (if there is one) of the image. Here the example of this is the image above in context of Jungian proposition: "Shadow is the name for our unconscious aspect. The shadow is a moral problem that challenges the whole ego‑personality".
[6] JUNG LEXICON A Primer of Terms & Concepts DARYL SHARP http://www.cgjungpage.org/jplexiconq2z.html