|CAT | ESP | ENG|
|Interactive multimedia open new possibilities for audio-visual creation. We present in this article an introduction to the theme which contains the background, the scope of contents, and several elements on structure and form. Finally, we contemplate its application to narrative.|
The latest offer from computers to the audio-visual field is interactive multimedia, which allow presenting texts, sound and images by selective action from the audience. Programmes resulting from the use of this technology are typically presented on a compact disc (CD-ROM, CD-I), but also at a distance by communication networks such as Internet. If we take into account the growing circulation of these two supports, we face a series of challenges, among them the writing, projection and design of these programmes. This study presents a series of considerations on these tasks, with the intention of approaching career scriptwriters to the new medium (1) .
Interactive multimedia is not a new technology for collaborating on an application such as text writing, video editing, or musical composition. Neither is it a technology in search of application as often happens with certain inventions. In fact, in the education ambit, it seems proven that interactive learning gives better results than passive learning. Stephen Wilson, an expert in interactivity, says (Wilson, 1994), "Interactive multimedia are seen as allowing users to follow their own associationist paths; to experiment and build on their own cognitive structures; and to link their actions with internal emotional and identity needs. In learning and information retrieval applications, the theories suggest the material will be remembered, used, and integrated better." Wilson himself continues, "In entertainment or art situations, the theories are extrapolated to predict that interactive events can be more profound and moving than non-interactive experiences".
Interactive multimedia is a set of tools for designing, assembling and delivering applications, that is to say, programmes.
This set of tools contributes to a "person-to-machine" relationship which is increasingly more fluid and richer in means. Its development makes this relationship possible in a two-way sense, consequently the "machine-person-machine" relationship or "person-to-person" relationship is also capable of being more fluid and richer in means. Interactivity is thus applicable to communication between people, in a gamut of contents which goes objectively from delivering instructions, through entertainment, to artistic expression.
Background to interaction
The issue of the machine, and especially the machine among people, has never been pleasing to artists. Contemporary art history is a Luddite history. Author and machine are incompatible, something only some Surrealists, Futurists, and very few others were able to disobey every once in a while.
There have been works specifically open to audience participation, such as those of the Dadaist theatre and of authors like Bertold Brecht, which would be a precedent to interactive works as to the will to defy audience passivity and staticism. Works of art known as participative (performances, happenings) of the 60's and 70's would also be precedents.
But the most direct background to interactive programmes is to be found in the world of computing, totally contrary to the Luddite world, where the technology which makes it possible has been hatched. In this world, the group of forms and movements by which dialogue is established between user and programme constitutes what is known as the interface.
The interface between user and computer was an idea only in the forties: dialogue with the computer was carried out by complicated operations such as changing plaques and electronic circuits. In the 60's, communication was established by means of tapes and perforated paper cards, an interface (if we can call it that) more practical but equally inhuman, because the language of the dialogue was the language of the computer.
The interface closer to a user than to a machine begins at the moment in which it becomes possible to digitalize images. As a consequence, the first graphic interfaces appeared, in the 60's, to which we owe the later acceptance of the personal computer in the 80's.
Currently, the interface is a key aspect of a computer programme: the success of a programme depends directly on the quality of its interface, to the point that when people say that they don't understand a programme or a computer, what they are saying in most cases is that they do not understand the interface. Because of this, interface design mobilises many programmers, psychologists, designers and experts under a speciality known as "Human-Computer Interface".
The prime ideas for interactive programmes and their reach are basically due to two people. On the one hand, Vannevar Bush who, in the 40's when computers were only useful for calculating bullet and bomb trajectories, imagined a digital system known as "Memex" for storing great interconnected bases of knowledge to which access was possible from diverse and multiple levels. On the other hand, Ivan Sutherland in the 60's invented image digitalisation and imagined glasses to move around within the images. Bush's ideas are the basis for modern systems of information exchange, closer to people's natural way of relating information, while Sutherland is the father of the technology which allows putting these ideas into practice and the first to discover the horizons of interactive communication.
Another character in this story is Ted Nelson, author in 1956 of the term "hypertext". Nelson also had the idea of another utopic data base, known as "Xanadu", like Citizen Kane's palace. Although "Xanadu" has never actually existed, Nelson's parallel reflections on methodological and linguistic order are essential for the development of interactive programmes.
What finally gave a start to technically feasible interactivity was the technology invented by Douglas Engelbert (the mouse, windows...) and Alan Kay (the first graphic interfaces) at the beginning of the 70's.
In the audio-visual ambit, the direct precedent to the interactive programme is interactive video. What makes interactive video go to interactive multimedia are compression technology and image transmission; one has the task of reducing information volume which images in movement imply, and the other to increase transference of information between devices. Actually, in the process of digital conversion, texts, image and sound are all converted into (miserable) bits of information; thus, basically there is only one medium and interactive programmes are "unimedia". As all are also multimediatic, the adjective is not called for and we can correctly name them interactive programmes.
Currently, the technical quality of interactivy is still quite limited: instead of on the whole screen, images in movement tend to be presented in a window on the screen at a rhythm of 15 or less images per second, which subtracts a great deal of attraction from the programmes. Besides, we must count on the low resolution of the whole screen, of the cathode ray tubes, the great handicap to digital audio-visuals till now.
But development of technology is evident and foreseeable and, besides, its possibilities as an audio-visual medium appear to be independent of its limitations. In this line, and to all practical effects, Celia Pearce, author of interactive programmes, gives the following advice (1994), "My suggestion to anybody designing interactives, particularly interactives which are story-based, is to forget about the technology as quickly as possible".
Characteristics of interactives
Leaving aside the technology of the issue, what are interactive programmes like? What is the nature of interaction?
Interactive multimedia propose a new reading of things, hence the considerable creative (and commercial) vein which they open. A book will always exist as such, and nothing can rival it as to format and content: poetry, literature and works in general in which there are authors and readers who specifically want the format of the monologue of the former to the latter. Besides the diversity of information supports, interactive rereading of a theme makes a lot of sense in itself for interactivity. Quite definitely, the medium must adapt to the content, and not the other way around.
Another premise is that an interactive programme must be interesting to navigate in, partly and wholly; if it bores the audience, if it doesn't promote interactivity, there is no programme. Besides providing information, an interactive programme must offer entertainment, must constantly satisfy interest. Summing up, the final objective of the voyage is as important as the road which leads there. That is to say that contents, besides allowing diverse focuses, points of view and slants in which to delve, must also dispose of abundant and attractive audio-visual resources.
A first way of characterising the contents of interactive programmes is based on the amount of interactivity they require. Among those which require less we find, for example, electronic books; those which require more are games; in between, we find educational and documentaries. A young audience appreciates interactivity more than an adult one, while the latter generally prefers contents with a narrative line. A museum-type content (generally informative) must leave time for contemplating and must not force audience over activity.
But it is clearer to speak of interactivity in qualitative rather than quantitative terms. Interactives are differentiated also by the kind of "required interactivity". The lower level corresponds to programmes offering reduced and simple options. Afterwards, we can find more complex requirements, such as overcoming obstacles to accede to other options. Higher up, options are not explicit so that, for example, people have to search for them. When the interaction required is qualitatively high, known as "contributory" (Wilson, 1993), people can add options for themselves, or can even have conditions for modifying or creating programmes for themselves.
Another way of distinguishing interactives is according to the capacity for "control" they give people, the degree of "autonomy" they allow for deciding what to do, where to navigate, etc. When the control possible is low, paths to choose are restricted and fixed, as in programmed teaching. In other cases, people have a certain control, but only in specific areas of the informative space. Finally, when the level of autonomy is high, access is possible to the whole space, by means of an over-all index, for example, which leads to the atoms of information, as is the case of encyclopaedias and reference works.
If we therefore take these two variables, the quality of interaction required and the autonomy and control allowed to the user, we have a characterisation of interactive programmes (fig.1).
Figure 1. Map of the actual interactive programmes
However, in view of the strong development of digital technology and future inventions and gadgets, we need a third dimension to characterise interactive programmes.
In fact, in the future there will be helmets, high-resolution glasses, gloves, suits and ultra-light sensors which will allow interacting with the computer in a practically transparent manner, involving different parts of the body and affecting several senses and ways of perception. This development can be represented on a third hub of co-ordinates which measure the "presence" and personal user involvement, the degree of "immersion" in images and sound.
Thus, if we take the three variables, interaction required, autonomy and presence as co-ordinate hubs, we have as a result a three-dimensional world (fig.2) which represents current and future interactive programmes.
Figure 2. A perspective of the interactive programmes (2)
When the sensation of immersion is high, interactive programmes are (or will be) known as "virtual environments". The co-ordinate centre of the interactive universe corresponds to the interactivity which in computing is known as "batch", while the maximum interaction, autonomy and presence zone constitutes a terra incognita open to speculations of all sorts.
This new presence/immersion hub is also correlated to another order of interactives which takes into account the kind of signals used when communicating with the computer.
As we know, Charles S. Peirce's semiotic analysis distinguishes three kinds of signs in communication: symbols (signs in which the relationship to the object is given by convention), icons (signs in which the relationship to the object is given by resemblance or analogy) and indexes (signs in which the relationship to the object is given by implication or contiguity).
With this idea in mind, author Paul Brown (1994) proposes a classification of interactives under the following categories: there is a "first generation" of interactives, in the era of circuits and perforated cards, in which communication was established based on words, that is to say, symbols. Current interactives, those of "screen and mouse", belong to a "second generation" and are based on icon communication above all. The next generations of interactive programmes (virtual environments and "realities") will above all use a language of indexes.
Metaphor, Structure, and Form
The interactive programme scriptwriter's aim is not a unique and closed work, but rather an environment and a context (3) , through the length and breadth of which information of educational, documentary, play or whatever content is delivered. This environment must have a conceptual and graphic representation, that is, the "metaphor" which welcomes users and nears them to the interactive experience.
The first time communication with a computer was established by means of a metaphor was on a graphic interface developed for the Xerox corporation; but Apple Computers, in the first place with a Lisa computer, but above all with the Macintosh, was the company which made it popular. The Macintosh interface metaphor is a desk on which the user manages, as in reality, documents (texts, graphics or scores), files where these documents are stored, a wastebasket where things with no interest are thrown away, etc. Thanks to this metaphor, along with a good graphic quality, there is a before and after in personal computing after Macintosh; the proof is that it has been adopted by all operative systems (4).
In today's idea of the metaphor, there is as much freedom as in any other design field: there is space for imagination, although it is useful to follow some recommendations (fig.3).
Figure 3. Guidelines for the metaphors design (5)
Metaphor must have enough audio-visual resources to make navigation attractive, because a poor conceptual and graphic metaphor makes the whole programme poorer.
The higher the interactivity required, the more powerful the metaphor, but the golden rule of interactives is that we must avoid the users getting lost in an excess of symbols; when this happens, the programmes' interest takes a dive.
Besides the main metaphor, corresponding to the total navigable space, an interactive programme normally also needs metaphors for subspaces and specific zones. It is also useful to define some of them for our own interactivity, associating it to the user's experience in travelling.
In this experience of travelling via computer, people's actions become real by relatively simple movements, at least with current systems. Possible actions are now: look/listen, type messages, handle a joystick, drag/click the mouse. In the future, we suppose that it will be possible to interact with other body parts, not only the hand. The computer responds, after a series of user actions, in three ways: playing an audio/video piece; presenting an image with options; or presenting a terminal image.
All these events, and the contents with which they associate, must be organised and structured in the form of a tree, in the manner of computer information flux diagrams. The possibilities of relating between events, the way of structuring in trees, are very varied (fig.4).
Figure 4. Some interaction patterns (6)
The information tree is thus a kind of visualisation of the interactive programme's script as to its structure and the dynamics by which it is displayed before the user.
All this part of the script of an interactive programme: the metaphor, the structure and the function dynamics, make up a universe which must be, in general terms, conceptually and graphically coherent. Some authors have elaborated a series of guides for designing interactives from a practical point of view (fig.5).
Figure 5. Some guides for interactive programmes design (7)
Franca Garzotto, investigator of interactive design methods, proposes a few general criteria for evaluating a programme's quality in these aspects. Among these criteria, Garzotto (1995) notes in the first place the programme's "richness", that is to say abundant information elements and paths to access them. In the second place, the "ease" of use, or the accessibility of information and the simplicity of the operations which lead to it. "Consistency" is also desirable, the programme's regularity, the similar management of similar elements. The user must be able to guess meaning and purpose from any element of the programme, its "self-evidence". Finally, Garzotto notes another evaluation criterion of an interactive's design: its "predictability", the capacity it gives the user to anticipate operation results.
The interactive medium adapts well to information, education, documentary and play effects; this alone would justify the industrial and professional expectations which it raises. Now, how do we adapt to fiction and to drama, how does interactivity affect narrative, what are the cinema possibilities of the new medium?
Interactivity implies "non-linear" narrative, as opposed to the usual "linear" and sequential narrative, known since the Greeks wrote the first comedy. But narrative, intractive or otherwise, and even if it be delivered intermittently in time, is received by the audience in a linear manner; therefore, all narrators thus face the same problem: unfolding a series of events along a line of time. The only difference is that in an interactive medium, more than one unfolding must be foreseen, many must be foreseen, and the more varied the better. A calligramme (fig.6), for example, would be a very elementary sample, a kind of flash, of a diversified work which allows many readings.
In the tale El jardín de los senderos que se bifurcan, the writer Jorge Luís Borges tells us of a Chinaman, Ts'ui Pên, the author of a unique book which is like a labyrinth, like a "labyrinth of symbols". Borges (1978) writes, "In all fiction, each time a man faces several alternatives, he chooses one and eliminates the others; in that of the almost inextricable Ts'ui Pên, he opts, simultaneously, for all of them. He thus creates several futures, several times, which also proliferate and bifurcate. The contradictions of the novel spring from this". Therefore, El jardín de los senderos que se bifurcan would be an ideal "non-linear book".
Figure 6. Calligramme from Joan Salvat-Papasseit (1962)
Audience attention in a game such as "Myst" is captured by rousing curiosity to solve an enigma and by causing more enigmas to rise on the path to its unravelling, with more obstacles and surprises. One way, therefore, of directing interactive fiction is inciting the audience "to discover the story" which is hidden. This method is applicable above all in detective and mystery plots.
We can also set up interactive fiction based on "alternative sequences" which the user has to decide on the way or at the very end. This method, for which there are precedents in some novels (known as "interactive"), has also been tested on TV shows.
However, it is more productive to intervene, besides the plot, on characters and protagonists; in this sense, a first possibility is "role playing". In games of this kind, characters are loaded with usually simple tools (such as a rucksack, some keys, a ladder or a hammer) with which to face the obstacles of their adventure. The idea can be applied to personal qualities and abilities, so that according to how these qualities are graduated, the individual stories or the overall story become different. Role playing has an inconvenience in that it requires a great deal of interactivity which is not to every audience's liking.
Another method can also be to present the story from the diverse "multiple versions" of the protagonists. The story develops as we get to know the different points of view. In a simple application, which is what interactive television can aspire to, the method has been tested having audiences zap between two or more channels which are each broadcasting different parts of the same macro story.
A last (but not least) model is "plot building". The "Sim-City" game is an example of this: the player has a series of tools to choose a field, build streets, lay electricity, build houses, factories, etc., but this colonising zeal has its disadvantages. The results can be interpreted as a narrative with infinite endings, but evidently this model is useful above all for games.
Figure 7. Some basic patterns for non-linear narrative
All these patterns (fig.7) for the interactive planning of a narrative can be combined among themselves, but, besides this, there is another possibility which multiplies them, both these and any other: the user, besides contemplating and managing certain narrative conditions, can also be a protagonist of the story.
In this case, the narrative becomes notoriously complicated, because possible plots are much more numerous. When foreseeing them, we must pay a lot of attention to this user/actor's point of view, as this is the most delicate character of all. Certainly interactive fiction in which the user is also actor conjures up a clash between author and user, with overtones of romance, about who is in control of the game, and, finally, of the work itself.
To sum up, stories which mesh into each other, and multiple plots which divide and rejoin. The complexity of an interactive narrative, as we can see, can be diverse, but Celia Pearce (1994) notes: "Even though the story may be non-linear from the system's point of view, the player's experience, even if it varies every time, should conform, on a emotional level, to the traditional elements that work in story structure"
A new medium
As an interactive programme presupposes a relationship with a user and an interposed screen, as in a cinema programme, some authors consider that there is a close relationship between the interactive medium and the cinema medium; in this sense they compare the plot demands in both media.
Ted Nelson (1990), father and guru of interactivity, says, for example: "Designing for the little screen on the desktop has the most in common with designing for the Big Screen (directing theatrical films). Interactive software needs the talents of a Disney, a Griffith, a Welles, a Hitchcock, a Capra, a Bob Abel. The integration of software cannot be achieved by a committee, where everyone has to put in their own addition (featuritis again). It must be controlled by dictatorial artists with full say on the final cut". Nelson also says that in the future it will be necessary to speak of "virtualities" to refer, at the same time, interactive programmes, cinema/television programmes, and computer programmes. In the same visionary line, other authors forecast the computer as "plot generator", that is, interactive systems with which the show will consist in to make shows (8).
Returning to reality, the current interactive media situation reminds us of when cinema only portrayed reality, and editing resources did not exist; we could say that as to impact and meaning, Lumière's train is to the cinema what the interface and the desk are to interactives. We must thus wait for linguistic resources to become defined and solid.
But beyond confluences and parallelisms, the interactive medium is substantially different. Cinema narrative is rooted in verbal and literary language, that is it follows the sequential and linear model of discourse. On the other hand, interactive narrative proposes a discourse in tree or mesh, rather than linear, form, and involves (and will involve even more) several senses at the same time.
On the one hand, the communication properties of the interactive medium make us think of a renaissance of the documentary genre as an extension which gives plurality in study levels and sensations felt by the user.
As to fiction and expression, the medium's specificity allows us to imagine, in the long term, interactive programmes different from passive programmes in the same way as cinema distanced itself from theatre or photography from painting, that is to say, becoming a specific genre.
1 For an introduction to interactive multimedia from other points of view, see, for example, Berenguer (1994) and Rickett (1993).
2 Inspired on an idea from David Zeltzer cited by Van Dam (1994)
3 Brian Eno suggests speaking of "unfinished" rather than "interactive" works.
4 For a comparative study of the metaphors of the Macintos operative system desk and other systems, see Marcus (1992).
5 Adapted from Halskov (1994)
6 Adapted from Wilson (1994)
7 From Wilson (1994), adapted from Greg Kearsley. See too Marcus (1992) and the classical book from Apple (1992)
8 See for instance Laurel (1987) and Don (1990)
Apple Computer, Inc. (1992) Macintosh Human Interface Guidelines. Addison-Wesley, Reading, Massachusets.
Berenguer, Xavier (1994) L'opció dels continguts, Culture Magazine, Generalitat de Catalunya, September, 1994.
Borges, Jorge Luís (1978) Ficciones. Alianza Editorial. Madrid.
Brown, Paul (1994) The ethics and aesthetics of the image interface. Computer Graphics, v. 28, 1, February, 1994.
Don, Abbe (1990) Narrative and the Iinterface. in Laurel (90).
Garzotto, Franca (1995); Mainetti, Luca & Paolini, Paolo Hypermedia Design, Analysis and Evaluation Issues, Communications of the ACM, v.38, num. 8, ACM, New York.
Halskov Madsen, Kim (1994) A Guide to Metaphorical Design, Communications of the ACM, v. 37, 12, December 1994.
Hayward, Philip & Wollen, Tana, (eds.) (1993) Future Visions, New technologies of the screen, British Film Institute, London.
Laurel, Brenda (1987) Computers as Theatre, Addison-Wesley, Reading, Massachusets.
Laurel, Brenda (ed.) (1990) The art of human-computer interface design, Addison-Wesley, Reading, Massachusets.
Marcus, Aaron (1992) Graphic Design for Electronic Documents and User Interfaces, ACM Press, New York.
Nelson, Theodore H. (1990) The right way o think about Software Design, in Laurel (1990).
Pearce, Celia (1994) The ins and outs of non-linear storytelling, Computer Graphics, v. 28, 2, May 1994.
Rickett, Frank (1993) Multimedia in Hayward (1993).
Salvat-Papasseit, Joan (1962) Poesies, Ariel, Barcelona.
Van Dam, Andries (1994) Vr as a forcing function: software implications of a new paradigm, SIGGRAPH'94 Course notes, ACM, New York.
Wilson, Stephen (1994) The aesthetics and practice of designing interactive computer events, Multimedia'94, ACM, New York.