Introduction to Drama and Artificial Intelligence

1.
This round table has been designed to provide a broad approach to the use of digital technology in theatre and performing arts. The issue about the relation between theatre, performance and new media, has been arisen since the Eighties, when contemporary dance started to introduce robots and electronics on stage, or when fringe theatre investigates the relation between live event and TV camera and monitors.
Since then, much has been written about it, so that it is now possible to follow different approaches to the question.
Here the panellists will talk about different topics: new dance and new media, the theatre and virtual reality, the simulation of space and time in virtual representation, the interactive stage and costumes. All of these are topics that deserve a full length discussion, and certainly this is just a start, and I invite whoever is interested to come and join us at tomorrow morning workshop, at 11 am, here at Supreme Council for Culture, for further details.

2.
Now, let me spent few minutes to discuss how theoretical links can be drawn between drama and artificial intelligence, so that more effective creative combinations of the two can be developed. The points I will try to make are:
1) Digital performance isn’t about technology but about language – it is aesthetic rather than prosthetic.
2) Theatre and Digital Multimedia are both live events, happening in the ‘here and now’.
3) Drama and Artificial Intelligence share much in common because the basic element of both disciplines is “actions” over, for example, ‘thoughts’, ‘words’, ‘descriptions’.
Often digital media has been seen from a too specific technological point of view. We should all agree that the question about digital theatre isn’t about increasing the quantity of gimmickry on stage. And we should because it is well known that theatre, although too often associated with poetry and literature, has always confronted new technology. As mere technology, digital media don’t represent any significant innovation in theatre.
This leads us to consider the clash -or the relation – between digital media and theatre from a more abstract point of view. Critical theory on the subject – particularly Brenda Laurel and Janet Murray – has clearly stated that in both cases (theatre and digital multimedia), the core is an event: a live event. Digital multimedia is closer to the theatre based notion of hic et nunc, – the here and now – than to the way we show images on screens.
Whereas some literature on digital media, for example Lev Manovich, grew out of the field of film studies and therefore tended to link digital media with cinema, later studies linked digital media more closely to computer studies and science. I would further suggest that the primary element in both computer science and drama is the notion of procedure and algorithm. Both are ‘procedural’ disciplines, based on a kind of algorithm.

The definition of Algorithm “is a finite list of well-defined instructions for accomplishing some task that, given an initial state, will terminate in a defined end-state”.

In other words, it is a set of input-output behaviours. When we run software, any software, on a computer, we are running a procedure that will make an event happen, a specific event, at that time in that space. Could we see drama as something similar? That the software is run by an operating system on a computer as the playscript is run by a director and actors on stage?
In semiotics it is said that digital representation increases the separation that already exists in video or filmic representation between the real thing and the represented one; up to the point where the digital image is quasi totally auto-referential. The mathematical model, situated within the representation process, between the signs and their references, is a diaphragm that moves the two terms away from each other. The images generated by the digital procedure will be inevitably auto-referential. Beyond a mere simulation of the real world, these signs refer to the binary code and to the software that interprets it. What we observe on the computer screen may not have any relation with the real world (if not a simulation, where – we know – the interpretation is more important than the signification). When we see a world in CG 3D, no referential object exists; only the idea of the creator (and again, i.e. his interpretation of the real thing). We cannot even suppose a distant relation between what we see and what really exists.
For example, watching a tree on the TV screen, there is a representation of the distance. The tree doesn’t share our time or/and it doesn’t share our space.
With the computer, the image of the tree is an event, because it takes place in the very moment in which we watch it. That specific string of data will be read on the fly – instantaneously – by an apposite plug in, thereby generating what we see, far from any referentiality. That tree “is” in that very moment and in that very place, as we are. The computer screen doesn’t show object but generates our own sight, the act of vision, because it happen only within the polarity of object and subject. It is, in its very nature, an interpretative act: the thing takes apparent form only in front of us. Such auto-referentiality is common to theatre too. The happening hic et nunc of the theatre performance prevents – or minimises – the representation of distance we have mentioned above. The performance, although refers to ancient stories, faraway worlds, happens always in the very moment we are watching it. The actors and the stage generate the event hic et nunc. Hamlet “is” every time we see him on stage. However, when you watch Laurence Olivier playing Hamlet in the film, you are watching a narrative, in the past, from a distance. Unlike cinema, «Drama is primary. It isn’t the representation (secondary) of something (primary); it represents itself, it is itself». This is the sequence of presents, that Peter Szondi identifies as the root of the drama.
Within this polarity of media (cinema, TV, theatre, digital), divided according the representation of the distance from the pre-existent object (then and elsewhere of the reproduced image) and the complete absence of that distance (here and now) – the virtual reality, the computer graphic and the other digital representations belongs to the latter side of the polarity, together with – as we said – the theatre, and – in a wider sense – the performance.

3.
Hence, there are – even if on a preliminary investigation – some interesting contacts, from a theoretical perspective, between the live event which is theatre and digital multimedia. So far the relation between digital multimedia and theatre doesn’t appear as a clash of opposites but rather a delicate attraction between equals

Both are live events, but this doesn’t mean that the actual appearance of the event has to be the focus of our interest. Too often the performance draws too much attention to the event itself, and you may end up talking about technology, rather than aesthetic. If on one hand it is worth illustrating the phenomenology of how the theatre, the stage, has been invaded by digital technology; on the other hand this could lead to a dead end road, to a long list of names and applications. Soon or later we discover that – as I said – the theatre event, the spectacle, IS, and has always been, all about technology, since its birth.

Aristotle wrote that the sixth element of the tragedy, «Spectacle, [opsis – the look, the appearance] while highly effective, is yet quite foreign to the art and has nothing to do with poetry. Indeed the effect of tragedy does not depend on its performance by actors, and, moreover, for achieving the spectacular effects the art of the costumier is more authoritative than that of the poet».

Although Aristotle is no longer the only authority, as he was in the Renaissance, and the contemporary aesthetic has moved a long way forward to investigate mass culture, collective creation, disputing the importance of ‘the author’, etc., let’s agree for our purposes here that the author is important. That it is an author (the ‘programmer’) who designs the theatre performance. So, behind theatre and behind digital performance there is a specific language, a code that the author uses to write the procedure that will be run in a certain time and space.
Drama, even from an Aristotelian point of view, is a set of rules to be run on stage. Here emerges a more profound connection between drama and algorithm. Drama is, by nature, procedural.

4.
In the last ten years, as the multimedia boom has started to settle down, a few scholars have started to focus on what has been called interactive drama. The aim was to formalise the rules of a dramatic event so that it could be encoded in an algorithm (to be performed interactively by a computer). Although it may sound like the scientists are trying to teach the computer how to write a play, that is not at all how we perceive it. We see it as a question of language.
As we said, drama isn’t only something that happens in a certain moment. It is also, a way to shape actions in a specific way. We may say that drama is the design of an event that has specific dramatic qualities. Here I don’t want to sketch out the notion of drama, I want only to stress that the language of drama has some specific features. Naturally, by ‘language’ I am not referring to the words written in the lines of the text, but to the language of actions.
Drama theory has largely discussed whether drama is about plot or characters. The argument is centuries-long and has changed according to the contemporary socio-psychological model, cultural/religious beliefs, and aesthetic theories. Briefly, scholars and artists, in the last two centuries, have basically agreed that the most effective outcome of drama is the fictional character. The character has been – and still is – considered the medium the author uses to bind us to his story, to his message, to his soul. Character has become the interface between the audience and the artist.
Approaching the question from a more plot- or action- based perspective, in 1895, George Polti defined Les trente-six situations dramatiques as a set of situations, or a way to achieve a certain goal . Here, the most important thing, rather then the accuracy of the number of definitions, is the idea of a finite pool of elements: Supplication (Implorer), Deliverance (Le Sauveur), Crime pursued by Vengeance (La Vengeance poursuivant le crime), Vengeance taken for Kindred upon Kindred (Venger proche sur proche), etc. From an “actional” point of view those situation can be described as an interaction between agents endowed with goals and emotional state. Indeed the Polti approach was influenced by the success of the pièce bien faite (as defined by Sarcey) and by the relevance of natural science and positivism in those years. Nevertheless, this approach defines a proto-structuralism through which it is possible to discuss the true language of drama. Along this road to the semiotics of drama, we encounter the effort made, in recent years, by artificial intelligence theorists to formalize human cognitive and physical behaviours.
It is well known that the first famous attempt to simulate a human being using a computer procedure was Eliza, in 1966. Joseph Weizebaum wrote a little software program to simulate a dialogical interaction between a human and a computer. The software was conceptually based on few simple assumptions about natural language and on a specific therapeutic dialogue (in fact, Eliza was mimicking a therapist). This was the birth of a generation of chatterbots (chatting robots), who are still evolving – especially in the last three years – for example as virtual assistants on the web.
Although the aim of Eliza wasn’t drama at all, it was clearly stated by Weizebaum that the interaction had to be based upon the dramatic paradigm of character. Eliza had to be designed as a specific fictional character in order to be believable within that specific interaction.
In more recent years artificial intelligence research has started to look beyond the level of natural speech, focussing on behaviours and, therefore, on actions. Agent theory analyses speech as just one part of the higher level of ‘action’ Via a rational assumption of human behaviour, the character’s actions are described as a ‘goals and plans system’: from a cognitive point of view, the action is the will of reaching a goal, the design of a plan to get this goal, in a specific environment and situation. The character is designed to interact with the world: therefore he is provided with a formal model of the world where he lives. In other words, within the artificial intelligence framework, the character has a local knowledge about where he is and what kind of rules there are; a goal to achieve; and he adopts plans accordingly.
More technically this is called the belief, desire, intention model (BDI). Naturally a large goal requires a large and complicated plan. This large plan would be made of smaller plans that fulfil smaller goals… and so on, following a fractal structure.
The BDI model conceptualizes the character as a highly interactive medium, as an entity always on the move to adapt, to relate with someone or something. Character is more and more becoming the paradigm for human computer interaction.

5.
A deeper analogy between drama and artificial intelligence is emerging here. It means that in order to achieve a better human-computer interaction, computer science has to assume the rules that drama has developed to design characters that seem alive.

If artificial character is about the representation of a ‘live’ being, then it can be said that the first artificial character has been created on stage – in the theatre. Hamlet is alive and is the outcome of a process designed by Shakespeare. Synthetic artificial characters have to compare themselves to this cultural background. On this basis, the relationship between theatre and digital multimedia appears to be more profound. Here we can abandon phenomenology and start to discuss the ontological level.
Critical theory has long discussed issues and problems around the essence of drama. For the last two centuries, manly thanks to Anglo-Saxon pragmatism and the French pièce bien faite, it has been clearly established that drama it is not – at least, not only – written poetry. The defining feature of drama is action. In other words, the dramatic author faces the challenge of organising actions to be performed on stage. Drama is the language of actions. Representing actions via a written text can be a difficult job. Over the decades, the more technologically developed the stage became, the more complex the job became for the director. Stanislavsky basically re-wrote Chekhov plays adding hundreds of descriptions and directions; Piscator adopted prompt-scripts divided into columns to match the spoken words with the other elements of the action; even as purist an author as Pirandello made complex use of stage directions and soon decided to run his own theatre company to better control his plays.
The individual line of text in the play (‘to be or not to be’) may not be the best means of representing action…While dance has developed a number of notation systems (for example the Labanotation), and semioticians have tried a few complex systems to annotate drama, there still isn’t a proper formal system to represent – as data – the complex inner and outer contents that we can usually see – on stage – in a single written line of a play’s text. Agent theory may be able to contribute here. Agent theories have endeavoured to formalise a proper language for representing actions, providing scholars and practitioners with a fairly reliable – although complex – system. Because it is a new way of describing actions, it can also be a new way to describe or produce drama.
As we said, drama uses the character as a medium and shapes his actions according to a set of rules. Drama is a live action performed at the present time by agents. Usually the performance follows a script more or less precisely designed by the author.
In the last decade, computer science and the videogame industry have been trying to encode the rules of drama to be able to design dramatic events that can be performed interactively.

One of the best known attempts has been Facade, the interactive drama designed by Michel Mateas and Andrew Stern. There the user can freely interact with two characters (husband and wife, Trip and Grace), within a given situation, via text and gestures. The aim is to simulate a dramatic event as in an actor improvisation. The chief achievement of the experiment is to have made clear that, in order create believable characters, the interaction must follows the rules of drama. In fact behind ‘Facade’ there is so-called “drama manager” that constantly shapes, redirects, and controls the two characters. The “drama manager” performs as a live author, or as a dramaturg on stage, who has a formal knowledge of how the drama has to be handled.

Mateas claims a sort of neo-Aristotelian theory of drama, where the user takes part in the six elements of tragedy together with the character. Although facade has been the first and most effective experiment up until now, it has to be said that the theory behind it is still developing.
The software has been designed to match that specific dramatic situation (husband and wife, who host a friend – the user – for a drink in their flat and have an argument). So there is a lack of general assumptions about drama. The character doesn’t have the ability to develop enough to maintain the audience’s emotional engagement for longer that 10 or 15 minutes. Moreover, the audience coincides with the user and vice versa.
At the head of Aristotle’s six formal causes there is the author. Reconsidering the six elements within an interactive environment necessitates – I believe – adding a new element called ‘procedure to the line, directly above ‘plot’. Hence characters are the set of attributes (of different kinds) shared both by the virtual and human agents (the user). The procedural quality of the drama comes above the character’s interactivity and the latter is an outcome of the former. ‘Procedure’ is a new element available to the author to shape the drama, in addition to character and the other elements, and it can be perceived by the audience as one of the materials of which the drama is made.

In the last few years we – at the CIRMA research group in Turin – have developed a theoretical system called Drammar that accounts for the character’s emotional state (and its changes), and for the building of a plot using drama units. In our system the character is designed upon the BDI paradigm to which we added an emotion paradigm (using a pre existent formal model of emotions). The character is a set of beliefs and emotions. He also has drama goals and plans to reach them. The behaviour is shaped according to the drama unit direction, i.e. a higher goal that aims to change the emotional state of the character. Drammar, actually, doesn’t aim to any specific application. It rather wants to lay the theoretical foundation for a wider paradigm for interactive drama.

Nevertheless, we are using this background in more practical applications, such as a virtual character on a mobile device who guides visitors around historical sites. I will discuss these applications deeper in tomorrow’s session if anyone is interested.

Drama and theatre are bound together so that any change in the former will affect the latter. The new paradigm of interactive drama challenges the performance and its components: actor and space. The theoretical framework I have tried to draw, allows us to figure out a new kind of performing space, which is able to respond automatically, intelligently to the performer’s actions. Stage and actors are no longer in opposition, but in symbiosis. Here the dream of the Avant-gardes (from Symbolism to Futurism) became a starting point, a lens through which the scholars and the artists may see new experiments. On the other hand, the body of the actor itself can be augmented thanks to technology. This augmentation coincides with the “augment” of the stage. The stage has to review its own limits, both practical and conceptual. The space of drama can be sensible and intelligent, but also dislocated, as has already happened in pervasive gaming.

On the higher level of narration, there is another point of view to investigate. To focus on plot and direction, means to overcome the fashion of the hypertext as something totally open, undetermined, and to move toward a particular form of procedural narrative. From this perspective, the notion of digital theatre sketched above, rather than stressing the kaleidoscopic nature of the pixels, highlights a so called procedural attitude. So, a ‘procedural’ drama produces a set of situations that, even if they can be customized by the user, are driven by a given structure that tends to shape the user’s emotional engagement. Moreover, behind this structure there is an author. After the last twenty years, during which digital arts, following the post-modern culture, have dealt largely with the loss of meaning, where interaction has mainly been interpreted as the fortuitous and peculiar relation between the user and the artwork, the time could now be right for investigating the field of the meaning, the possibilities of a well structured narration. To the random function raised at an aesthetic level, it is possible to oppose the strong notion of plot as a production of meaning.

[This text was presented at the conference session I held in Cairo on Drama, Performance and Digital Multimedia, on the 5th of September 2007, followed the next day by a workshop on the same topics. The other conference material is here]

PDF [380 KB]