Language and Computational Creativity

Language and Computational Creativity: Beyond the expanded conversation — cybernetic pattern flows

A Definition of Creativity

Boden in Creativity and Art defines creativity as “the ability to come up with ideas or artifacts that are new, surprising, and valuable.” (Boden, 2010) She states, in relation to the difficult questions of computational “consciousness, intentionality, and the role of ‘brain-stuff’ and/or embodilment” that she, “no where claim(s) that computers are ‘really’ creative.” For creativity in computers “I am really asking what aesthetically interesting results can computers generate and how?” She wonders what might lead someone to suggest that a particular computer system might display qualities that are similar to human creativity. (Boden, 2010)

Ada, Countess of Lovelace, 1840

The Analytical Engine. Trial model of a part of the Analytical Engine, built by Babbage, as displayed at the Science Museum (London)

A Early Pre-cursor Related to Computational Creativity and an Embodied Approach to Learning

In the year 1842, in her notes to the Analytical Engine Ada Lovelace, the first computer programmer stated that a computer could potentially be used to create “scientific” pieces of music. (Babbage, 1961, p.249) Thus, even in the infancy of computer programming, the potential for exploring aesthetic computational creativity was being discussed. Turing’s description of the ACE (Automatic Computing Engine), the first digital computer, saw the potential for a machine with programmed responsive, “operative” input and output “organs,” (Turing, 1986, p.36) pointing to early notions of embodiment.
Definition of Computational Creativity

Computational Creativity has been defined by the The Association for Computational Creativity as “the study and simulation, by computational means, of behavior, natural and artificial, which would, if observed in humans, be deemed creative.” In terms of creative processes, one experiences a series of multi-modal pattern flows over a lifetime, and potentially develops their linguistic and creative abilities through these (Seaman, 2005). We can computationally model and/or abstract human multi-model ‘pattern flows’. I call this Polysensing (Bonin, 2002). In order to really expand modes of computational creativity, we need to articulate new approaches [dare I say creative] to explore such an elusive embodied learning terrain. I discuss this potential below in the section related to Neosentience. (Seaman and Rössler, 2011)

Creativity: At The Heart of Cybernetics — Gordon Pask

In The Cybernetics of Human Learning and Performance, Pask suggests that creativity is at the heart of cybernetics:

The analogy expressed or represented in the language employed to account for events is a metaphor. In this sense; Cybernetics is the science or the art of manipulating defensible metaphors; showing how they may be constructed and what can be inferred as a result of their existence. (Pask 1975, p13)

Central to creative practice are both metaphors and analogies. What is the difference between these two concepts? I will here point at the fact that a metaphor is a pattern of relation— “an object, activity, or idea that is used as a symbol of something else” (Merriam Webster). When I articulate a metaphor I am articulating a relationality between patterns. To develop a metaphor is in itself a creative act of potential meaning production. This can be poetic and/or pragmatic. I use the term pattern openly – patterns of thought, patterns of experience, patterns of articulation, and language patterns framing all of these. I understand patterns in a space of subject↔object unity, and in terms of the embodied sensing of environments over time. I also understand the importance of pattern recognition to computational creativity.

An analogy is a particular ‘variety’ of pattern relation — ‘the act of comparing two things that are alike in some way’. Here we also point to qualities of correspondence. One problem is in understanding how creativity arises in the individual in that we are not privy to the depth of our own pattern-related thought processes. To a certain degree we must infer certain kinds of creative operations.

In terms of learning to be creative as a human, we start at a very early age with play. With random movements enabling us to discover our limits. With the production of sounds and eventually with the learning of words. Later, we live and take classes where many ideas are discussed, and conversations about creativity are held. We explore many creative processes. Some of our productions are reinforced, others not. We build up a sense of our creative potentials and of the potential to explore differing qualities of aesthetics. Much of this happens in conversation with a community and in conversation with ourselves. All of this is embodied through patterns of differing relations.

A Cybernetic/Historical Beginning To An Expanded Notion of Conversation

Maturana defines cybernetics as “the science and art of understanding” (ASC Website).
In terms of developing an expanded notion of conversation related to pattern flows, Maturana early on provided this definition of the linguistic domain:

The linguistic domain as a domain of orienting behavior requires at least two interacting organisms with comparable domains of interactions, so that a cooperative system of consensual interactions may be developed in which the emerging conduct of the two organisms is relevant for both… The central feature of human existence is its occurrence in a linguistic cognitive domain. The domain is constitutively social. (Maturana, 1970)

If we define interactive systems as consensual domains, we might also call certain interactive processes particular forms of conversation. From Gordon Pask’s multi-perspective uses of the term conversation, we can hold a cybernetic conversation by exploring natural-language discussion; via analogy; as well as through metaphor. Pask’s work explores each of these approaches at different times. Here computers can become a ‘conversational’ vehicle of creative thought. In my book Recombinant Poetics I discuss a parallel notion, that we are on the cusp of a non-logocentric linguistics that arises in computational-consensual domains. (Seaman, 2010, p.11). An example of an analogical “material conversation” is Pask’s explorations into electrochemical computing discussed in part in “Physical analogues to the growth of a concept”:

I can say what a solution means. This will be the case if, instead of talking about solutions and dynamic equilibria, I interact with the assemblage, regard it as similar in a functional manner, and employ it as an extension of my own thinking processes. (Pask 1958, p 915)

We can continuously flow between these differing forms of conversational foci via creative computational processes, exploring a pluralistic sense of what discourse, language, and conversation in particular, can actually be. In A Comment, A Case History and a Plan, Pask discusses : “aesthetically potent environments” :

It may, …respond to a man [or woman emphasis Seaman], engage him [her] in conversation and adapt its characteristics to the prevailing mode of discourse.
(Pask 1971, 76)

Here, the notion of a “prevailing mode of discourse” is central, where differing forms of conversation function as discourse vehicles for scientific research, for learning systems, and for artistic creation. Pask was already on to this notion of the ‘expanded’ conversation in the service of interactive art production in the 50’s, exemplified in his work Musicolour (1st version 1953)(1956). And later in his Colloquy of Mobiles (1968), where the computer functions as a vehicle of “creative,” if not sensual “poetic” conversation.

A Collquy of Mobiles (1968) Gordon Pask. Early interactive work exploring an artistic notion of conversation

Pask’s Conversation Theory and Its Long-Term Potentials

Pask developed a fascinating approach to teaching/learning which he called Conversation Theory. “His first machines were special purpose teaching and learning machines of enormous sophistication (he later came to think of them as learning environments)… (Glanville & Müller, 2007, p24) His own writing on the subject is quite oblique, many terms have definitions presented elsewhere in the text. (Pask, 1975a) In discussing Conversation Theory Pask speaks of:

…A machine system for realizing a limited but non-trivial conversation. Attention was focused upon two sorts of conversation associated with distinct regulating heuristics; namely conversations for exhibiting or externalizing learning and problem solving strategies in a minimally biased fashion (the CET Heuristic) and tutorial conversations goverened by an uncertainty regulating (tutorial) heuristic. In both cases the topic of the conversation, formally, its domain was fixed and characterized by a paired description of knowable relations; namely, the entailment structure and the task structure. The conversation took place within a normative framework (the experimental/tutorial contract) in which the student agreed to “speak” an object language (the command and question language of CASTE); though the contract itself was negotiated in the metalanguage of an external observer. (Pask, 1975, p.141)

In discussing the importance of conversation to Pask, Ranulph Glanville, a student and colleague of his states “Conversation, the careful analysis and formulation of it and the introduction of it as a serious means of communication, is possibly Pask’s greatest contribution to Cybernetics and the understanding of human behavior.) (Glanville & Müller, 2007, p 23.) Paul Pongaro (who studied and worked with Pask), in a text entitled “Conversation Theory in Two Pages” states:

For us to understand each other, there are minimum requirements. We may both utter the word “cup” or “happiness” or “cybernetics”, but, what is required for each of us to know we agree on the meaning? A conversation, surely. You explicate how a cup is used, and what it is for. I hear your views, re-compute your perspectives, and come as close as I can get to your meaning of “cup.” But is your meaning (or, to say it more carefully, my view of your meaning) consistent with my own, pre-existing view? Are there conflicts? And that is only the half of it. After I exteriorize _my_ view of why a cup is what it is and how it is used, does your view of my view of a cup resonate (and not conflict) with your original view? In summary, if we resonate together in our views of “cup”, then (as named by Conversation Theory) we have “agreement over an understanding” – in both metaphorical and formal terms. (Pongaro, 1999-2000)

The long-term question is ‘could a computer also be enculturated’ through conversation and multi-modal synthetic sensing. Could it be ‘brought up’ to become ‘self-aware’ related to creative multi-modal pattern recognition and creation, by abstracting and re-embodying human creative processes, and aspects of human learning— via conversation and differing forms of synthetic observation? I will later discuss Luc Steels, whose robots begin to do just that.

Unpacking Pask’s Multiple Forms of Conversation

When Pask undertakes his electrochemical experiments he is creating an ‘operational’ analogy. And when he has ‘conversations’ via his systems he is exploring differing forms of consensual domain. In many of his works Pask demonstrates a broad definition of ‘language’ that couples/expands the use of natural language with other forms of non-logocentric manifestations.

In Pask’s Anti-Hodmanship paper he suggests that man/machine interaction language “should have many of the capabilities of natural language…the language should be capable of representing analogy or, in literary terms, metaphor (either loose or structured).” (Pask, 1972, p 241). The construction of metaphors and analogies are creative acts enabling the entertainment of a particular qualities of relationality through patterns.

Continuing along this “expanded” linguistic line, in his introduction to The Cybernetics of Human Learning and Performance, Pask provides this interesting articulation:

In dealing with systems of any kind, cybernetics is primarily concerned with establishing isomorphism (one to one correspondences) rather than the validation of propositions that are true (or have a chance of being true) or else are false. The basic mode of argument and development involves analogy, strict analogies of which isomorphism is a special case (Pask 1975, p13) …Reflective and relativistic schemes … are bound up with a participant observer rather than a classical external observer. (Pask 1975, p13)

Pask in MicroMan states:

The part of our mind that synthesizes must be aware of the content of thesis and antithesis; it resolves a conflict between two personalities in the brain, one upholding the thesis and the other antithesis…this means taking the dialectical model somewhat further than many orthodox philosophers would wish. Unfortunately such a model does not clarify the question of how we actually reach synthesis. Is it always by logical deduction? One could argue that juxtaposing an improbable thesis and antithesis sometimes leads to creative synthesis….And we might find that creativity is less far removed from logical thought than we sometime suppose. If this is the case, it might one day be possible to make computers behave in ways that appear to be, or indeed are, creative. (Pask and Curran 1982, p71)

I understand creativity to be a synthesis of a n-dimensional set of associations that become enfolded. In a later chapter in MicroMan entitled “Metamorphosis of Machines and Man”, Pask states:

We can implement a version of a multi-dimensional world view as a model in a computer, using complex manipulations which some would claim to encompass both thought and consciousness. It would entail what Douglas Hofstadter (in Gödel, Escher, Bach: An Eternal Golden Braid) calls a ‘vortex’ of computer programs, each acting simultaneously on the others lower in a hierarchy during their execution. (Pask and Curran 1982, p210)

Pask goes on to propose a holistic thought model:

Several foci of attention coexist, each with its own autonomous and potentially independent clock and counter. Each focus embodies one loop-like organization, and each is a possible world of action, or a person, or an institution. The coordination of the clocks and counter, if it occurs, occurs by virtue of the whole process rather than according to predetermined rules. A system modeled in this way is truly self-organizing. It is this co-ordination of potentially independent units (the absence of a common controller which is consciousness, a property of the whole and not the parts. The degree of consciousness is determined by the degree of co-ordination. (Pask and Curran, 1982, p74)

The contents of consciousness are loop-like organizations shared between different foci… mental activities are primarily inventive or creative— learning entails discovery, action involves imagination, analogizing and venturing into the unknown. In principle we believe that individuals could converse creatively through the medium of a population of computers, a system that allows for both conflict and conflict resolution (thesis—antithesis, then synthesis, emphasis the author). (Pask and Curran, 1982, p 210-211)

Later we will discuss Steels robot interactions which play out Pask’s concepts in their creation of new languages.

Some Additional Approaches to Computational Creativity

Schmidhuber is exploring machine learning, artificial general intelligence, neural networks, digital physics and art in the service of computational creation. He began discussing creativity in 1990 and is highly articulate in his text Formal Theory of Creativity & Fun & Intrinsic Motivation (1990-2010). The book Computers and Creativity includes a number of exciting approaches to the field of computational creativity including a text by Schmidhuber (McCormack and d’Inverno, 2011):

The simple but general formal theory of fun & intrinsic motivation & creativity (1990-) is based on the concept of maximizing intrinsic reward for the active creation or discovery of novel, surprising patterns allowing for improved prediction or data compression. It generalizes the traditional field of active learning, and is related to old but less formal ideas in aeshthetic theory and developmental psychology.” (Schmidhuber 1990-2010, p1)

Schmidhuber’s approach perhaps stems from more traditional logical models of computation, yet defines a unique way into to the field based on reward.

A Related Embodied / Language Community Approach

Luc Steels, has been exploring a fascinating related approach but has been looking more at notions of an expanded sense of language creation and conversation as contributing to learning in a situated robotic entity, equipped with synthetic senses, Such work falls within the ‘embodied embedded’ paradigm. His embodied approach also includes notions related to the functionality of a language community. In his talk at Oxford, the Simonyi lecture entitled Can Robots be Creative Enough To Create Their Own Language? (Steels, 2012) he discusses the possibility of creating meaning while making representations. In the talk Steels discusses “Meaning Conceptualisation” where one translates meaning into a form… he states this could be language, paintings, music etc…. In particular he is interested in pointing at how syntactic structures, semantics, and lexicons etc. can be constructed by robots. He also discusses the potential of re-contextualising particular word meanings, to bring out new meanings. He suggests this activity requires a kind of synthetic creativity, where robots actually learn to invent new concepts. In order to capture the meaning, new modes of expression and even new drawing modes may have to be found (he suggests), as well as the development of new language— the creation of new terms. He describes this as arising through a scientific exploration of Language Games stemming from Wittgenstein (Wittgenstein, 1958), where embodied systems of active software agents are explored via real robots that play out language games alone, in pairs, and in language communities.

Can Machines Be Creative Enough To Invent Their Own Language? Luc Steels. For the past decade, I (Steels) have been doing experiments with my team trying to see in how far we could program a group of robots to autonomously create and learn from each other a symbolic communication system with similar properties as we find in human languages

Inventions of Words by Robots Suggesting Other Kinds of Invention

Steels discusses the creative invention of words in this robotic language community. One starts with a Prototype (a kind of formal synthetic sensed perturbation in space) and then one can invent a new word — just syllables being combined – leading to the invention of lexicons of relations, and to concepts. The creation and the learning of new words is thus explored in the robotic domain. He discusses the accretive shifting of meanings as more instances of shared use are “experienced”, and points to the notion of lateral inhibition dynamics. (see for example Amari, 1977) The robot builds links and then goes through a collective ongoing process, evolving (as one example) a basic color language from scratch. The best example of a prototype color is a “nearest neighbor” computation. In the experiment he presented, Steels states “We don’t give a lexicon and don’t give color categories.” He pointed out that success gradually went up to 95%. Over 1000 language games were played (200 by each agent). He discusses this as “A cultural evolutionary process.” Steels states there is no telepathy. That this plays out as a democratic process. The more games, the more similar the associated choices become. This is truly uncanney to watch. The robot builds up a kind of scoring system where successful links are reinforced in the community. Thus one increases the score when you have a successful association. The robot simultaneously shifts the weight on the unsuccessful score, decreasing its ongoing value. Thus, this is a process oriented, accretive approach to meaning production. Seaman discusses a related approach in his paper – Pattern Flows | Hybrid Accretive Processes Informing Identity Construction. (Seaman, 2005) One can also see the relation of such an approach to Pask’s Conversation Theory work.

Multiple Synthetic Sensual Modes of Application

Along with words the robots can define a form of Action language – they can they learn about actions via synthetic observation. He describes the “Posture Game,” Showing more than pointing… Here words are developed for talking about actions and making categories of differing actions. Of course this “problem” space is quite complex. It explores cognitive body image, visual body image, and motor body image (triangle) and as he discusses it, this relates to the functionality of mirror neurons. In the question section of the discussion he was asked about the anthropocentric use of language and he talked about the need to draw on language that we have developed to explore new concepts. He expressed that the slight mis-use of language is necessary because otherwise he would be “naked” before them, without a means to discuss his experiments.

Mis-using Existing Language to Create and Articulate New Concepts

The most unique part of his talk was about the use of existing language to reflect new concepts. Here he discusses the use words with one function, used to articulate another function. A “metaphorical” extension… this is where the creative leap happens when you want to shift word meaning over to new domains. Yet he points out that the shift can only happen within a sufficiently sized community. Perhaps this is the robotic equivalent to Pask’s definition of Cybernetics, given the above — related to “manipulating defensible metaphors”. Thus one can imagine a long-term exploration of aesthetics and aesthetic processes in the service of an embodied, embedded approach to computational creativity. Steels states:

Can Machines Be Creative?

• It depends on what we mean by creative
• If creativity is
  • To come up with novel solutions to problems
  • The answer is yes
• But it requires a solid network of knowledge and contexts where creativity is challenged. (Steels 2012)

Steels states:

Most importantly it is also crucial that meaning gets grounded in the context through the sensori-motor apparatus of the robot, and unless we have corpora that contain vast amounts of data on grounded interactions it is not possible to apply statistical machine learning techniques. (Steels, 2011) (Steels, 2000)

Thus, such a system plays out in an embodied approach to sensing and environment, and seeks to transcend more traditional “logic” based forms of learning. To my mind, this leads to a form of environmental understanding that historical AI can not reproduce.

Alternate Approaches To Computational Creativity

Another set of approaches has been discussed by Cariani in in Creating New Informational Primitives in Minds and Machines:

Three basic strategies for using artificial devices to create new meanings and purposes present themselves:

1. via new human-machine interactions (mixed human-machine systems in which machines provoke novel insights in humans who then provide new interpretations for machine symbols),
2. via new sensors and effectors on an external world (epistemically-autonomous evolutionary robots that create their own external semantics), and
3. via evolving internal analog dynamics (adaptive self-organisation in mixed analog/digital devices or biological brains in which new internal linkages are created between internal analog representations that are coupled to the external world and goal-directed internal decision states). (Cariani 2011, p397)

The potential of the creation of new synthetic senses to expand the multi-modal sensory arena is great. Seaman believes that along with Cariani’s and other’s approaches, a separate research agenda must also be developed that is reflexive or functions as a pattern-oriented meta-level. We often become self-aware as we make creative decisions. How can we develop a machinic self-awareness related to creativity? Ironically, in thinking about how to do this in machines, we contribute to our own human understanding.

Modeling Creativity — A second order Cybernetic Approach to Creative Processes and in turn Computational Creativity

I currently take a ‘mindful awareness’ approach to my own aesthetic processes, and apply this to aspects of authoring computationally creative systems. Varela, Thompson and Rosch in The Embodied Mind, suggest:

Its purpose is to become mindful, to experience what one’s mind is doing as it does it, to be present with one’s mind. What relevance does this have to cognitive science? We believe that if cognitive science is to include human experience, it must have some method of exploring and knowing what human experience is.” (Varela, Thompson, and Rosch, 1991, p.23)

I seek to take different aspects of my experiences, as well as my aesthetic ideas that have been built up over a lifetime, and define a set of parameters that become programmed in generative works of art exploring image, music and text. These works draw from a database of media materials and processes. Here, I seek to model my own creative processes and have aspects of them become operative within this system. I call this Re-embodied Intelligence (Seaman 1998), an example of an operative analogy:

Re-embodied intelligence can be defined as the translation of media elements and/or processes into a symbolic language enabling those elements and processes to become part of an operative computer-mediated system. The ability to “translate” the aesthetic conceptions of an “author” into a form that is operative within a technological environment is fundamental to the creation of interactive (and other forms, emphasis the author) artworks…In the creation of artworks the artist employs modes of thinking that might be considered illogical, nonsensical, intuitive, metaphorical, non-linear etc. The intelligence embodied in an individual’s art practice, functions in the service of their poetics. (Seaman, 1998)

In an ongoing manner, through mindful-awareness practices, I ‘transfer’ and make operative my history of pattern knowledge and aesthetics, and apply this in new ways in computational systems. I infer from Wittgenstein that I can never fully know a pattern in itself [language](Casti, 1994, p.7), I can only point. I become interested in authoring ‘pointing machines’ – machines that become vehicles of meta-meaning processes. In particular I explore chance processes that call in the media from a specific range of choices. This functions as a form of “loading” the dice in terms of aesthetics. I also develop particular substitution processes that enable a focused grammar and/or poetic approach to the text, to emerge. The generative music explores musical loops and/or modules of sound played back in differing orders. Thus the cybernetic loop enables the participant to explore content in an interactive manner, and/or observe computational systems play out elements of a cybernetic process exploring probabilities. We might again say, like in the work of Pask, the participant has a metaphorical “conversation” with the interactive system.

Modeling Creative Processes Computationally

It is not so easy to model the elusive qualities inherent to creative thinking and to aesthetic processes [like composing music] although this can be a goal. Seaman’s paper – Nonsense Logic suggests:

There is a poignant irony to the fact that the computer, a mechanism entirely predicated on symbolic logic, can be used to explore non-sense as well as illogical and elusive resonant artistic content… A work of art can be seen as an organism-like vehicle of content that is both generated and experienced through interaction. (Seaman, 1999)

Thus we explore human to human conversation, human/machine interaction, and in the final case machine to machine exchanges.

The Engine of Many Senses – 2013 – Present. Auto-generative work exploring multiple forms of computation.[1] Bill Seaman with Todd Berreth

Self-Programming and New forms of Computational Observing Pertaining to Computational Creativity

In his paper Morphogrammatics and Computational Reflection — of programming, Kaehr writes:

Reflective computational systems allow computations to observe and modify properties of their own behavior, especially properties that are typically observed only from some external, meta-level viewpoint. (Kaehr, R. date not set)

How might machines become autonomous creative observers? Central to creative practice is in defining metaphors and analogies as discussed above.

Glanville, who worked and studied with Pask points to his particular interest in such systems in his text The Value of being Unmanageable: Variety and Creativity in CyberSpace:

Others write about the physical and the practical. I am interested in the ephemeral, writing ephemerally, dreaming, but with what I hope is a certain aphoristic precision… (in computers) I can develop ideas: a way of considering. (Glanville, 1999)

This way of considering extends human knowledge production processes and expands creative possibilities.

Ascott (artist/ theorist/cyberneticist), who collaborated with Pask, and was thinking about some of these ideas early on, in Behaviourist Art and the Cybernetic Vision he pointed to a cybernetic self-organizing, organism-like, set of loops related to creative processes, behavior and change, where the computer “derives its initial program or code from the artist’s creative activity, and then evolves in specific artistic identity and function in response to the environment which it encounters.” (Ascott, 1966, p. 11)

Koestler’s The Act of Creation discusses creativity across Art, Science, Music and Humor. His concept of bisociation (thinking on two levels simultaneously in the service of creative production) has been central to developing new ideas in my own artistic and informational practices. (Koestler, 1964)

Neosentience Research – Informing a Holistic Approach to Computational Creativity

What role do emotions play in creative processes? Do we need to create synthetic feeling or emotions to augment computational creativity? We discuss this (and many other ideas) at length in Neosentience / The Benevolence Engine. (Seaman and Rössler, 2011) Such a robotic system would seek to embody the ability to become enculturated— to learn… We pragmatically say a machinic system is Neosentient if it can display the following attributes:

It—

• learns
• intelligently navigates
• interacts via natural language
• generates simulations of behaviour (it ‘thinks’ about potential behaviours) before acting in physical space
• is creative in some manner
• comes to have a deep situated knowledge of context through multimodal sensing
• displays mirror competence

We understand this to be a very different goal for intelligence to that of the Turing Test (Seaman and Rössler, 2011, p246). We are deeply interested in contemporary notions of personhood.

It is interesting to note that Gregory Bateson in A Sacred Unity / Further Steps to an Ecology of Mind discusses an active “creative filter” that intervenes in our construction of the world. (Bateson 1991, p. 264) Along this same line, Ernst von Glaserfeld in An Exposition of Constructivism: Why Some Like it Radical states, “the concepts and relations in terms of which we perceive and conceive the experiential world we live in are necessarily generated by ourselves.” (von Glaserfeld, 1990) We ask how the knower comes to be creative? Can we understand and abstract this, and transfer the nature of this functionality to computational systems? I project meaning onto patterns (constructivism). In part though my living as a social being, I construct/create a world of understanding and framed experience out of these diverse pattern flows. I develop meta-patterns via reflection. I learn that I can have different kinds of “conversations” with patterns, and in particular as an artist /designer with aesthetic patterns, including self-reflection.

Neosentience Cover – Designed by Bill Seaman

Toward Autonomous Creative Systems

In Metaphors we Live By, Lakoff and Johnson begin with this concept “Metaphor is pervasive in everyday life, not just in language but in thought and action.” (Lakoff and Johnson, 1980) Can we create systems that can generate metaphors and analogies on their own?

Ray Kurzweil in How to Create A Mind – The Secret of Human Thought Revealed also explores machinic creativity as a topic:

The neocortex is a great metaphor machine… Every one of the 300 million pattern recognizers in our neocortex is recognizing and defining a pattern and giving it a name, which in the case of the noecortal pattern recognition module is simply the axon emerging from the pattern recognizer that will fire when that pattern is fired. That symbol then becomes part of another pattern. Each one of these patterns is essentially a metaphor. (Kurzweil, 2012, p113)

Here we can also bring in the enculturation of a computer – how is it we can create metaphors and employ them in a constructive manner? As mentioned at the beginning of the paper, Turing spoke of input and output organs. (Turing 1986, p.36 & p.108) suggesting the beginning of entertaining pattern flows via “perceptive” systems.

Unpacking the Differing Stages of Computational Creativity

There are multiple stages or “readings” for unpacking notions surrounding computational creativity:

• The creation/authorship of computational systems as a tool for creativity; [e.g. photoshop]
• The authorship of systems that enable conversations and generative construction capabilities – (inter-authorship); [e.g. Seaman’s World Generator (Seaman, 2010)]
• The authorship of systems that enable a high-level of human/computer symbiosis, where the computer augments intelligence through the dialogics of interaction; [e.g. see Linklider’s text 1960]
• Generative systems that embody an aesthetic sensibility and generate works with an emergent outcome; (the programmer/author is creative and the system functions as a vehicle of this creativity) [e.g. Seaman’s current artistic work – The Engine of Engines]
• The authorship by humans of code-driven machines that function ‘autonomously’ and display a degree of creativity in a bio-mimetic and/or bio-abstracted manner – The creation of creation.
(here humans author a system with potentialities of becoming creative) [Future functionality]
• The authorship of learning systems that enable a computer to define its own sense of aesthetics. (Here the computer becomes enculturated/socialized) [Future functionaility]
• The authorship by machines of other code driven machines that display machinic creativity, and perhaps explore their own creation of creation and new sense of aesthetics through meta-levels of discourse. Here machines create/author other creative machines and write their own code. [Future functionaility]

I have pointed to a history of conversation with multiple meanings and processes attached to it – from the pragmatic use of natural language; to the employment of analogues of thought processes as a means of discourse; to the employment of analogy and metaphor. I draw on an expanded sense of linguistics to frame creative interactions, and the learning of creativity. We contemplate how to abstract these processes such that in time a machinic system may define its own sense of creativity through enculturation, and later though its own sense of abstraction, expanded conversation, and human/machine socialization. In fact the autonomous machine may develop completely new aesthetic forms that humans have never considered.

Notes

1) See the related didactic paper The Engines of Engines – Toward a Computational Ecology, Bill Seaman, 2012, https://link.springer.com/chapter/10.1007/978-3-642-28111-2_29?no-access=true [back]

Bibliography

Aleksander, I. and Burnett, P. (1987) Thinking Machines – The Search for Artificial Intelligence. New York: Alfred A. Knopf.
Amari, S. (1977) Biological Cybernetics 3. VIII. 1977, Volume 27, Issue 2, pp 77-87 Dynamics of pattern formation in lateral-inhibition type neural fields
Ascott, R. 1966. Behaviourist Art and the Cybernetic Vision. Cybernetica, International Association for Cybernetics, Namur, IX, pp.247-264.
Babbage, C. (1961), Charles Babbage and his Calculating Engines: Selected Writings by Charles Babbage and Others, New York: Dover Publications, Inc. Written to clarify the work Sketch of the Analytical Engine Invented by Charles Babbage, by L. F. Menabrea. p.249
Bateson, G. (1991) A Sacred Unity, Further Steps to An Ecology of Mind. Harper Collins Publishers: New York, New York
Beer, S., SAM in Reichardt, J. (1968), Cybernetic Serendipity, Praeger, London England
Boden, M. (2010) Creativity and Art, Oxford University Press
Bonin, V. (2002) Bill Seaman and Ingrid Verbauwhede – Poly-sensing Environment. Toward the Development of an Integrated Distributed Technology Exploring Poetic/ Informational Grammars of Attention and Functionality. http://www.fondation-langlois.org/html/e/page.php?NumPage=49 (accessed 21 November)
Cariani, P. (2011) Creating New Informational Primitives in Minds and Machines, in Computers and Creativity, in McCormack, J. and d’Inverno, M.(2011) Computers and Creativity, Springer, NY
Casti, J. (1994) Complexification: Explaining an Illogical World Through the Science of Surprise. New York: Harper Collins. P. 7
Fernandez, M. (2005) Gordon Pask: Cybernetic Polymath http://www.banffcentre.ca/bnmi/programs/archives/2005/refresh/docs/conferences/maria_fernandez.pdf (accessed 3, August 2014)
ICCC 2013 Sydney Australia, The Fourth International Conference on Computational Creativity, http://www.computationalcreativity.net/iccc2013/ (accessed June 22, 2014)
Kaehr, R. Morphogrammatics and Computational Reflection — Applying insights from the retro-grade recursivity concept of morphogrammatics to questions of reflectionality and interactionality of programming, (date not set) http://memristors.memristics.com/MorphoReflection/Morphogrammatics%20of%20Reflection.html (accessed 14 July, 2014)
Koestler, A. (1964) The Act of Creation, New York: Macmillan Co.
Glanville, R., & Müller, K. (2007) Gordon Pask, Philosher Mechanic, echoraum, Vienna
Glanville, R. (1999) The Value of being Unmanageable: Variety and Creativity in CyberSpace
http://www.researchgate.net/publication/229027886_The_Value_of_being_Unmanageable_Variety_and_Creativity_in_CyberSpace (accessed 25, October 2014)
Gunther, G. (Gunther, 1962) Proposal For a Basic Study of the Semantic and Syntactic Properties of Many-Valued and Morphogrammatic Systems of Logic. 1962, Biological Computer Lab Archive, Champaign/Urbana, Illinois, Illinois State University.
Koestler, A. (1964), The Act of Creation, New York: Macmillan Co.
Kurzweil, R. (2012) How to Create A Mind – The Secret of Human Thought Revealed, Viking: New York
Lakoff G. and Johnson, M. (1980) Metaphors We Live By, University of Chicago Press: Chicago
Licklider, J. (1960) Man-Computer Symbiosis, IRE Transactions on Human Factors in Electronics, volume HFE-1, pages 4-11, March See also http://groups.csail.mit.edu/medg/people/psz/Licklider.html (accessed 15 October, 2013)
Maturana, H (1978) The Biology of Language: The Epistomology of Reality (1978) http://ada.evergreen.edu/~arunc/texts/cybernetics/maturana/BofLanguage.pdf p.11
Maturana, H. ASC Website, http://www.asc-cybernetics.org, (accessed 6 March, 2015)
McCormack, J. and d’Inverno, M. (2011) Computers and Creativity, Springer, NY
Pask, G. (1958), “Physical analogues to the growth of a concept”, in Mechanisation of Thought Processes, Proceedings of a Symposium held at the National Physical Laboratory on 24–27th November, Volume II National Physical Laboratory Symposium No. 10, London: Her Majesty’s Stationery Office 1959.
Pask, G. (1972) Anti-Hodmanship: A Report on the State and Prospects of CAI 241, http://pangaro.com/pask/pask%20anti-hodmanship%20computer-aided-instruction.pdf (accessed 10, July, 2014)
Pask, G. (1975) The Cybernetics of Human Learning and Performance, Hutchinson & Co Publishers: London (p.13)
Pask, G. (1975a). Conversation, Cognition, and Learning. New York: Elsevier.
Pask, G. with Curran, S. (1982), Micro Man: Computers and the Evolution of Consciousness, New York: Macmillan, pp. 133–35.
Pangaro, P. (1999-2000) Conversation Theory in Two Pages. http://pangaro.com/published/cyb-and-con.html (Accessed 7, March 2015)
Reichardt, J. (1968), Cybernetic Serendipity, Praeger, London England
Schmidhuber, J. (1990-2010) Formal Theory of Creativity & Fun & Intrinsic Motivation (1990-2010). http://www.idsia.ch/~juergen/ieeecreative.pdf (accessed 10 July, 2014)
Schmidhuber, J. (1990-2010b) Formal Theory of Fun & Creativity Explains Science, Art, Music, Humor, http://www.idsia.ch/~juergen/creativity.html (accessed 10 July, 2014)
Schmidhuber, J. (1990-2010c) A Formal Theory of Creation To Model the Creation of Art, http://www.idsia.ch/~juergen/creativitybook.pdf (accessed 10 July, 2014)
Seaman, W. with May G (1996-present) The World Generator / The Engine of Desire – Generative Virtual Environment
Seaman, W. (2010) “Recombinant Poetics / Emergent Meaning as Examined and Explored Within a Specific Generative Virtual Environment.” VDM press, Verlag Dr. Müller
Seaman, W. (1998) “Emergent Constructions: Re-embodied Intelligence within Recombinant Poetic Networks,” Digital Creativity 9. 3, pp 153–160
Seaman, w. (1999) Nonsense Logic, in Reframing Consciousness, edited by Ascott. R., Intellect Press
Seaman, W. (2014) A Multi-perspective Approach to Knowledge Production, Kybernetes
Seaman, W. (2005) Pattern Flows | Hybrid Accretive Processes Informing Identity Construction, Convergence Magazine 7.2, special Issue on Intelligent Environments
Seaman, W. & Rössler, O. E. (2011) Neosentience / The Benevolence Engine, Intellect Press, p246
Steels, L. (2000) Language as a Complex Adaptive System. In Schoenauer, M., (ed.) Proceedings of PPSN VI, LNCS, 1917. Springer-Verlag, Berlin. pp. 17-26
Steels, L. (2011) Why We Need Evolutionary Semantics. http://www.csl.sony.fr/downloads/papers/2011/steels-11f.pdf (accessed 2015, 5 March)
Steels, L. (2012) Oxford Simonyi lecture. Can Robots be Creative Enough To Create their own Language, http://www.youtube.com/watch?v=n6876onk7sI (accessed 5, March, 2015)
Steels, L., Spranger, M., van Trijp, R., Hofer, S. and Hild, M. (2012a). Emergent Action Language on Real Robots. In Luc Steels and Manfred Hild (Eds.), Language Grounding in Robot, 255–276. New York: Springer.
Steels, L. (2000) Language as a Complex Adaptive System. In Schoenauer, M.,(ed.) Proceedings of PPSN VI, LNCS, 1917. Springer-Verlag, Berlin. pp. 17-26
Turing, A. (1986) Volume 10 in The Charles Babbage Institute Reprint Series for The History Of Computing: A. M. Turing’s ACE Report of 1946 and other papers. Cambridge: MIT Press p.36 & p.108
Varela, F., Thompson, E. and Rosch, E. (1991) The Embodied Mind, Cognitive Science and Human Experience. Cambridge/London: MIT Press.
von Glaserfeld, E., (1990) An Exposition of Constructivism: Why Some Like it Radical, http://www.oikos.org/constructivism.htm (accessed 14 July, 2014) Found in Constructivist Views On the Teaching and Learning of Mathematics (1990), edited by Robert B. Davis, Carolyn A. Maher, Nel Noddings. Reston, Va.: National Council of Teachers of Mathematics, Winograd, T. and Flores, F. (1986) Understanding Computers and Cognition: A New Foundation for Design. Norwood: Ablex Publishing.
Wittgenstein, L. (1958). Philosophical Investigations. 3rd  edn. Translation: G.E.M. ANSCOMB. New Jersey: Prentice Hall.