Autonomous Artists, Agency, and Intent

One of the biggest themes in AI art recently is autonomous artists. Would you believe that autonomous artists have a history going back centuries? Yes, centuries. The first autonomous artist that I am aware of was made in 1774 by Pierre and Henri Jaquet-Droz, an automaton called Le Dessinatuer (The Draughtsman).

Le Dessinateur operates via a system of mechanical gears, cams, and levers, driven by a spring-wound mechanism. It produces only four different drawings, but it does so without any intervention once set in motion. Interestingly, this automaton was made as an advertising vehicle to bring attention to the Jaquet-Droz brothers’ watch-making business. In fact, severa of the early automata such as John Joseph Merlin’s Silver Swan, Maillardet’s Automaton and Piguet & Meylan’s automata were created with the purpose of advertising watch-making businesses.

The mystique of the autonomous sparks something deep in the human imagination, perhaps akin to the godlike power of creating life. Its definition though is quite simple: any system that does something complex without human intervention.

Before I began to question what autonomous systems might mean in art, I had my first experience with them participating in the DARPA Grand Challenges to develop autonomous vehicles back in 2005. Back then no one was using the word ‘autonomous’ because few people understood it outside of AI academia. Most journalists instead called the cars we were building ‘self-driving.’

As autonomous cars drive from point A to point B without human intervention, one can think of an autonomous artist as something that creates art without human intervention. But in the context of art, these systems raise the question of where the act of art-making begins – whether the generation of intention is to be considered a human intervention in this process, or not. In this, the mystery of the autonomous bleeds into the mysteries of creation, the territory of the unknowable and undefinable.

One of the first autonomous artists I encountered was Simon Colton’s The Painting Fool. The Painting Fool was a landmark in computational creativity as one of the first systems to incorporate context-aware decision-making, and it made art of surprising sophistication. One such image is shown below.

Simon Colton’s Painting Fool, 2013

I spoke at length with Simon about what autonomy was and whether or not my robots and his Painting Fool were autonomous artists. He made a very clear point that I remember to this day. Simon told me they may be autonomous, but that they lacked intent, and since they lacked intent they were not artists. While they were autonomous artwork-makers, his view was that they were not strictly speaking artists because they lacked intent. For him, the key was distinguishing “autonomous artists” from “artists.”

Another striking autonomous artist from this same breakthrough period is Deussen & Lindemeier's eDavid. I found out about them when my name turned up as a source in a paper they wrote. Deussen and Lindemeier wrote that “painting can be seen as an optimization process in which color is manually distributed on a canvas until one is able to recognize the content.” Their robot did this by making marks, analyzing the marks and using that analysis to make more marks to optimize the image.

Deussen & Lindemeier's eDavid

One critique of these autonomous artists from the past might be that they do not pick their subject matter and style, whereas current autonomous artists do. But from a historical perspective, autonomous artists have been selecting their own content for many years. I was by no means the first to do so, but my robotic system ARTonomoushas been deciding on its own content since 2005, and its own style since 2016. In the early years I had it scraping news sites for trending topics and painting the hot topics of the day. In the following painting, it was given 6 pieces of art and asked to create a painting after performing a photoshoot with my wife. What is interesting here is that ARTonomous performed every single aesthetic decision from selecting its favorite photo, to starting the painting, running it through multiple rounds of neural networks, and arriving at a final abstract composition.

Another major influence for me and for the field was of course Harold Cohen. He actually convinced me to stop naming my robots because it gave the false impression that they had agency. Both his AARON and my ARTonomous are not robots, but software systems. He would actually get irritated when people named physical robots with anthropomorphic names because he thought that it misled people into thinking they had agency, and he was adamant that if humans had anything to do with the process, the software and robots were just a tool in between human feedback loops.

”By definition processing is a deterministic affair, and for any single run its functions are predetermined and invariant. Feedback from the result to the functions themselves has no part in this process. On the other hand feedback is clearly a part of the human art-making process, or indeed of any intelligent process, and if the only feedback possible within the computer environment is via the human user, then the computer is a tool in no essential way different from any other tool…" Harold Cohen (Parallel to Perception)

His argument was that without complete autonomy, the system lacked agency. There is another interesting word being used more and more that we really should go more into: agency.

Right now a lot of emphasis is being placed on agency and agentic AI. So what is it? The simplest definition I have found is NVIDIAs definition.

"AI agents are advanced AI systems designed to autonomously reason, plan, and execute complex tasks based on high-level goals."

Pretty simple, and by this definition there have been agentic AI artists since at least the 1970s. Harold Cohen’s AARON perfectly fits the description of an agentic AI.

Recently there has been a new generation of autonomous artists. While they are ontologically identical to the previous autonomous artists, one major difference is that they are centered around LLMs. This new generation is much more sophisticated in their processes, but they still perhaps face the limitations that Simon Colton and Harold Cohen point out. Intent and agency are still wide-open questions.

Beneath the glamour of the new, I think that the fascination with autonomous art comes from an age-old desire to see the pure emergence of creation, without the barrier of the self. In the past, people have approached this through the muses, the oracle, the subconscious, through chance, and now we approach it through the prospect of an artificial mind. AI is probably still incapable of generating intent, but AI art and autonomous systems give us a way to explore the nature and presence of emergence. Perhaps it is this quality that underlies our fascination with autonomous art, this threshold where information transforms into imagination.