With “Robot Artisan,” I generate text and images through multiple machine learning algorithms. The AI develops illogical, inhuman artwork that is used to show the divide between man and machine. These robotic creations are then manipulated and edited to create a piece of art. There are multiple stages in the process where there is collaboration between the AI and the human. I train a model made on art history lectures and textbooks to generate art commentary to pair with these pieces.
With my background as a traditional artist, I know there is a fear of being replaced by machine learning. Instead of running away from advances in technology, I explore the use of machine learning as a new process to create. I embrace these imperfect creations of the machine, as they exist in the uncanny valley. They serve as a reminder for why artificial intelligence cannot replicate art without human intervention (for now.)
Through a process of curating text, selecting images, and editing the final pieces, I conduct the robot artist to make a specific kind of art. I guide it through the process of creating something based on the vast history of art history resources and images in the model. Without this human intervention, machine learning lacks any sense of design or purpose. My intent is to spark discussion around what it means when new technologies are used to create art.
I embrace AI generated art and text as part of my creation process with this project. It allows me to work at a scale unimaginable in my own workflow. The product that evokes conversation around technology and art. Though this project can be enjoyed on purely aesthetic grounds, my intent is for people to confront the reality of the modern world as it applies to creating art. Artists might view this series of work as an uncomfortable challenge to their process, but I do not intend to show them an adversary. Instead I want artists, especially traditional artists, to realize that machine learning can be a tool. AI generated is a new paradigm shift in the creation process, in the same way that a camera or printmaking expands on the possibilities of creating art.
Machine learning also has an exciting aspect of accessibility. I want to showcase the methodology used to create so that I can inspire people to pursue creating art. AI generated art removes barriers of skill and time, unlocking access to people who would otherwise not create.
Some of the only barriers that remain are the technical knowledge and technical demands that machine learning requires. Most often you have to have access to expensive hardware in order to efficiently operate a machine learning algorithm. Thankfully with advances in cloud computing, there are free options like Google Colab notebooks and services that have been built on the same algorithms. These offer the opportunity to execute in the cloud and remove the need for expensive hardware.
I’m able to focus on the creation of art through machine learning by taking advantage of the work done by passionate programmers and software engineers. In terms of technical knowledge, I have a rudimentary understanding of programming that I use to explore these algorithms. It is through experimentation that I am able to find ways to generate the kind of art that I want to make. With my experience working with these algorithms, I want to share my process and learnings in a way that removes this technical barrier for other people.
A gallery experience for the artwork would feature these paintings as framed prints on canvas. This treatment of the works AI generated art enhances the conversation around the authenticity of art made through this process. Bringing this artwork from the digital realm into the traditional setting of a gallery will spark conversation around the product as well as the process.
This project also offers an opportunity to showcase how the works were made. In an exhibition, these pieces will be shown in their stages of development, alongside an installation where the viewers would be invited to collaborate with the machine learning on their own. The viewers would be able to make sketches of their own, input a unique prompt that they create, and then watch as the pieces are created in real time. Their work would be added to the exhibit as an addition to a digital slideshow. This gallery experience would be recreated online through a website where you could see the final piece, the iterations, and submit your AI creations in a digital gallery.
Revealing the process of creation dispels the notion that art is a realm only for a certain skilled few. Inviting the viewers behind the curtain allows them to see how this technology can empower them to create on their own, and reveals the hidden process behind these machine learning algorithms. The use of technology in this project furthers my mission of inspiring artists and sharing my belief that anyone can create. The process and gallery experience challenges artists to explore new technologies to create.
Methodology
Scraping art history resources allows me to train the AI textgen model to generate commentary and titles. I accomplish this by building a library from art history articles on Wikipedia, Project Gutenberg, and art history lectures. With any resource, I ensure that the use of their materials is within their terms and conditions. I wrote a python script to automate acquisition of this text.
Massaging the data is required since it has special characters and different formatting across the sources used. Massaging the data, which is a way to say formatting for a specific use, requires a series of actions performed on the text. The current process involves executing a series of actions to remove special characters, titles and headers, and to put all sentences on one line. This can be automated with a python script. This allows the model to be trained in a unified manner. This process produces the best model and allows much more creativity and consistency when generating the text.
Training and generating text from these resources, I utilize Google colab to run the AI textgen model. My current model is trained on 267k words for 12 hours. Ideally this model would be trained on at least a million words for multiple days to allow for more creative prompts. The current model produces illogical text, which actually adds to the feeling of it being in the uncanny valley. By modifying the parameters, I am able to have control over how creative the model can be. An example of one of the parameters that can be modified is temperature, which controls how close the text is to the original. A higher temperature allows the AI to “imagine'' new ways to combine words and form sentences.
Title and commentary to pair with each of the pieces is found by searching the generated text for keywords from the title in generated text to build out commentary. These selections allow the opportunity for human curation to select relevant text. The prompt for machine learning is created by including an artist and descriptive terms to shape the final piece. Example: “Even necessary time by certain painting made of hourglasses in the style of picasso.” In this case “Even necessary time by certain” is a part of the generated text for the piece. The addition of other terms allows for more specific outputs. Including an art type (painting), with material (made of hourglasses), and created by an artist (in the style of picasso) gives the algorithm a robust seed to work from.
I create AI generated art in Big Sleep + Deep Dream (old 2 step method) or VQGAN + CLIP (preferred) using the generated title as a prompt. The latter method is far superior in speed as well as quality. It also offers the opportunity for more control through the control parameters in the dual model. Parameters include how different the iterations are from each other (learning rate), how closely it matches the original (mse_weight), selecting a different model trained on different resources, The prompt is run multiple times in order to find the foundation for the piece. As the iterations are run, the prompt is changed in order to shape the outcome.
Finalizing the work requires going through an AI upscaler to get print quality images. The images generated from VQGAN + CLIP have maximum dimensions of 512 x 512 pixels. The upscalers offer the ability to increase the size of the art 800% through machine learning. With these enlarged images, I use photo manipulation to combine multiple iterations of the same prompt together. This final step allows me to refine the final work into a piece of artwork that feels more human. Through the editing process I select different textures, colors, and compositional elements to craft an artwork that lives in the uncanny valley. At times, there is a need to bring in digital painting to create a desired effect or add to the piece. This human intervention creates a product made from the collaboration of humans and machines.
Showcase the final pieces and highlight the process. Revealing the behind the scenes shares the creation process with the viewer, inviting them to create their own generated artwork. The gallery would include all of the AI generated text and images, as well as an explanation of how the project was created. A gallery exhibit would feature:
The final artwork printed on canvas paired with their generated text and commentary.
Time-lapse showing looping video of the iterations as they morph closer towards the final product.
A workstation that invites the viewer to join the creative process by making a drawing and putting it through the machine learning algorithm. They would be able to make a digital sketch and then put in their own text prompt to add their piece to the installation.
A digital installation, either projector or other screen, that would share the creations made by the visitors to the gallery in a slideshow.
Example: First in the series
Prompt: robot artisan incandescent painting made of stain glass face made of eyes in the style of van gogh
Title: It Dead Art Because Fake Titles were Used By Robot Artisan
Commentary: There could potentially apply conclusions over simply innate systematic preference elements. More systematic representation is common and less natural poetic reality. Limited edition is held up until represent human.
