AI-powered robot called FRIDA creates art
FRIDA is a robotic painting system that collaborates with humans to create unique works of art. The robot uses artificial intelligence and machine learning to simulate how it would paint an image with brush strokes and evaluates its progress as it works. Its final products are impressionistic and whimsical, lacking the precision sought in traditional robotic endeavors. In fact, if FRIDA makes a mistake, it riffs on it, incorporating the errant splotch of paint into the end result.
FRIDA’s team is led by School of Computer Science Ph.D. student Peter Schaldenbrand, along with Robotics Institute faculty members Jean Oh and Jim McCann. The project has attracted students and researchers from across Carnegie Mellon University, with the goal of exploring the intersection of human and robotic creativity.
Users can direct FRIDA by inputting a text description, submitting other works of art to inspire its style, or uploading a photograph and asking it to paint a representation of it. The team is experimenting with other inputs as well, including audio. They played ABBA’s “Dancing Queen” and asked FRIDA to paint it.
Although FRIDA uses AI models similar to those powering tools like OpenAI’s ChatGPT and DALL-E 2, which generate text or an image in response to a prompt, FRIDA is not an artist. Instead, it is a system that an artist could collaborate with. The artist can specify high-level goals for FRIDA and then FRIDA can execute them.
FRIDA uses large vision-language models trained on massive datasets that pair text and images scraped from the internet, such as OpenAI’s Contrastive Language-Image Pre-Training (CLIP), to understand the input.
The robot’s actual brush strokes are used to train the simulator to reflect and mimic the physical capabilities of the robot and painting materials.
The team seeks to address some of the limitations in current large vision-language models by continually refining the ones they use, training data contributions from various cultures to avoid any cultural bias.
FRIDA’s team’s main goal is to promote human creativity through FRIDA, and the robot is not intended to take artists’ jobs. The team hopes that artists will use FRIDA as a tool to express their ideas in painting.
