SciTech Roundup 2/13

CMU Research Roundup

Robot FRIDA paints with artificial intelligence

While DALL-E and AI-generated art technology is widely available, it's difficult to translate this digital technology into physical technology. Researchers at Carnegie Mellon's Robotics Institute hope to change this with FRIDA, a robotic arm taped with a paintbrush. Like online AI-art generators, it can accept text prompts and files of other artworks for inspiration, but the researchers have also experimented with audio input as well. To create the physical piece, however, FRIDA first learns how to use its paintbrush. Then, FRIDA generates a digital piece using models similar to DALL-E and simulates how it would paint the artwork with brushstrokes. FRIDA then starts to paint, and compares the brushstrokes it produces to the simulation, adjusting the simulation accordingly. After hours, FRIDA produces a painting. The researchers emphasize that FRIDA is not an artist, but rather hopes to help humans express themselves artistically through technology.

Recognizing objects not by visuals, but by motion

Have you ever recognized someone not by the way they look, but the way they walk? Carnegie Mellon researchers in the Department of Psychology collaborated with the National Institute of Mental Health to investigate how humans use motion cues on top of visual cues to understand the world around us. In their study published in "The Journal of Neuroscience," they created a set of animations that replicated the movement of an object or animal, but didn't contain the image, shape, or colors of the original object/animal. After presenting the videos to 430 participants, they found that participants accurately identified the objects 76 percent of the time. Then, to identify which parts of the brain were involved in identifying motion cues, they scanned the brains of 15 other participants while they viewed the six most highly recognized object videos and their corresponding static images with the object in picture. They hope that their findings can assist in health care, to help clinicians study populations with difficulty in social perception, and in artificial intelligence, to help artificial intelligence see the world the way humans do.