Stanford, Google researchers create virtual town populated by AI

A team of researchers at Stanford and Google worked to populate a town with what they call “generative agents.” These are computational software “agents” that simulate believable human behavior. In short, these researchers aimed to populate a town with artificial intelligences that closely mimicked human behavior.

The authors of this paper describe a Sims-like simulation where users can interact with a small town of twenty-five characters called “Smallville.” The goal of this project was to have these agents develop realistic behavior both on an individual and emergent level. As an example of this behavior, the paper talks about a Valentine’s Day party. The user states that one of the agents wants to throw a Valentine’s Day party; it follows that the agent will spread invitations to the party and the other agents will ask each other out on dates to the party — like humans would.

Each agent was given a one-paragraph description about their identity, including their occupation, relationship with other agents, and seed memories. For instance, “John Lin is a pharmacy shopkeeper at the Willow Market and Pharmacy who loves to help people. He is always looking for ways to make the process of getting medication easier for his customers; John Lin is living with his wife, Mei Lin, who is a college professor, and son, Eddy Lin, who is a student studying music theory; John Lin loves his family very much; John Lin has known the old couple next-door, Sam Moore and Jennifer Moore, for a few years; John Lin thinks Sam Moore is a kind and nice man; John Lin knows his neighbor, Yuriko Yamamoto well; John Lin knows of his neighbors, Tamara Taylor and Carmen Ortiz, but has not met them before; John Lin and Tom Moreno are colleagues at The Willows Market and Pharmacy; John Lin and Tom Moreno are friends and like to discuss local politics together; John Lin knows the Moreno family somewhat well — the husband Tom Moreno and the wife Jane Moreno.”

Realistically, this was 25 instances of ChatGPT interacting with each other. While the paper includes graphics, these are just models of the underlying work that was being done. The different areas of the “map” are associated with some kind of word, which the agents will “remember” and associate with that “location.” As time passes in Smallville, each agent will output a statement, which will be translated into some language that affects the “world.”

Over time, these agents will also create and build relationships. For instance, an interaction between two agents named Sam and Latoya will change depending on if they are friends or not. When they first meet, Latoya might say, “I’m here to take some photos for a project I’m working on.” In a later interaction, Sam might say, “Hi, Latoya. How is your project going?” to which Latoya replies, “Hi, Sam. It’s going well!”

But how exactly can one evaluate how “human” a model is? The researchers took advantage of the agents being able to respond to user inquiries. The researchers “interviewed” the agents in five categories: self-knowledge, memory, plans, reactions, and reflections. Human participants observed a “day” in these agents’ lives and then ranked the believability of the agents’ responses to the questions from most to least believable. Overall, the study found that while the agents were able to answer questions consistent with their knowledge, they sometimes failed to remember certain details.

A proof of emergent behavior comes from the spread of information from the agents. In one instance, the number of agents who knew about an agent’s mayoral candidacy increased from one to eight; in another instance, the number of agents who knew about an agent’s party increased from one to twelve, all without user intervention.

In some cases, agents made mistakes relating to what is considered “proper behavior.” In one instance, agents chose to enter an occupied one-person bathroom in the college dorm because college dorm bathrooms are usually meant for more than one person. In other cases, some agents entered stores after they had closed, as they did not understand that the store was closed (which some would consider normal human behavior).

Smallville is one of the first generative agent worlds, and at the end of the paper, the authors suggest that future researchers look into expanding the modules that these researchers used.