How life scenes shape consciousness and create memories

Life is made up of a series of small events: making your morning coffee, letting the dog out, opening your laptop, letting the dog back in. All of these events add up to a full day. Our brains are constantly observing and processing the events that make up our daily lives, says Jeff Zacks, the University of Washington’s Edgar James Swift Chair in Arts and Sciences and chair of the Department of Psychological and Neurobiological Sciences.

"Understanding where events begin and where they end is critical to understanding the world," Zacks said.

In two new papers, Zacks and other researchers from the Faculty of Arts and Sciences and the McKelvey School of Engineering explore this key process in human cognition.

Zacks led a study in which computer models were trained to watch more than 25 hours of video footage of people performing simple, everyday tasks, like cleaning a kitchen or cooking, and then make predictions about what would happen next. The study yielded a surprising finding: The computer models were most accurate when they responded to uncertainty. When the model was particularly unsure about what would happen next, it reset its settings and reevaluated the scene, which improved its overall understanding.

Co-authors of the study, published in PNAS Nexus, are Thanh Nguyen, a graduate student in the Zachs Dynamic Cognition Lab; Matt Bezdek, a senior research scientist in the lab; Aaron Bobick, a professor and dean of the McKelvey School of Engineering; Todd Braver, the William R. Stakenberg Professor of Human Values and Moral Development; and Samuel Gershman, a neuroscientist at Harvard.

Zacks had previously theorized that the human brain is particularly sensitive to the small surprises that fill our lives. He suggested that people overestimate a scene whenever they register something unexpected, a phenomenon known as "prediction error." But the finding that the successful computer model paid more attention to uncertainty than to prediction errors cast doubt on that theory.

"We do science," Zacks said. "We revise theories when confronted with new data."

Surprises are still important, and there is no need to abandon the concept of prediction error entirely, Nguyen said. "We are starting to think that the brain uses both mechanisms," he said. "It is not a question of choosing one over the other. Each model can make a unique contribution to our understanding of human cognition."

The Role of Memory in Event Processing

Maverick Smith, a research scientist at the Dynamic Cognition Lab, also studies the relationship between understanding events and memory. Working with Heather Bailey, a former WashU postdoctoral fellow who is now an assistant professor at Kansas State University, Smith co-authored a review article in Nature Reviews Psychology, gathering growing evidence that long-term memory is closely tied to the ability to logically and accurately determine where one event ends and another begins.

"There are many individual differences in the ability to identify the beginning and end of events, and these differences can strongly predict how well people remember events later," Smith said. "We hope to develop an intervention that can improve memory by helping people better segment events."

The influence of age on the perception of events

Like Zacks, Smith relies on video clips to better understand how the brain processes events. Instead of cooking or cleaning, his videos show a person shopping at a store, setting up a printer, or performing other mundane tasks. In various experiments, viewers press buttons when they believe an event is beginning or ending. Smith then tests participants’ memory of the video with a series of written questions.

Smith found that older people have a harder time processing events, which may play a role in age-related memory decline. "There may be a way to intervene to help them remember events in their lives better," he said.

Further research

Zacks, Nguyen, Smith, and other members of the Department of Psychological and Neurobiological Sciences have ambitious plans to further study the brain’s ability to process and remember events. Zacks’s team is working on using fMRI to track 45 participants’ reactions to videos of everyday events in real time. “We’re studying the actual neurodynamic processes of these cognitive functions,” Zacks said.

Another study tracks eye movements, providing new insight into how we see the world. "When people watch everyday activities, they spend a lot of time watching people's hands," Zacks explained.

Smith is currently using video-based experiments to test whether he can improve the memory of study participants — including older adults and those with Alzheimer’s disease — by making it easier to identify boundaries between events. Ultimately, he’d like to understand how observations of events are stored and maintained in long-term memory.

"Some people are clearly better than others at segmenting events into meaningful chunks," Smith said. "Can we improve that ability, and will that lead to better memory? Those are questions we're still trying to answer."