Scientists Translated Brain Signals Into Movies With Surprising Accuracy

Mouse brain activity was used to recreate 10-second videos, offering a new way to study how vision is represented in the brain.

Scientists led by University College London (UCL) have reconstructed videos using only brain activity recorded from mice, allowing them to recreate what the animals were seeing.

The findings, published in eLife, could help researchers better understand how the brain handles visual information and may offer new ways to study how different species experience the world around them.

In recent years, scientists have become increasingly interested in how the human brain makes sense of signals from the eyes. Researchers have shown images and movies to people in fMRI scanners and have tried to decode visual information in the brain down to the pixel level.

The new work follows that same broad goal, but it uses single-cell recordings in mice instead. This approach can provide a more detailed view of how the brain represents visual scenes. Using activity from the visual cortex alone, the team was able to produce high-quality reconstructions of videos the mice had watched.

Lead author Dr. Joel Bauer (Sainsbury Wellcome Centre at UCL) said: “We wanted to have a better way of investigating how the brain interprets what we see. The current methods of understanding what specific groups of neurons are representing are not very generalizable to situations that haven’t been specifically tested for. And so, we wanted to develop a method that can capture what is being represented in the brain and compare that to reality.”

Neurons recreate visual scenes

The method could help scientists examine the gaps between what is actually shown and how the brain represents it. Those differences may reveal how particular visual cues influence neural representations.

Dr Bauer and colleagues used a dynamic neural encoding model that had been developed by another team for the 2023 Sensorium Competition. The model predicts the activity of individual neurons (brain cells) based on movies shown to mice, while also taking into account the animal’s own movements and pupil diameter.

Working with the same dataset, the UCL team improved the model by comparing two things: the predicted neuron activity if a mouse had seen a blank screen, and the actual activity of the neurons (measured using a microscopic imaging technique that detects which individual brain cells are firing based on localized boosts in calcium levels). This allowed the scientists to begin with a blank movie and gradually adjust its pixels through an algorithm until the output video closely matched the one shown to the mouse.

More cells improved detail

After the model had been trained, the researchers could reconstruct a 10-second movie from a mouse’s neural activity alone. The brain activity was recorded while the mouse watched a video that had not been used to train the model.

Dr. Bauer added: “Using this approach, we were able to achieve high-quality reconstructions of 10-second video clips. The accuracy of the reconstructions improved with the inclusion of data from more individual neurons, demonstrating the importance of comprehensive neural data.”

To measure how well the reconstructions matched the originals, the team used pixel correlation, comparing each pixel in the original movie with the corresponding pixel in the reconstructed version. The timing of the two videos showed only minimal differences. The researchers now plan to improve both the resolution and visual coverage of the reconstructions by collecting data that can support sharper and wider reconstructions of the visual scene.

Perception differs from reality

The team now intends to use the technique to learn more about how the brain processes visual information. In particular, they want to understand why the brain’s visual representations can differ from what is directly in front of the eyes.

Dr. Bauer concluded: “We don’t have a perfect representation of the world in our heads. The visual processing pipeline skews and warps our representation in a way that modifies information. This deviation between reality and representations in the brain is not necessarily an error but a feature, reflecting how our minds interpret and augment sensory information. We want to explore how this happens in the brain.”

Reference: “Movie reconstruction from mouse visual cortex activity” by Joel Bauer, Troy W Margrie and Claudia Clopath, 10 March 2026, eLife.
DOI: 10.7554/eLife.105081

Never miss a breakthrough: Join the SciTechDaily newsletter.
Follow us on Google and Google News.