EPFL Creates First Functional Language AI

EPFL researchers have developed the first artificial intelligence model of language in the brain that accounts for both the arrangement and function of neurons.

The human brain is highly organized, with neurons (nerve cells responsible for sending chemical and electrical signals) arranged in tissue and forming clusters based on their functional roles. Although researchers understand that certain groups of neurons specialize in tasks such as processing verbs or nouns, the formation of these functional groups remains unclear.

Previous AI language models have captured these specialized clusters of neurons but have not examined their spatial arrangement in the brain.

Researchers at the NeuroAI Laboratory, affiliated with the School of Computer and Communication Sciences (IC) and the School of Life Sciences (SV), have developed a new AI language model called TopoLM. This model replicates not only the functional clustering of neurons but also the spatial arrangement of these cells in the brain for the first time.

Building on work around how the brain processes vision, we made some relatively small changes to how a language model organizes itself internally, adding a rule that encourages the model's internal representations to be 'smooth' spatially. The resulting model, TopoLM, develops spatial clusters of its internal components that functionally match the activity we see in human brains when they process language.

Martin Schrimpf, Assistant Professor and Head, NeuroAI Lab, EPFL

In their study, TopoLM: Brain-Like Spatio-Functional Organization in a Topographic Language Model, the researchers describe how TopoLM accurately predicts the development of the spatio-functional organization of the language system in the cortex, the brain's outer layer. This paper was selected for oral presentation at ICLR 2025, one of fewer than 2 % of papers chosen for this honor.

Schrimpf explained, “This was basic research to understand how spatial clusters or functional clusters in the brain originate in the first place, and what this new model suggests is that it might be driven by a single, basic rule about spatial organization where close-by neurones just tend to behave similarly.”

The researchers suggest that TopoLM provides a framework for improving AI’s functional alignment with human cognition, with potential applications in neurolinguistics and brain-inspired computing.

This is an exciting step towards building artificial intelligence systems that are organized more like the human brain. One of our main goals is to build better models of the brain in general, and with TopoLM, we are one step closer to clinical applications that might help us to help people with language disorders or similar language deficits.

Badr AlKhamissi, Study Author and Doctoral Assistant, NeuroAI Lab, EPFL

This study also addresses the issue of interpretability, which refers to understanding the internal workings of complex Large Language Models (LLMs) and how they operate in practice. In typical LLMs, each artificial "neuron" is represented mathematically by vectors. To understand what the model has learned, these vectors need to be examined individually or in small groups.

TopoLM makes it possible to observe and examine significant groupings of vectors when it organizes its internal components into clusters. Since this organization reflects meaningful categories, it becomes easier to understand how the model represents and processes language.

What are the next steps in this research? The EPFL team plans to test the model's predictions in human brains.

“This model works so well, far better than we expected, that we want to test if its predictions are true. There are clusters in the model that we have not yet observed in the human brain because nobody has looked for them yet. We will be working with colleagues in the United States who experimentally work with humans to run a new imaging study to find out if these clusters also appear in the human brain,” concluded Schrimpf.

Martin Schrimpf collaborated with Badr AlKhamissi, Neil Rathi, and Hannes Mehrer to develop the TopoLM model.