EPFL’s Swiss Plasma Center (SPC) has several years of experience in plasma physics and plasma control approaches. DeepMind is a scientific discovery company, acquired by Google in 2014, that is dedicated to unraveling intelligence to progress science and humanity.
Plasma inside the TCV tokamak. (Image Credit: Curdin Wüthrich /SPC/EPFL).
DeepMind and SPC have formulated a new magnetic control technique for plasmas based on deep reinforcement learning and, for the first time, applied it to real-world plasma in the SPC’s Tokamak Research Facility (TCV). Details of their study have been published in the journal Nature.
Tokamaks are donut-shaped devices for carrying out research on nuclear fusion, and the SPC is one of the few research organizations in the world that has an operational one. These devices employ a robust magnetic field to confine plasma at very high temperatures — hundreds of millions of degrees Celsius, much hotter than the sun’s core — so that nuclear fusion can take place between hydrogen atoms.
The energy emitted from fusion is being explored for use in producing electricity. The reason for the SPC’s tokamak uniqueness is that it facilitates a range of plasma configurations, hence its name: variable-configuration tokamak. That means researchers can use it to explore new methods for confining and regulating plasmas. Plasma’s configuration is associated with its shape and position in the device.
Controlling a Substance as Hot as the Sun
Tokamaks form and preserve plasmas through a series of magnetic coils whose settings, particularly voltage, must be carefully regulated. Otherwise, the plasma could strike against the vessel walls and weaken.
To stop this from taking place, scientists at the SPC first test the configurations of their control systems on a simulator before employing them in the TCV tokamak.
Our simulator is based on more than 20 years of research and is updated continuously. But even so, lengthy calculations are still needed to determine the right value for each variable in the control system. That’s where our joint research project with DeepMind comes in.
Federico Felici, Study Co-Author and Scientist, Swiss Plasma Center, EPFL
DeepMind’s experts created an AI algorithm that can generate and maintain precise plasma configurations and trained it on the SPC’s simulator. The training involved first allowing the algorithm to try many different control approaches in simulation and accumulating experience.
Based on the accumulated experience, the algorithm created a control strategy to yield the requested plasma configuration. This required the algorithm to first run through many different settings and then analyze the plasma configurations that ensued from each one. Next, the algorithm was required to work the other way — to generate a particular plasma configuration by detecting the right settings.
After being trained, the AI-based system could create and preserve a wide variety of plasma shapes and progressive configurations, including one where two individual plasmas are maintained concurrently in the vessel. Finally, the researchers verified their new system directly on the tokamak to observe how it would act under real-world circumstances.
The SPC’s partnership with DeepMind goes back to 2018 when Felici first met DeepMind experts at a hackathon at the company’s London headquarters. There, he described his research team’s tokamak magnetic-control issue.
“DeepMind was immediately interested in the prospect of testing their AI technology in a field such as nuclear fusion, and especially on a real-world system like a tokamak,” says Felici.
Our team’s mission is to research a new generation of AI systems—closed-loop controllers—that can learn in complex dynamic environments completely from scratch. Controlling a fusion plasma in the real world offers fantastic, albeit extremely challenging and complex, opportunities.
Martin Riedmiller, Study Co-Author and Control Team Lead, DeepMind
A Win-Win Collaboration
After talking with Felici, DeepMind was keen to work with the SPC to create an AI-based control system for its tokamak.
We agreed to the idea right away, because we saw the huge potential for innovation. All the DeepMind scientists we worked with were highly enthusiastic and knew a lot about implementing AI in control systems.
Ambrogio Fasoli, Study Co-Author and Director, Swiss Plasma Center, EPFL
For his part, Felici was fascinated with the remarkable things DeepMind can achieve in a short time when it concentrates its efforts on a particular project.
DeepMind also gained a lot out of the collaborative research project, illustrating to both parties the advantages of using a multidisciplinary method.
The collaboration with the SPC pushes us to improve our reinforcement learning algorithms, and as a result can accelerate research on fusing plasmas.
Brendan Tracey, Study Co-Author and Senior Research Engineer, DeepMind
This project is likely to pave the way for EPFL to obtain other collaborative R&D opportunities with more organizations.
“We’re always open to innovative win-win collaborations where we can share ideas and explore new perspectives, thereby speeding the pace of technological development,” says Fasoli.
Journal Reference:
Degrave, J., et al. (2022) Magnetic control of tokamak plasmas through deep reinforcement learning. Nature. doi.org/10.1038/s41586-021-04301-9.