Autonomous AI System Selectively Grips and Moves Individual Molecules

Molecules are known to be the building blocks of day-to-day life. As such, a majority of the materials are made up of these molecules, somewhat like a LEGO model containing a host of different bricks.

Although each LEGO brick can be removed or shifted easily, this task cannot be done so easily in the nanoworld. For example, molecules and atoms act in an entirely different manner in macroscopic objects and every brick needs its own “instruction manual.”

Now, an artificial intelligence (AI) system developed by a team of researchers from Jülich and Berlin uses a scanning tunneling microscope to autonomously learn how to grasp and shift individual molecules.

Reported in the Science Advances journal, the new technique is not only applicable for research purposes but also useful for innovative production technologies, like molecular 3D printing.

Rapid prototyping, popularly known as 3D printing, enables fast and low-cost production of models or prototypes. This method has long since become a significant tool for the industry.

“If this concept could be transferred to the nanoscale to allow individual molecules to be specifically put together or separated again just like LEGO bricks, the possibilities would be almost endless, given that there are around 1060 conceivable types of molecule,” described Dr Christian Wagner, head of the ERC working group on molecular manipulation at Forschungszentrum Jülich.

But there is one issue. While the scanning tunneling microscope is a handy tool for moving individual molecules to and fro, there is always a need for a unique custom “recipe” to guide the tip of the microscope to spatially organize molecules in a targeted way.

However, it is not possible to compute or comprehend this custom recipe because the mechanics on the nanoscale are highly complex and variable.

But in the end, the tip of the microscope boils down to a rigid cone and not a flexible gripper. The molecules lightly attach to the tip of the microscope and can only be positioned in the correct location via advanced movement patterns.

To date, such targeted movement of molecules has only been possible by hand, through trial and error. But with the help of a self-learning, autonomous software control system, we have now succeeded for the first time in finding a solution for this diversity and variability on the nanoscale, and in automating this process.

Dr Stefan Tautz, Professor and Head of Quantum Nanoscience Institute, Forschungszentrum Jülich

A solution to this advancement lies in the supposed reinforcement learning—a unique variant of machine learning.

“We do not prescribe a solution pathway for the software agent, but rather reward success and penalize failure,” described Dr Klaus-Robert Müller, professor and head of the Machine Learning department at TU Berlin.

The new algorithm repeatedly attempts to solve the job at hand and gains knowledge from its experiences. It was only a few years ago that the general public came to know about reinforcement learning through AlphaGo Zero.

An artificial intelligence system like this autonomously created strategies for winning the extremely complicated game of Go without assessing the human players—and it learned to beat professional Go players just after a few days.

“In our case, the agent was given the task of removing individual molecules from a layer in which they are held by a complex network of chemical bonds. To be precise, these were perylene molecules, such as those used in dyes and organic light-emitting diodes,” described Dr Christian Wagner.

In this case, the unique challenge is that the force needed to shift the molecules should never surpass the bond strength with which the molecule is attracted by the tip of the scanning tunneling microscope, because this bond has a tendency to break.

“The microscope tip therefore has to execute a special movement pattern, which we previously had to discover by hand, quite literally,” added Dr Wagner.

The software agent initially carries out haphazard movement actions that tend to break the bond between the molecule and the tip of the microscope, but over time, it also develops rules as to which movement would be the most promising one for success in which scenario, and thus becomes better with every cycle.

But the application of reinforcement learning in the nanoscopic range comes with its own additional difficulties. The metal atoms constituting the tip of the scanning tunneling microscope can eventually end up moving slightly, which modifies the bond strength to the molecule every time.

Every new attempt makes the risk of a change and thus the breakage of the bond between tip and molecule greater. The software agent is therefore forced to learn particularly quickly, since its experiences can become obsolete at any time. It’s a little as if the road network, traffic laws, bodywork, and rules for operating the vehicle are constantly changing while driving autonomously.

Dr Stefan Tautz, Professor and Head of Quantum Nanoscience Institute, Forschungszentrum Jülich

To resolve this difficulty, the team allowed the software to learn an easy model of the setting where the manipulation occurs simultaneously with the initial cycles. Following this, the software agent concurrently trains both in its own model and in reality, which speeds up the learning process quite considerably.

“This is the first time ever that we have succeeded in bringing together artificial intelligence and nanotechnology,” Klaus-Robert Müller emphasized.

Up until now, this has only been a ‘proof of principle. However, we are confident that our work will pave the way for the robot-assisted automated construction of functional supramolecular structures, such as molecular transistors, memory cells, or qubits—with a speed, precision, and reliability far in excess of what is currently possible.

Dr Stefan Tautz, Professor and Head of Quantum Nanoscience Institute, Forschungszentrum Jülich

Artificial intelligence (AI) was given the task of removing individual molecules from a closed molecular layer. First, a connection is established between the tip of the microscope (top) and the molecule (middle). Then, the AI tries to remove the molecule by moving the tip without breaking the contact. Initially, the movements are random. After each pass, the AI learns from the collected experiences and becomes better and better. Video Credit: Copyright Forschungszentrum Jülich/Christian Wagner.

Journal Reference:

Leinen, P., et al. (2020) Autonomous robotic nanofabrication with reinforcement learning. Science Advances. doi.org/10.1126/sciadv.abb6987.