How neural networks are making soft robots easier to control

Robots built out of soft, flexible materials hold the potential to do things that rigid robots can’t. Lacking traditional links and joints, soft robots have more degrees of freedom than their rigid cousins, making it possible for them to navigate tight environments. And because they’re soft, they are generally considered to be safer in situations where they need to interact with people – think, for example, of a robotic arm designed to deal with human anatomy in medical interventions.

Yet soft robots are difficult to control. They have infinite degrees of freedom, and the movement of their materials causes friction and non-linear deformation which are difficult to model mathematically in a reliable way, explains Cesare Stefanini, Professor of Robotics at MBZUAI. But, he says, this is an area where artificial intelligence can help.

Math and data

At a basic level, there are two common approaches to robotic control. Model-based approaches are often used with rigid robots designed to work in controlled environments, predicting a robot’s behavior with precision.

Data-driven approaches are based on observed data about how a robot moves. Data related to a robot’s movements are captured and used to train a system, typically a neural network, about how to control the robot.

Stefanini says that a growing number of researchers have become interested in data-driven approaches. One reason is that advances in data collection and processing have made doing so feasible. Another is that robots themselves are becoming more complex in their design, as researchers look to nature for inspiration.

Where model-based approaches fall short, data-driven approaches can fill the gap, he says.

Stefanini is co-author of a study on a new data-driven approach to robotic control that was recently published in the journal IEEE Transactions on Robotics and presented at the International Conference on Robotics and Automation (ICRA) in Vienna, Austria. In the study, Stefanini and his co-authors propose a configuration and space planning control strategy for soft robot arms. They use a recurrent neural network with an architecture known as bidirectional long short-term memory (biLSTM) to learn the behavior of the robot. They tested their approach on several tasks, like reaching a target and avoiding obstacles, and found that theirs performed better than other methods.

Zixi Chen, Qinghua Guan, Josie Hughes, and Arianna Menciassi are the co-authors of the study.

How they did it

The researchers’ soft robotic arm is shaped like a tube divided into three sections, or modules. Cables run through the middle of the three modules and are connected to motors. Force can be applied to any one or combination of the sections at a time. Separating the arm into modules allows each one to be treated like a joint, turning a continuous system into discrete systems, breaking down a large problem into smaller ones.

That said, even with the discrete modules, control of the system is challenging due to friction and non-linear deformation. Another challenge is known as hysteresis, which describes how the current state of a robotic system depends not only on its inputs but on the history of how it got to its current position. Accounting for this requires capturing — and making sense of — previous positions of the robot in relation to time.

This is where the researchers turned to the neural network.

They moved the robot in many different positions using two strategies, one that moved the robot in random directions and another that moved it more deliberately. Across these movements, they used cameras to capture the robot’s “ground truth” position and encoders in the motors and other sensors in the robot to capture internal sensing data. They compared the observed and internal data and used them to train the biLSTM neural network to learn the robot’s behavior. The neural network also acts as the system’s forward model, predicting how it will respond to an input.

More versatile control of soft robots

The researchers found that their approach performed better than others on position and orientation control tasks. And they were able to have their robot perform other challenging tasks, like avoiding obstacles.

Stefanini says that the approach made controlling the system more precise than would otherwise be the case. And because the neural network itself is compact, it can be run locally on the robotic system. “We don’t need to use communication to connect to a high-performance computer, which would result in delays,” Stefanini says. “You can do everything on board.” In addition, when the system is operating, it doesn’t need the optical tracking system and works with internal sensing feedback alone.

Other studies had used neural networks to control soft robots, but these focused on individual tasks. The researchers’ approach can adapt to many different tasks.

While this work represents the potential of data-driven approaches to robotic control, Stefanini says that it is built on fundamental insights generated by model-based approaches. And while data-driven approaches are advancing today, traditional ideas about robotic control are still important for the field. “We will never abandon what we have already learned, of course,” he says. “We want to combine insights from the model-based approach with the data-driven approach.”