Mechanics and mimicry: Yoshihiko Nakamura’s journey in robotics

As humans, we have long been fascinated by the prospect of building machines that look and act like us. That said, the desire to create robots — or androids, or automatons, as they have also been known — has often been met with a mix of curiosity and skepticism.

Indeed, when Yoshihiko Nakamura, professor of robotics at the Mohamed bin Zayed University of Artificial Intelligence, began his undergraduate studies in Japan in the 1970s, he didn’t imagine that robots could be the focus of academic research. “For me, they were the subject of manga and science fiction,” he says.

At the time, researchers who were working in robotics described their innovations as “artificial arms” or “artificial legs” instead of referring to robots and risking not being taken seriously, Nakamura adds.

What’s more, there weren’t academic journals or conferences dedicated specifically to the field of robotics until the mid-1980s. “Professors in that period tried to establish journals that focused on robotics in association with the mechanical engineering society or electrical engineering society, but those organizations weren’t interested in welcoming robotics,” he explains.

It was during this period, however, that the robotics field began to become more formalized and established, just as Nakamura was completing his doctorate at the University of Kyoto.

Nakamura has always been curious about the mathematics of movement, which has proven to be a consistent theme in his research, even if the specific innovations and applications he has worked on have varied widely. He has also been interested in the mathematics of optimization and “non-linearity in robotics, since there are many complex non-linear phenomena in robotics,” he notes.

After stints teaching at the University of Kyoto and the University of California, Santa Barbara, Nakamura joined the newly formed department of mechano-informatics at the University of Tokyo in 1991. He taught and conducted research there for nearly three decades and was appointed professor emeritus in 2020. Nakamura joined MBZUAI last year.

Imitation game

Nakamura has been motivated in his research by ideas from a variety of fields, including neuroscience, psychology and linguistics.

In the late 1990s, neuroscience researchers published a foundational study in which they identified a set of cells in the brain of monkeys that played an important role in movement and learning. They called these cells “mirror neurons” due to their connection to the process of “mimesis,” or imitation, which melds watching and doing. By analyzing the brains of monkeys, neuroscientists found that mirror neurons were active when monkeys observed behavior in other monkeys and also when they performed the behavior themselves.

Mirror neurons exist in other mammals as well, including humans, and seem to play a role in learning and language acquisition.

As Nakamura and coauthors described in research that was inspired by the original mirror neuron paper, the neuroscientists’ findings “suggest that the behavior recognition process and behavior generation process are combined as the same information processing scheme” in humans.

Biological concepts have often served as inspiration for computer scientists, and Nakamura adapted this idea of mirror neurons to the realm of robotics. “Humans start building knowledge by mimicking others around them. We were interested in developing a way in which robots could learn and build knowledge from others as well,” he says.

A challenge of designing a program that can learn by observing behavior is that the program must turn the continuous stream of information that it gathers observing the world into a series of concrete events that can be classified into symbols, Nakamura explains. “The fundamental model of the intelligence of animals, particularly mammals, is that they have a type of symbolic understanding of the world,” he says.

By discretizing movement and classifying these movements into symbols, it is then possible to add another layer to a system that can form a relationship between the symbols and human language.

Kinematics and quakes

In the 2000s, Nakamura and collaborators also developed efficient ways to compute the kinematics and dynamics of very complex human motions, which had previously been extremely computationally intensive. These findings represented major achievements and advanced the field’s understanding of how human motion can be recreated by machines.

Following the earthquake and tsunami that struck Japan in 2011, Nakamura applied his expertise to a robot that was built to be used in the Fukushima nuclear power plant, which was flooded by the tsunami. “The tsunami was a big event, and we were all shocked” by the impact it had on Japan, Nakamura says. “Many robotics researchers in Japan at the time wanted to do what we could to help society.”

Nakamura and colleagues developed a hydraulic technology that was used in a humanoid robot which they brought to the nuclear plant as part of a competition to develop new technologies to assist efforts at the reactor.

Though in the end their robot wasn’t used at the site, the experience influenced the direction of Nakamura’s research. “I’m still of course interested in scientific motivations and mathematical optimization, but at the same time I want to work on projects that could have an impact on society,” Nakamura says.

In the past several years, Nakamura has continued to develop technology designed to capture and model human motion. At the University of Tokyo, he and his colleagues built a system to capture the motions of Japanese athletes, including a judo champion, table tennis Olympian, and football players. “Using only four cameras we were able to capture three-dimensional human motions without using any other technology,” Nakamura says.

Future directions at MBZUAI

Nakamura decided to come to MBZUAI because he was intrigued by the interdisciplinary nature of the university and the opportunity it provides to collaborate with colleagues in natural language processing, machine learning and computer vision.

Robotics has significantly evolved since Nakamura began his career, with many more journals, conferences, and professional societies dedicated to its study. It is also an area that is abundant with possibility for students. “The field is becoming very busy, and it’s exciting to see that there are many different directions in robotics research that students can choose from,” Nakamura says.

Even with these many opportunities in robotics for students today, Nakamura believes that the students themselves are best positioned to determine the direction of their research: “I think perhaps the best way students can uncover something new is to follow their own subtle differences of interests in their own research,” he says.