Going under the hood to improve AI efficiency

For Xiaosong Ma, the ever-expanding world of artificial intelligence provides an exciting challenge of efficiency and sustainability. 

“What I enjoy working on most are themes of ‘how do we reduce waste’ and ‘how can we use resources better,” smiles the acting department chair and professor of computer science at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI). 

“With AI, everybody agrees that it is a huge workload right now. Every day it is burning a substantial and growing fraction of the total data center energy or total computer cycles in the world. Every day there are tons and tons of wasted cycles and energy, and this is a really interesting and important efficiency problem to look at, both from the systems and the data perspective.”

These are the kinds of challenges Xiaosong aims to address through the computer science department — a new department at MBZUAI that she believes enhances the university’s holistic approach to AI. 

“People today focus a lot on AI algorithms and AI applications, but less effort is spent looking under the hood to see whether we can do the same things but using a fraction of the resources,” she explains. “MBZUAI is the perfect place to do this because we’re working together with so many experts in different AI domains, such as NLP, robotics, machine learning and more.” 

From supercomputing to systems 

Xiaosong’s passion for ‘looking under the hood’ and understanding how to improve on waste and sustainability was born out her parents’ work as software engineers in the 1960s and 70s.  

“I started at a pretty young age because my parents were among the first generation of software engineers in China,” she says. “Their major in university was computational math, but back then they didn’t really have computers to work on. Later, they would use Siemens and IBM machines and began to learn programming languages, and as a result, as a child I was exposed to things like Basic.  

“When I finished high school, in the summer before starting college for computer science, my mother actually taught me programming in C, which remains a really wonderful memory.” 

Following her undergraduate education at Peking University, Xiaosong went to the US to start her Ph.D. studies at the University of Illinois. There she got exposed to graduate-level CS training, specifically in high-performance computing, which became a key focus for her. 

“This is a truly fascinating area where you work with parallel computers and focus on issues of efficiency and scalability,” she says. “And today, if you look at large scale AI training or inference, a lot of the problems and solutions are actually shared.” 

After graduating, Xiaosong joined North Carolina State University as professor while jointly working with Oak Ridge National Laboratory, where she continued her research with high performance computers. She then moved to the Qatar Computing Research Institute at Hamad Bin Khalifa University as principal scientist, where she took on a quite different challenge.  

“This is where my work transferred a bit from the supercomputing world to the systems area,” she explains. “I was doing experiments on one or a few nodes, which allowed me to focus more within these computers and look at the smaller scale of things. This really drew me even more into areas such as storage hierarchy — going down to tiny, tiny areas like the register and different levels of cache, where storage becomes faster, smaller, and way more expensive in terms of money and energy consumption. 

“We looked at questions such as how should we move data? Where should we put data? Where should we promote them, and demo them? This is a very interesting topic which you can study for so many different applications — especially in AI.” 

Making a sustainable impact 

Addressing these questions has helped Xiaosong to make significant contributions to waste reduction, efficiency, and sustainability in recent years — notably with large scale distributed graph systems, and graph random walk systems.  

For example, a system she and her team published in 2016 called Gemini became one of the highest impact papers of OSDI (a leading systems conference) that year. 

“Gemini generated good, single machine graph computing efficiency, and also scalability to multiple nodes,” she says of the system that they found to outperform all well-known existing distributed graph processing systems, delivering up to 39.8x improvement over the fastest among them.  

“That system has influenced several internet companies’ graph engine design, for example Tencent in China, which owns the powerful tool WeChat that’s used by billions of people.” 

Following that, Xiaosong worked on efficient graph random walk algorithms, which influenced system implementation by the likes of ByteDance — the owner of TikTok. Another of her joint projects, ShenTu — a large-scale graph engine that ranked 70 trillion web page links in seconds, was nominated for the prestigious Association for Computing Machinery Gorgon Bell Prize in 2018. It also performed spam page detection on a web graph with 12 trillion links, considering factors such as popularity and trustworthiness.  

Given these achievements, it’s no surprise that Xiaosong has been honored with multiple awards and accolades — including both the DOE Early Career Principal Investigator Award and the NSF CAREER Award — and was named as an ACM Distinguished Member in 2020. 

But why is her work important to the future of AI? For Xiaosong, efficiency across different system layers, and reducing waste in system software to better support applications, is a fundamental need as AI continues to grow. 

“Whatever we do from the architecture to the system level is to make things more efficient, and that makes AI more sustainable,” she says. 

“But it’s not just that. Computer science works on systems, security, and safety topics, and these are all deeply related to AI — they are crucial. 

“We need people to do this vital work. We need people to deal with next generation chips and memory design, as well as new databases. This might not be the glamorous side of AI for a lot of people, but it is essential.” 

Leading MBZUAI’s computer science department, Xiaosong will focus on these various aspects, helping students learn how to analyze complex computer science and AI problems, take a scientific, innovative, ethical, and socially responsible approach to computer science research, and solve complex problems in the field. 

“We need to encourage students who love algorithms, designing next generation data structures, and playing with operating systems and even hardware,” she says. 

“The most important thing is that we give them a platform, a voice, and an environment to do innovative work and bring out great research advances.”