Can AI Learn Like Us? Unveiling the Secrets of Spiking Neural Networks

It’s no secret that AI is data hungry and requires powerful computing resources to operate. This places limitations on the ability of AI to fulfil its potential in certain situations, such as when smaller devices with limited power and processing capacity are being used, or when AI is needed in remote areas with limited connectivity. Examples could include the use of portable CT scanners in hospitals and clinics, security cameras, monitoring equipment and drones in remote areas.

This is a challenge that Hilal Mohammad Hilal AlQuabeh, who recently completed his Ph.D. in machine learning (ML) at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), was keen to tackle. The Jordanian national’s research focused on techniques designed to improve the efficiency with which ML algorithms learn, with minimal or no loss of efficacy. It is a tough field of study within AI, but one that will become increasingly important as AI grows and expands into ever more areas and runs on a full range of devices.

“There are many areas of machine learning that I find interesting, but I was keen to tackle this challenge because, if not addressed, it could have a detrimental effect on sectors including healthcare, transport and logistics, and agriculture – particularly in remote areas. And I believe that the benefits of AI should be available to everyone,” AlQuabeh said. 

AlQuabeh’s research focused on ML methods known as pairwise learning and multi-instances learning, and how to make them operate effectively even in lower resource environments. “Essentially, I looked at how machines can learn from different types of examples that are designed to enhance critical problem-solving and to improve their ability to spot nuanced details that they might ordinarily miss – and how to achieve this with limited information and resources,” he said.  

Pairwise learning in AI is akin to teaching a computer to make decisions by comparing things two at a time. Instead of looking at the whole picture, it focuses on understanding which of two options is better, AlQuabeh explained.  

“This method helps in ranking or choosing the best among several choices by evaluating them in pairs. For example, when recommending products, pairwise learning helps the algorithm learn which product is preferred over another, making decisions based on comparisons and improving its ability to understand user preferences,” AlQuabeh said.  

The technique can be useful in applications such as anomaly detection and fraud prevention, and information retrieval and ranking.  

Multi-instance learning, on the other hand, deals with sets of data points – referred to as bags – where at least one element within the set belongs to the target class. The task can be likened to showing someone a bunch of flowers and asking them to identify the type of flower present, even if not all flowers in the bunch belong to that type. The technique can be applied to tasks such as drug discovery and anomaly detection.  

AlQuabeh took his research to another level by exploring how these techniques could help algorithms to learn more effectively from unbalanced data or learning, where the distribution of classes in a dataset is uneven.  

“An example of unbalanced data could include the prevalence of disease in a randomly selected population sample. The vast majority of people will show no signs of disease, and this imbalance can lead the model to perform well on the majority class but poorly on the minority class,” AlQuabeh said. “My research involved developing algorithms that can better understand this type of data.” 

While doing research for his Ph.D., AlQuabeh also had the opportunity to work on developing ML models to help reduce the energy consumption required by ML in certain applications – applying his learnings to a high-tech drone project by another Abu Dhabi-based entity. For this, AlQuabeh took inspiration from the field of neuroscience, and deployed a technique called spiking neural networks (SNNs), which mimic the spikes or brief electrical pulses that enable communication between synapses in the brain. Unlike traditional neural networks, SNNs consider the timing of these spikes, which can enhance efficiency and real-time processing capabilities. 

To further raise efficiency, AlQuabeh combined SNN techniques with the concept of sparse neural networks, which are, as their name suggests, neural networks with far fewer connections than in a regular neural network. The method is akin to funneling dispersed traffic in a city onto a small number of efficient highways, AlQuabeh explained.  

“When creating spiking in a sparse neural network, the math becomes simpler,” he said. “Which means you can encode information more cheaply and efficiently in the memory and reduce the computation load.” 

AlQuabeh also investigated the merits of SNNs when using adversarial training for algorithms, a rigorous training technique which seeks to confuse the AI system by adding potential complexity to the data, such as obscuring or blurring parts of training data for computer vision systems.  

On the back of his research, AlQuabeh published a paper on the use of SNNs with adversarial training, titled ‘Certified Adversarial Robustness for Rate Encoded Spiking Neural Networks’, at the prestigious International Conference on Learning Representations (ICLR) in Vienna, Austria, in 2024. He said that the paper, which was co-authored by Huan Xiong and Bin Gu, assistant professors of ML at MBZUAI, as well as Bhaskar Mukhoty and Giulia De Masi, was well received and demonstrated an advancement in the field. “It was a great experience to publish a paper on an area that has the potential to bring about real improvements to the efficiency of AI systems and to see all the hard work pay off,” he added. 

Looking ahead, AlQuabeh is keen to continue working on theoretical machine learning challenges and would like to continue conducting research at MBZUAI.  

He has been highly impressed by the university, its facilities, the caliber of the faculty, and the camaraderie among students and researchers. Amid the highly complex theoretical work, AlQuabeh has also appreciated MBZUAI’s sports facilities, including its swimming pool, which he uses regularly, and the many friends he has made from countries including France, Montenegro, India and China.