More than meets the eye: Identifying hidden causal variables with causal representation learning

Over the centuries, our understanding of the natural world has often been advanced by the discovery of phenomena in nature that were previously hidden to us. The study of infectious disease, for example, was greatly advanced by the invention of the microscope, which made it possible to observe pathogens like bacteria and viruses that aren’t visible to the eye alone. Genetic sequencing technologies provide researchers a way to gain insight into the relationship between genes and health, which was not possible simply by observing a person’s physical condition. The examples go on.

Kun Zhang, acting chair of machine learning, professor of machine learning and director of the Center for Integrative Artificial Intelligence (CIAI) at Mohamed Bin Zayed University of Artificial Intelligence, is working to develop machine-learning techniques that can be used to identify what are known as hidden causal variables, the “underlying concepts or objects” that drive the relationships between cause and effect in the world.

“A fundamental way to increase human knowledge is to uncover the hidden world by analyzing what we can measure and making the hidden world transparent to us,” Zhang said.

Zhang is first author, along with researchers at Carnegie Mellon University, of a study that proposes a new approach for identifying hidden causal variables based on observable data. The study will be presented at the International Conference on Machine Learning (ICML 2024), which is being held this month in Vienna. Researchers from MBZUAI are authors on 25 studies that will be presented at the conference, which is one of the largest and most significant annual meetings in the field of machine learning.

A major challenge in the study of causality is determining what phenomena or objects are in fact responsible for cause and effect. Decades ago, it was thought that causality could be ascertained by analyzing observable variables. “But there are many examples that encourage us to abandon this assumption,” Zhang said.

Take for example an image created by a digital camera. The pixels in a digital photo have observable characteristics, such as brightness and color, and across the photo pixels have relationships to each other. But there are no causal relationships between pixels, in that a pixel causes another to exhibit certain characteristics, Zhang explained. The pixels in the photo are simply reflections of phenomena in the world, such as light and spatial relationships between objects.

Zhang and his colleagues’ approach is called causal representation learning and it assumes that measured variables are generated by variables that aren’t directly observed, which are also known as latent variables. Yet because measured variables are the result of latent causal relations and variables, machine-learning models can be used to derive the latent causal relations and variables from observable data.

Zhang and his team’s approach can be used in different settings and with data that has a variety of distributions. “In this paper, we are talking about fundamental settings in which we can determine hidden causal variables and their relations when we have multiple distributions,” Zhang said. For example, he is interested to use this approach to analyze functional magnetic resonance imaging (fMRI) data to better understand connections between regions of the brain. Doing so is challenging, however, due to the variability of fMRI data. “From one day to the next, fMRI data of the same patient can change,” Zhang said. “But causal representation learning provides an opportunity to uncover the relations between brain regions” that isn’t possible with other techniques.

Though previous research has also explored causal representation learning, this is the first study that does so in so called non-parametric settings, which are more open-ended and variable than parametric settings.

In addition, the researchers’ approach doesn’t require experimental intervention, which is a technique that makes changes to a variable with the goal of identifying cause and effect. Clinical trials for new drugs are examples of an interventionist approach, where one group of patients receives the drug that is being studied and the “control” group does not. With all else being equal, a clinical trial designed in this way can provide scientists with an idea of the effect of the drug on patients.

Zhang’s long-term aspirations are ambitious: he wants to automate the process of discovering the hidden variables that drive cause and effect in the world. “As machine scientists, I think we should find a way to increase human knowledge. In order to do this, we need to discover the truly existing relations so that we can understand what’s going on and find a way to measure and control these relations.”

“Newton discovered F = ma and Einstein discovered E = mc². How can machines gain similar capabilities?” Zhang asks.

The hope is that doing so would dramatically accelerate scientific discovery.