Improving through argument: a symbolic approach to fake-news detection

Like any powerful technology, large language models (LLMs) can be used for good or for ill. While these systems hold the potential to benefit society, it’s also necessary to be aware of the risks they pose, says Chong Tian, a doctoral student in machine learning at MBZUAI.

One major hazard is that they can be used to create sophisticated, fake news on a massive scale.

Fake news detection is an active area of study, and researchers have developed different methods to identify it. But it’s an extremely difficult task because, by its nature, fake news is a moving target.

Fake news often relates to current, or in some cases, real-time, events. Detectors that are trained on data about past events might not be effective against fake news about events that happen in the future, Tian says.

The tools that people use to generate fake news are also continually evolving and people develop new strategies to evade detection. And people change, too: the public’s understanding and awareness of events is different from one day to the next. “New narratives are constantly being crafted to exploit current societal moods, and a static, isolated approach to fake-news detection simply cannot keep up with these interconnected dynamics,” Tian says.

To address these challenges, Tian and researchers from MBZUAI developed a new approach for fake-news detection they call symbolic adversarial learning framework (SALF). It employs a method called agent symbolic learning in an adversarial framework. SALF uses two LLM-powered agents — a generator and a detector — in a setup that moves back and forth like a debate. The generator produces a passage, and the detector tries to identify hallmarks that are common in fake news.

At the end of the debate, another LLM, playing the role of a “judge,” is given the full transcript of the debate and determines the veracity of the generator’s statements. Throughout this process, both agents are optimized and “co-evolve.” The idea is that the generator gets better at writing convincing fake news while the detector gets better at identifying it.

And they did.

The researchers tested the SALF generator on state-of-the-art fake-news detectors and found that it degraded performance by 53.4% on a Chinese-language dataset and by 34.2% on an English-language dataset.

Tian; Assistant Professor of Machine Learning, Qirong Ho; and Assistant Professor of Natural Language Processing, Xiuying Chen are authors of a study about SALF that was recently presented at the Conference on Empirical Methods in Natural Language Processing (EMNLP 2025) in Suzhou, China.

The symbolic learning approach

The speed at which LLMs are rapidly improving provided an inspiration for the team’s research. Even in few-shot, or even zero-shot, scenarios, Tian says, current LLMs can generate fake news that evades carefully hand-crafted detection models. “This led us to focus our work on a higher level of abstraction, and agent symbolic learning is a natural fit for that goal,” he says.

The great benefit of agent symbolic learning is that it works in natural language. It simulates two fundamental algorithms used to optimize neural networks, backpropagation and gradient descent. But instead of numerical computations, the system generates text-based descriptions of what went wrong and how to improve it. Training happens through “iteratively refining the prompts for both the generation and detection agents based on their performance,” the researchers explain.

Because this process happens in readable text, it can easily be interpreted in a way that other approaches can’t. Most fake-news detectors act like “black boxes” where it’s difficult to understand why they flagged one thing as true and another as fake. With SALF, the different versions of the fake news, the arguments made by the agents, the symbolic loss, and the symbolic gradient are all in human-readable text, Tian says.

Interpretability is important when it comes to fake-news detection because people need to understand how and why models make the decisions they do.

What they found and what comes next

SALF was able to teach the agents quite a bit about how to produce convincing fake news.

On a Chinese-language dataset, Weibo21, SALF degraded the performance of a DeepSeek V3-based detector by 85% according to a metric called F1-fake score. “This is fascinating because it reveals that while LLMs possess vast general knowledge, their out-of-the-box application in a specialized domain like fake-news detection can be surprisingly brittle,” Tian says.

Since SALF also uses DeepSeek V3 as its underlying LLM, this degradation in performance could be due to a phenomenon known as model coupling, a vulnerability where an attacker uses the same model to generate fake content that is used to detect it. Model coupling illustrates the “risks of relying on a monoculture of models for both generation and detection,” Tian says.

The detector also improved its ability to identify fake news by 7.7% on content produced by the generator. The system was also shown to converge quickly, with results optimized after two rounds.

The researchers plan to continue their work on SALF.

Since so many people around the world get their news from short videos on social media, they plan to expand their SALF framework to multimodal data. Research on detecting misinformation in this medium is still in the early stage, and very little of it is being done with LLMs and with results that are interpretable.

The researchers also see broader potential for SALF as a general-purpose framework that can be applied to other challenges related to LLMs, like hallucination. In this case, an adversarial setup could be created to train an LLM to detect and correct its own factual inconsistencies.

That said, no one technological solution is going to fully solve the problem of fake news. “Often the weakest link is not the technology but the human element,” Tian says. “Social engineering will always be a more effective key for bypassing defenses than a purely technical exploit.”