New agentic framework tests whether AI can write fact-checking articles

On any given day, a huge amount of misleading information is posted online across social media networks, message boards, and content platforms. Fact-checkers play an important role in the fight against mis- and disinformation by identifying dubious claims and writing articles that explain why these claims are misleading. But composing fact-checking articles is a time-consuming process, and fact-checkers can’t keep up with the flood of false information that’s posted online, says Dhruv Sahnan, a doctoral student in Natural Language Processing at MBZUAI.

Researchers have explored how AI can help with aspects of fact-checking by automating tasks like retrieving evidence and verifying claims. But no one had developed an AI system designed specifically to write fact-checking articles, until now.

Sahnan is co-author of a new study that proposes an agentic AI framework called QRAFT that is designed to write fact-checking articles by mimicking the workflow used by human fact-checkers. “We want to help fact-checkers by developing AI tools that can help them do their work faster,” Sahnan says.

A study about the framework was presented at the 64th Annual Meeting of the Association for Computational Linguistics (ACL) in San Diego, California. David Corney, Irene Larraz, Giovanni Zagni, Ruben Miguez, Zhuohan Xie, Iryna Gurevych, Elizabeth Churchill, Tanmoy Chakraborty, and Preslav Nakov are co-authors of the study.

Learning from experts

To design QRAFT, the researchers started by interviewing those who are most knowledgeable about writing fact-checking articles — human experts from leading fact-checking organizations. “We wanted to really understand what fact-checking articles are and how they are drafted,” Sahnan says. “Too often, engineers don’t talk with experts before they build solutions.”

According to the experts, effective articles must accurately explain the claim being made, describe the origin of the claim, explain how the evidence is relevant and helps in the verification of the claim, be well structured and written in a clear writing style, and explain why fact-checking the claim is important.

 QRAFT is composed of three agents: a planner agent that extracts key facts relevant to the claim from the evidence and drafts an outline, a writer agent that composes the draft, and an editor agent that reviews the draft and collaborates with the writer agent to refine it.

In the editing phase, QRAFT simulates the editorial process by employing conversational question-answering interactions between the writer and editor agents, creating a list of edits and refining the draft to ensure high standards of writing.

The researchers provided the preferences of the human fact-checkers they interviewed in the form of an instruction prompt to guide QRAFT as it generated the fact-checking articles.

Testing QRAFT

The researchers evaluated QRAFT in two ways: through automatic metrics that are commonly used to assess the quality of machine-generated text and through expert evaluations by having professional fact-checkers manually rate the quality of the generated fact-checking articles.

On the automatic assessments, QRAFT performed better than other approaches, including a agentic pipeline for long-form text generation called Storm and GPT-4o-mini prompted to write an article in one pass, which the researchers call Vanilla-GPT.

QRAFT scored highest on measures of factual accuracy and produced the fewest fabricated citations, a problem that fact-checkers identified in interviews as one reason they are reluctant to use AI. QRAFT also made fuller use of source material, citing more than 90% of the available evidence in its articles, compared to about 30% for Vanilla-GPT.

The results of the human evaluation were quite different. The researchers had the three AI systems (QRAFT, Storm, and Vanilla-GPT) generate articles for 12 claims made on the topic of climate change. They gave the expert reviewers the articles written by the three AI systems along with articles on the same topics written by professional fact-checkers. The reviewers weren’t told how each article was drafted. The experts rated the articles and ranked them by how close they were to being publishable.

The reviewers judged the expert-written articles as the best. QRAFT ranked second, but the fact-checkers said its articles would still require substantial editing before publication. As the researchers write in the study, the human evaluations reveal key limitations of using LLM-based frameworks for fact-checking article generation, “indicating the need for constant expert supervision.”

The fact-checkers pointed to specific weaknesses of the AI systems. In some cases, the articles written by QRAFT and Storm would include “extraneous details” that weren’t relevant to why a claim might or might not be true. In other cases, QRAFT presented relevant facts but didn’t explain how they related to a claim’s veracity.

What explains the difference between the automatic and human assessments? Sahnan says that it’s possible that existing metrics for automatic evaluation of long-form text don’t provide enough signals about how the text would be used in the real world. The metrics might also not account for the preferences and writing styles of human fact-checkers. “Automatic evaluations just see if the information in an article matches what a ground truth article says, but human evaluations might be looking at other things,” he says.

The experts also explained that an article fully written by AI is less helpful than it might seem because a fact-checker would still need to verify every argument in it against the cited sources. Reviewing a draft written by AI would take nearly as long as writing one from scratch. But the experts did say that the AI-composed drafts provided useful ideas about structure.

Building more collaborative AI

Sahnan says that even if QRAFT wasn’t able to generate fact-checking articles that met the bar set by human fact-checkers, the project illustrated the complexity of using LLMs to generate long-form text.

It also showed the importance of learning from experts about what they need and want in AI tools. “As AI engineers, we feel that what we build is useful to people, but that’s not always the case,” Sahnan says. “We shouldn’t assume what experts need.”

He also says that human-AI collaboration is an important next step for these technologies. Experts want more control over the tools they use, and if AI is to provide meaningful benefits to users, AI systems should be interpretable and provide rationales for the decisions they make.