Making LLM accuracy a matter of fact

Natural language processing (NLP) master’s graduate, Hasan Iqbal, is committed to facts. So much so that he developed a fact-checking framework that has become the basis of his academic passion.

When Hasan Iqbal arrived at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) in 2023, he planned to take his learning to the next level by studying computer vision, one of the University’s core fields of research. Equipped with a bachelor’s degree in electrical engineering from UET Lahore and experience at a software development company in his native Pakistan, he saw it as the natural next step.

However, his belief in the sanctity of facts led him, instead, into natural language processing (NLP), conducting research on fact-checking and factual evaluation of large language models (LLM) under the supervision of Professor Preslav Nakov — Department Chair of NLP.

“We cannot make right answers wrong, or wrong answers right,” explains NLP master’s graduate Iqbal – one of 25 students to graduate from the department as part of the Class of 2025. “They are what they are, and that objectivity appeals to me. Other NLP fields can be quite subjective, such as summarization or machine-generated text detection, but in fact-checking the facts are facts.

“This is what drove me towards fact-checking in NLP at MBZUAI. I started collaborating with others, such as Dr. Yuxia Wang, who was leading research on developing frameworks for fact-checking, and I eventually led the research for the development of OpenFactCheck, which has become very well-known at the University.”

OpenFactCheck is a unified framework composed of three modules — ResponseEvaluator, LLMEvaluator, and CheckerEvaluator — and is the basis of his thesis, ‘A Unified Agentic Framework for Automated Fact-Checking and Factual Evaluation of Large Language Models’.

“We want to analyze what the LLMs are saying to us,” he says. “We want to check whether the facts are correct or not; whether the LLM is hallucinating or not. And there are three main modules to help us do that.

“The first is ResponseEvaluator, which is an automatic evaluation system that users can customize to verify claims. LLMEvaluator assesses how factual an LLM is across several benchmark datasets. And CheckerEvaluator assesses how good the fact-checkers we’re using are.

“It’s an ambitious architecture, and it was published at the Empirical Methods in Natural Language Processing (EMNLP) conference in 2024.”

Taking factuality further

It wasn’t only EMNLP that published Iqbal’s work with OpenFactCheck. While that particular conference published two of his papers, both the North American Chapter of the Association for Computational Linguistics (NAACL) 2025, and the International Conference on Computational Linguistics (COLING) 2025 also accepted his research..
Iqbal also enjoyed a more active role in COLING, which took place in Abu Dhabi—the first time it has been held in the Middle East—with MBZUAI as local organizers of the conference.

“I was a media chair at COLING, which was really exciting,” he said. “To have this conference in Abu Dhabi was really special. Abu Dhabi is doing so much in the field of AI and I am very happy to be in the middle of it all.”

Having already received an offer to pursue a Ph.D. at MBZUAI, Iqbal is poised to take his research on fact-checking to the next level. “I’m excited to continue my work with Professor Nakov and try to improve the field of fact-checking as much as I can,” he says. “We can look at resolving areas such as cost efficiency and the low quality of evidence from parts of the world where the internet is not so commonly used, or the language is low-resource.

He will also be exploring different types of fact-checking. “Temporal fact-checking is about how facts change over time; spatial is where facts can change across regions; and memory-based is about creating generic memory structures to enhance our fact-checking so that we can provide better retrieved augmented data from multiple sources.

“I’m also already working on multilingual fact-checking in Urdu, and then Chinese, Arabic and other low-resource languages.”

Collaborative atmosphere

Iqbal believes that the success of his MBZUAI experience so far is due to an ethos that he had not seen in other universities: that of partnership and teamwork.

“The first thing I liked when I came here was the atmosphere of collaboration,” he explains. “We had regular brainstorming sessions where the supervisors brought students together to discuss everything, and that really helped us to understand the bigger picture and see new perspectives.

“Added to that, it was great to work with such talented professors, who were always on hand to help. Professor Nakov is such a significant figure in NLP, so it was exciting to learn from him.”

Away from the academic rigours of his degree, Iqbal says he made some good friends at the University, and that, together, they found ways to unwind.

“I love to play board games with my friends, and also some co-op games with my wife, as well as watching anime or movies,” he says. “The work here can be very intense, so it’s important to have ways to relax.”

But despite understanding the importance of downtime, he admits that he’s always got one eye on a long-term and long-held aspiration.

“I actually want to be a professor in the field of NLP someday — that is my dream,” he asserts. “I want to teach and never miss an opportunity to impart my knowledge to somebody. It’s a long path, but I’ll do my best to get there.”