Safety of Deploying NLP Models: Uncertainty Quantification of Generative LLMs
Monday, May 06, 2024
When deploying a machine learning (ML) model in practice, care should be taken to look beyond prediction performance metrics such as accuracy or F1. We should ensure also that it safe to use ML-based applications. This entails that applications should be evaluated along other critical dimensions such as reliability and fairness. The widespread deployment of large language models (LLMs) has made ML-based applications even more vulnerable to risks of causing various forms of harm to users. While streamline research effort has been devoted to the “alignment” via various forms of fine-tuning and to fact checking of the generated output, in this talk, we focus on uncertainty quantification as an effective approach to another important problem of LLMs. Models often “hallucinate”, i.e., fabricate facts without providing users an apparent means to discern the veracity of their statements. Uncertainty quantification (UQ) methods could be used to detect unreliable generations unlocking the safer and more responsible use of LLMs in practice. UQ methods for generative LLMs are a subject of bleeding-edge research, which is currently quite scarce and scattered. We systemize these efforts, discuss common caveats, and provide suggestions for the development of novel techniques in this area.
Post Talk Link: Click Here
Passcode: #^sa2=6e
Speaker/s
Dr. Artem Shelmanov currently holds a position of a Senior Research Scientist at MBZUAI, in the Natural Language Processing Department. He earned his PhD in 2015 from FRC CSC RAS with a thesis on semantic-syntactic parsing for information retrieval. After his graduation, he secured several research grants and continued his work on methods for semantic parsing and semantic relation extraction as a Senior Research Scientist. His research findings were implemented in various information retrieval engines for scientific literature and patent search. In 2018, Dr. Shelmanov joined SAS Institute as a Team Leader of a text analytics group in the Commonwealth of Independent States (CIS), where he helped developing business-oriented NLP solutions. From 2019 to 2021, while working as a PostDoc at Skoltech, he contributed to establishing a joint industrial laboratory with the largest telecommunication company in the CIS. In this role, he guided research on active learning methods for information extraction tasks. He also implemented several projects related to medical text processing with the Philips Research Center. In 2021, he helped to establish AIRI, the leading non-profit AI research institution in the Commonwealth of Independent States (CIS). Untill the end of 2022, he served as the head of a research group focusing on active learning and uncertainty quantification for NLP models. His work has been recognized with a Special Gratitude award for outstanding achievements. His research contributions have been published in well-regarded international conferences and journals such as ACL, EMNLP, NAACL, EACL, AACL, NeurIPS, Nature Scientific Reports, ECIR, and LREC. He is one of the authors of a work that was recently recognized with the Best Resource Paper Award at EACL-2024. His current research interests include uncertainty quantification for generative and non-generative LLMs, debiasing techniques to promote fairness in LLMs, memory-augmented models, and methods for machine generated text detection.
Related
Polygenic Score Modeling to Investigate Genotype-Phenotype Associations
Read More
Trustworthy Machine Learning: Transparency, Collaboration, and Evaluation
Read More