MBZUAI teams shine in competition

Two teams from MBZUAI took home top honors from the IEEE SLT international hackathon, held in Qatar in January 2023. The teams, consisting of MBZUAI master’s students and research assistants, competed against 18 other teams from more than 30 countries.

The online hackathon took place during two months, followed by two days of intensive onsite hacking at the Qatar Computing Research Institute. Participants from many top universities took part, including EPFL, University of Notre Dame, Technische Universität Braunschweig, IISc, IIT Madras, and MBZUAI, among many others.

Projects entered into the competition ranged from Arabic spoken dialect identification to code-switching speech recognition for African languages, low-resource data generation in Indonesian languages, and multilingual speech recognition for Indian languages.

Autodub

The team of MBZUAI master’s students Gokul Karthik Kumar (computer vision) and Bokang Jia (machine learning), with teammate Aakash Sasikumar from the University of Alberta and mentored by Yifan Peng from Carnegie Mellon University, won the Best Potential Impact Project award for Autodub.

“We’re grateful for the opportunity to collaborate with leading researchers and learn about their exciting work during this hackathon,” Kumar said. “The knowledge gained will help improve the Autodub platform and drive further innovation. We hope that our achievement will motivate colleagues to participate in more international events and continue to make a positive impact in the field of AI.”

Autodub is a human-in-the-loop AI dubbing platform that integrates transcription, translation, voiceover, and background audio separation. The goal is to create a world free of language barriers, particularly for video content.

As a platform it’s designed to allow users to import video, generate subtitles using AI transcription, select a target language, and translate. The platform then replaces the source speech with speed-adjusted AI speech in the target language and extracts and mixes the background music from the source video.

The platform also features practical human-in-the-loop UI design, model agnostic, extensible to new languages, error cascading, synchronization without training time-constrained machine translation, improvement through user interaction, and more.

Commentator Voice Synthesizer

The second MBZUAI team, led by Research Assistant Massa Baali, won the Craziest Idea Award for Commentator Voice Synthesizer. The goal of the synthesizer is to enhance the engagement of video games by creating a speech generator for a football commentator.

“It’s important to create expressive text-to-speech and it’s a unique challenge to make the voices sound natural,” Baali said. “The mentorship aspects of the event — the chance to work shoulder-to-shoulder with leading professors in speech during the event — made the hackathon experience quite productive and fun.”

The project involved a pipeline of data collection, transcription, segmentation, and expressive text-to-speech synthesis. Baali and her team’s synthesizer performed so well, in fact, that during demonstration the audience found it challenging to distinguish between the synthesizer and a real, live commentator.

The IEEE SLT 2022 is the first flagship speech conference to have visited the Middle East. The SLT-CODE hackathon, organized as part of the conference, was open to anyone interested in speech and language technology, including researchers, programmers, user experience designers, practitioners, and enthusiasts. The goal of SLT-CODE was to build a community of people who want to explore and improve speech technology, particularly for underrepresented languages and regions.

“The success we’ve had as students across hackathons in energy, transport, space, agriculture, and more highlights the university’s commitment to promoting student innovation and excellence in AI,” Kumar said.

Pioneers 4.0 win for students

A team of MBZUAI students also took home runner-up honors in March of 2023 for their submission at the Pioneers 4.0 Hackathon Series.

The series, which was organized by the UAE Ministry of Industry and Advanced Technology in collaboration with EDGE, aims to foster innovation and co-creation to solve real-world challenges with the latest technologies within an immersive and inspiring environment.

MBZUAI student team consisted of (second from left) Faris Almalik M.Sc. in machine learning, Osama Mohammed Afzal M.Sc. natural language processing, Younes Balah M.Sc. in machine learning, Mugariya Farooq M.Sc. in machine learning, and Adnan Khan M.Sc. computer vision.

The MBZUAI student team adopted a structured approach which involved cleaning and visualizing data, analyzing correlation factors, and suggesting potential improvements. The team provided an innovative solution utilizing an AI technology.