FinVolution Global Data Science Competition Showcases LLMs in Voice Deepfake Detection

• Voice data augmentation and large language model (LLM) recognition impress judges

• Final datasets included the latest LLM-synthesized voices, and re-recorded fake voices, making recognition more challenging.

• Competition highlights the increased risks of deepfakes with new technologies

FinVolution Group, a leading fintech service provider, concluded the 9th FinVolution Global Data Science Competition on July 23. As part of the IJCAI 2024 challenge track, a top international AI conference, this year's event spotlighted the use of large language models (LLMs) for effective voice deepfake detection.

The rapid advancement of deepfake technology blurs the lines between AI-generated and real voices, raising concerns about personal privacy and financial security. Participants of the contest employed deep learning and adversarial AI techniques to develop models capable of detecting fake voices from the provided datasets. The finals featured various algorithmic models and training approaches, including LLMs and traditional end-to-end recognition technologies.

The winning team achieved an impressive fake voice recognition rate of over 99% in the preliminary round and nearly 80% in the semifinals. Qiang Lyu, FinVolution's algorithm scientist, attributed the disparity to varying complexities in the datasets.

"The preliminary round's datasets primarily consisted of cloned voices generated by end-to-end TTS (text-to-speech) systems, which were easier to identify," Lyu explained. "In the semifinals, however, the datasets included the latest LLM-synthesized voices, re-recorded fake voices, and even mixed real and fake voices in over five languages, including English, French, and Spanish. This complexity made recognition more challenging."

Addressing LLM-generated deepfakes with LLMs

Judges were impressed by the innovative solutions and practical business implications presented by contestants. Many used voice data augmentation to expand their datasets and employed the latest LLM recognition technologies to enhance inference efficiency and reduce processing time.

Lei Chen, Vice President of FinVolution Group and Head of its Big Data and AI Division, commented, " The competition showcased a range of effective voice deepfake detection solutions. Some contestants even used LLMs to detect deepfakes generated by other LLMs, demonstrating promising results. These approaches have significant potential for the fintech sector with further development."

Chen added, "Research on voice LLMs needs to address academic challenges and then translate these solutions into practical applications to meet real-world business needs."

Yuxiang Wang, COO & CTO of FinVolution Group, concluded, "As deepfake generation technology evolves, detection method must keep pace to achieve a balance between offense and defense. We hope that the FinVolution Global Data Science Competition will continue to push the boundaries of technology, empowering talent, academia, and industry, and fostering new innovations."

The IJCAI 2024 Challenge

Launched in May, the 9th FinVolution Global Data Science Competition attracted 461 teams and 709 participants from leading universities and companies worldwide.

The nine finalist teams presented their solutions during on-site defenses, showcasing the diverse potential applications of deepfake detection technologies. Judges from top institutions, including Zhejiang University, Shanghai Jiao Tong University, and Renmin University of China, evaluated the presentations.

As part of the IJCAI (International Joint Conference on Artificial Intelligence) Competitions and Challenges track, a top international AI conference, the winners will attend and present their solutions at IJCAI 2024 in Jeju, South Korea. FinVolution is a proud sponsor of IJCAI 2024.

Since its inception, the FinVolution Global Data Science Competition has drawn nearly 10,000 researchers from top university and enterprise globally. Previous themes have included "dialect distance recognition in speech," "smart retail cabinet product recognition," and "credit services for small and micro businesses."