Initiated by the Doubao (Seed) Team, the Doubao Large Model Fund opened for applications in June 2024, focusing on cutting-edge research topics in the field of AI large models. Open to universities and scholars worldwide, the Fund also marks ByteDance's first-ever research fund in the field of large models.

Thanks to its high-quality topics, the Doubao Large Model Fund has received over 200 applications from 63 universities around the world to date, including Tsinghua University, Peking University, Nanyang Technological University, and the University of Sydney.

The review panel assessed applicants on their research capabilities, academic innovation and feasibility, as well as the potential impact of their work, ultimately selecting 23 scholars for collaboration.

List of Selected Scholars

*Sorted alphabetically

Yanwei Fu – Fudan University

Selected Topic: Native Video-Text Generation for Large Models
Academic Profile: Yanwei Fu is a Professor at the School of Data Science, Fudan University, a distinguished professor of Shanghai universities (i.e. Oriental Scholar), and the winner of the IEEE ICME Outstanding Paper Award 2019. He has published 150 papers in top-tier international journals and conferences in the field of computer science, such as IEEE TPAMI, IEEE TIP, CVPR, ECCV, and ICCV, with 15 of them as first or corresponding author in TPAMI. He has been granted over 20 Chinese patents and 9 U.S. patents.

Di He – Peking University

Selected Topic: Efficient Structure Design and Optimization for Large Models
Academic Profile: Di He is an Assistant Professor at Peking University, formerly a senior researcher in the Machine Learning Group at Microsoft Research Asia, with research interests in generative models and Al for Science. His team's graph neural network, Graphormer, won the championship in related challenges. He has served as an academic expert and reviewer for top conferences and has published several papers at leading machine learning conferences. He received the ICLR 2023 Outstanding Paper Award and was nominated for the ICLR 2024 Outstanding Paper Award.

Weiran Huang – Shanghai Jiao Tong University

Selected Topic: Drawing Code Generation Based on Visual-Language Models
Academic Profile: Weiran Huang is an Associate Professor and PhD supervisor at the Qingyuan Research Institute, Shanghai Jiao Tong University, with research interests in machine learning, covering both the theories and applications of algorithms. He has published dozens of papers in top international conferences, including the three top machine learning conferences (ICML, NeurIPS, and ICLR), as well as the major computer vision conferences (CVPR, ICCV, and ECCV). His representative work was selected as a candidate for the ICCV Outstanding Paper Award 2023.

Lingpeng Kong – University of Hong Kong

Selected Topic: The Enhancement of the Effective Length of Optimized Long Context Models
Academic Profile: Lingpeng Kong is an Assistant Professor in the Department of Computer Science at the University of Hong Kong. He earned his PhD from the School of Computer Science at Carnegie Mellon University in 2017. He worked as a research scientist at Google DeepMind before joining the University of Hong Kong. His research focuses on natural language processing and machine learning, including representation learning, structure prediction and generative models. He has been the area chair of machine learning and natural language processing conferences (ICLR, NeurIPS, ICML, etc.) for many years.

Qiuqiang Kong – The Chinese University of Hong Kong

Selected Topic: Semantics-based Audio Encoding and the Implementation of the Universal Audio GPT-4o System
Academic Profile: Qiuqiang Kong is an Assistant Professor in the Department of Electronic Engineering at the Chinese University of Hong Kong, with research interests in audio signal processing. His representative works include weakly labeled audio event detection and separation, large-scale audio classification systems, and the construction of large-scale music datasets, where he collected and transcribed the world's largest piano dataset, GiantMIDI-Piano. He has published over 50 papers in leading journals and conferences, and received the IEEE SPS Young Author Best Paper Award 2022 and the ISMIR Best Paper Award 2022.

Xiaodan Liang – Sun Yat-sen University

Selected Topic: Multi-Role Identity-Preserving Video-to-Text Generation with Large Models
Academic Profile: Xiaodan Liang is an Associate Professor at the School of Intelligent Systems Engineering, Sun Yat-sen University, and a co-supervisor at the university's HCP-12 Laboratory. She is a recipient of the National Thousand Talents Program for Top Young Talents, a visiting professor at MBZUAI, and a Senior Member of IEEE. She has published over 80 papers in leading journals such as IEEE T-PAMI and at top conferences including CVPR, NeurIPS, and ICML, with more than 26,000 citations on Google Scholar. She has won the First Prize of the China Society of Image and Graphics Science and Technology, the Shi Qingyun Female Scientist Award, and the Alibaba DAMO Academy Young Fellow Award.

Jian Li – Tsinghua University

Selected Topic: In-Depth Research on the Interpretability of Large Model Data
Academic Profile: Jian Li is a Tenured Associate Professor at the Institute for Interdisciplinary Information Sciences, Tsinghua University. His research interests mainly include algorithm design and analysis, machine learning, and database, etc. He has published more than 100 papers in leading international conferences and journals, and won the Best Paper Award of VLDB 2009 and ESA 2010, the ICDT 2017 Best Newcomer Award. He has also been recognized through the Tsinghua 221 Basic Research Talent Support Program, the Ministry of Education's New Century Talent Support Program, and the National Natural Science Foundation's Outstanding Young Scientists Fund. He has led and participated in various scientific research projects, including the General Program of National Natural Science Foundation of China, China-Israel International Cooperation Projects, and the Youth 973 Program.

Zongqing Lu – Peking University

Selected Topic: Self-Reward Mechanisms in Visual-Language Models

Academic Profile: Zongqing Lu is a Tenured Associate Professor at the School of Computer Science, Peking University. He is one of the National Young Talents, and he is a BAAI Scholar and the director of the Multimodal Interaction Research Center at the Beijing Academy of Artificial Intelligence. He mainly studies reinforcement learning, multimodal large models, and general intelligent agents. He also serves as the area chair of ICML, NeurIPS, and ICLR.

Qian Yanmin – Shanghai Jiao Tong University

Selected Topic: Efficient Construction of Large Models for Audio Generation under a Novel Cross-Modal Unified Framework

Academic Profile: Yanmin Qian is a Distinguished Professor in the Department of Computer Science and Engineering at Shanghai Jiao Tong University. He is a Changjiang Scholar under the Ministry of Education, a recipient of the National Outstanding Youth Fund, and The Prize of First Class for the Wu Wenjun Science and Technology Award (as the lead contributor). He is currently a senior member of IEEE, a member of ISCA, and also one of the 13 founding members of the international open-source project Kaldi speech recognition toolkit. He serves as the chair and area chair for the international conference program committee, and a reviewer for multiple journals and conferences. He has published over 300 academic papers, with more than 15,000 citations on Google Scholar, filed over 80 Chinese and U.S. patents, and received 3 Best Paper Awards. Under his leadership, his team has won 6 international competition championships.

Baixin Shi – Peking University

Selected Topic: World Knowledge-Driven Algorithms for Decoupled Editing of Image Attributes

Academic Profile: Baixin Shi is the Deputy Director of the Institute of Video and Visual Technology and a Tenured Associate Professor at the School of Computer Science, Peking University. He is also a BAAI Scholar. His research focuses on computational photography and computer vision, and he has published over 200 papers. He has received the CVPR 2024 Best Paper Nomination, and the Okawa Research Grant Award in 2021. He is the chief scientist of major projects of Ministry of Science and Technology, and has been selected in the National Young Talents Program. He is a member of the TPAMI/IJCV Editorial Board and the area chair of the three major CV conferences. He is a senior member of IEEE, CCF, and CSIG, and a distinguished lecturer of APSIPA.

Jingwei Sun – University of Science and Technology of China**

Selected Topic: Modeling and Optimization of MoE LLM Inference Performance

Academic Profile: Jingwei Sun is a Special Associate Researcher at the School of Computer Science and Technology, University of Science and Technology of China. He earned his PhD from the university in 2020. His research focuses on high-performance computing, parallel program performance modeling, prediction and tuning, as well as efficient deep learning inference. He has published 16 papers in high-performance computing journals and conferences such as TC, TPDS, TACO, and IPDPS, and holds 4 authorized patents.

Bing Su – Renmin University of China

Selected Topic: Pretraining Methods for Generative Visual Models

Academic Profile: Bing Su is a Tenured Associate Professor at the Gaoling School of Artificial Intelligence, Renmin University of China, and a recipient of the National High-Level Young Talents Program. His research focuses on computer vision, with over 30 papers published as first or corresponding author in CCFA journals and conferences. He has been selected in multiple programs and led projects funded by the National Natural Science Foundation of China. He serves as an associate editor for journals and as an area chair of CCF-A conferences.

Jianyong Wang – Tsinghua University

Selected Topic: Novel and Efficient Neural Network Architectures for Generative Models

Academic Profile: Jianyong Wang is a Tenured Professor in the Department of Computer Science at Tsinghua University. He mainly studies machine learning and medical data mining. He has served as BAAI Scholar of Beijing Academy of Artificial Intelligence and was the inaugural Vice Chair of the Academic Committee of the Jiangsu Key Laboratory of Big Data Security and Intelligent Processing. He is a two-time recipient of the HP Labs Innovation Research Award. He was selected for the Ministry of Education's "New Century Excellent Talents" Support Program in 2007 and the Okawa Research Grant in 2009. In 2017, he was named an IEEE Fellow and a founding Fellow of CAAI.

Limin Wang – Nanjing University

Selected Topic: Enhancing the Generalization and World Knowledge Reconstruction Capabilities of Generative Models

Academic Profile: Wang Jianyong is a Professor at the School of Computer Science at Nanjing University and a researcher at the State Key Laboratory for Novel Software Technology. He has published many papers in top conferences and journals with over 25,000 citations on Google Scholar. He serves as a member of the Editorial Board of the IJCV. He won the ActivityNet Challenge Video Recognition Track Championship 2016 and many more.

Wenguan Wang – Zhejiang University

Selected Topic: Multi-Modal Large Model Based on Dual Learning and Knowledge Distillation

Academic Profile: Wenguan Wang is a Researcher under the Hundred Talents Program at the School of Computer Science, Zhejiang University, and a recipient of the National Excellent Youth Fund (Overseas). His research interests include artificial intelligence and computer vision. He has published more than 80 papers in top journals and conferences, with more than 18,000 citations on Google Scholar and an H-index of 72. He was awarded the "World's Highly Cited Researcher" by Clarivate Analytic and the ACM China Rising Star Award.

Xiangdong Wang – Institute of Computing Technology, Chinese Academy of Sciences

Selected Topic: Fine-Grained Understanding Capability Enhancement for Audio Multimodal Large Models

Academic Profile: Xiangdong Wang is a Doctor, a Senior Engineer and Master's Supervisor at the Institute of Computing Technology, Chinese Academy of Sciences. He leads the Natural Human-Computer Interaction Research Group at the Research Center for Ubiquitous Computing Systems. His research interests include human-machine interaction, machine learning, voice interaction, audio processing, etc. He has published 60 papers in AAAI, TASLP, ICASSP, COLING, EMNLP, NAACL, INTERSPEECH and other famous conferences and journals, and holds more than 30 authorized patents. He won the second prize of the Beijing Science and Technology Award and the first prize of Technological Invention of the China Computer Federation as a team member.

Ying Wen – Shanghai Jiao Tong University

Selected Topic: Self-Improvement Strategies for Large Models Based on Reinforcement Feedback

Academic Profile: Ying Wen is a Tenured Associate Professor at Shanghai Jiao Tong University. His research focuses on multi-agent learning, reinforcement learning, and the application of game theory. He obtained his PhD and research master's degrees from the Department of Computer Science, University College London in 2020 and 2016, respectively. He has published more than 40 research results in the top international conferences in related fields such as ICML, NeurlPS, ICLR, IJCAI, AAMAS. He won the CoRL 2020 Best System Paper Award and the AAMAS2021 Blue Sky Track Best Paper Award.

Chenjun Xiao – The Chinese University of Hong Kong, Shenzhen

Selected Topic: A Large Model Training System Based on Reinforcement Learning and Planning Search Algorithms

Academic Profile: Chenjun Xiao is an Assistant Professor at the School of Data Science, The Chinese University of Hong Kong, Shenzhen. His major research interests include RLHF, offline reinforcement learning, strategy optimization theory, and Monte Carlo Tree Search (MCTS) algorithms. He serves as a research scientist at Noah's Ark Lab. He has published more than 20 papers in top AI conferences, and many of his work have been selected for the Oral/Spotlight sessions at the three major machine learning conferences (ICML, NeurIPS, and ICLR), among which, his work on MCTS won the AAAI-18 Outstanding Paper Award.

Zeke Xie – The Hong Kong University of Science and Technology (Guangzhou)

Selected Topic: Optimization Strategies and Generalization Metrics for Large Models

Academic Profile: Zeke Xie is an Assistant Professor in the Data Science and Analytics Thrust and AI Thrust at the Hong Kong University of Science and Technology (Guangzhou). His research interests include machine learning, deep learning theory, and generative AI, etc. He holds a Bachelor of Science degree from the University of Science and Technology of China and a Master's and PhD from the University of Tokyo. He has published papers in top conferences and journals, including ICLR, ICML, NeurIPS, and ICCV, with more than four hundred citations on Google Scholar and an H-Index of 10.

Yujiu Yang – Tsinghua University

Selected Topic: Exploring Complex Reasoning and Deep Analytical Capabilities in Large Models (System 2)

Academic Profile: Yujiu Yang is the Director of the Intelligent Computing Lab at Tsinghua University. His research interests include pattern recognition, machine learning, natural language processing, visual image understanding and analysis. He has led over 50 scientific research projects, contributed to the development of 8 national standards and 3 industry standards, and holds more than 10 granted Chinese invention patents. He has published over 150 papers and received multiple awards for scientific and technological advancements, as well as artificial intelligence science and technology honors.

Yue Xiangyu – The Chinese University of Hong Kong

Selected Topic: Native Alignment and Instruction Fine-Tuning Technologies for Multimodal Large Models; Structural Design and Efficient Training Algorithms for Visual Understanding and Generation of Unified Large Models

Academic Profile: Xiangyu Yue is an Assistant Professor and PhD supervisor at MMLab at The Chinese University of Hong Kong. His work focuses on artificial intelligence and computer vision, and has published more than 40 papers in international authoritative journals and conferences, with more than 8,000 citations on Google Scholar and an H-Index of 27. He has received the Lotfi A.Zadeh Award, and has served as an area chair for top computer conferences such as CVPR 2024 and NeurIPS 2024.

Zongzhang Zhang – Nanjing University

Selected Topic: Compute-Efficient Alignment Methods for Reinforcement Learning

Academic Profile: Zhang Zongzhang is an Associate Professor and PhD supervisor at the School of Artificial Intelligence, Nanjing University. His primary research area is reinforcement learning. He has published over 60 papers in CCF-recommended Class A and B Chinese and English journals and international conferences. He holds 16 Chinese invention patents and 2 U.S. invention patents, with 12 licensed or transferred to enterprises. His research outcomes have been applied in business scenarios such as autonomous driving, warehouse logistics, network security, and gaming. He serves as an editorial board member for the journal Intelligent Computing, a junior editorial board member of the journal Frontiers of Computer Science, the area chair for NeurIPS, and a senior program committee member for AAAI, IJCAI, AAMAS, ICAPS and ECA conferences.

Xin Zhao – Renmin University of China

Selected Topic: Data-Efficient Training and Complex Reasoning Abilities

Academic Profile: Xin Zhao is a Professor at Gaoling School of Artificial Intelligence, Renmin University of China. His research interests include information retrieval and natural language processing. He has published over 200 papers, with more than 20,000 citations on Google Scholar. He has led the research and development of the YuLan Large Language Model, and organized the creation of the comprehensive large language model review paper A Survey of Large Language Models (preprint) as well as the Chinese book Large Language Models. He has received the Wu Wenjun AI Excellent Young Scientist Award and CCF-IEEE CS Young Scientist Award.

Starting from August 2024, ByteDance will successively finalize agreements with the selected scholars mentioned above and initiate the fund disbursement process.

About the Doubao Large Model Fund

The Doubao Large Model Fund has launched 18 cutting-edge research interests and 65 reference topics, spanning areas such as large language models, multimodal understanding and generation, machine learning algorithms and systems. Since its announcement, the initiative has received significant attention from scholars worldwide.

In recent years, ByteDance has maintained continuous investment in academic research and university-enterprise collaboration, achieving several high-quality outcomes.

The long-term investment in technology stems from the team's deep understanding of knowledge innovation and technological progress. Especially when the large model era comes, the team aims to provide sufficient and stable computing resources, financial support, and opportunities to explore and address the critical challenges of large models for academic research.

The Doubao Large Model Fund will continue to contribute to the flourishing of academic research, driving advancements in large models and related technological fields.