The International Conference on Learning Representations (ICLR), recognized as the premier academic campaign in the field of deep learning, will be held in Singapore on April 24-28, 2025. During this period, the first Open Science for Foundation Models (SCI-FM) Workshop will be held concurrently at the conference venue. The workshop is now accepting submissions from the industry.

Nowadays, foundation models have shown significant value in areas such as natural language processing, computer vision, speech recognition, and multimodal understanding. However, high-performance foundation models often have their key technical details hidden, such as training data and architecture design, limiting their further development.

In this context, the first Open Science for Foundation Models (SCI-FM) Workshop will be held during the ICLR 2025. The workshop aims to promote open-source, sharing, and reproducibility of foundation models. We invited scholars related to FMs to participate in the workshop, with the aim of empowering the development and prosperity of FMs.

The Open Science for Foundation Models (SCI-FM) Workshop is now open for paper submissions, and researchers in the field are welcome to submit their work.

All accepted papers will be presented as posters. In addition, we will select approximately 3 papers for brief oral presentations and 2 papers for outstanding paper awards.

1. Paper Requirements

The Workshop encourages authors to provide as much open source material as possible in their paper, including but not limited to datasets, models, and training processes. Submissions should fall into two categories: short papers and regular papers, with page limits of 6 pages and 9 pages, respectively. References and appendix should be included at the end of the paper.

Topics may include, but are not limited to the following:

• Open Datasets: Acquisition, curation, and synthesis of pretraining, instruction, and preference datasets through manual or algorithmic methods.

• Open Foundation Models: Pretraining strategies including data scaling, model architecture, multi-modal and multi-task pretraining. Learning algorithms such as meta-learning, model fusion, model merging, and continual learning designed for open, scalable models.

• Open Training Protocols: Training dynamics research on phenomena such as scaling laws, interpretability, complexity analysis, and emergent capabilities. Alignment techniques including prompt tuning, prefix tuning, instruction tuning, and reinforcement learning with human AI feedback.

• Open Evaluation: Establishment of open evaluation benchmarks, evaluation protocols, and metrics.

• Open Compute Efficiency Techniques: Focus on model distillation, compression, quantization, and optimization of attention or memory mechanisms for improved compute efficiency in foundation models.

• Open Multi-Modal Foundation Models: Expansion to multi-modalities such as vision and audio.

• Open Interactive and Agent Systems: Development of conversational AI, interactive learning models and multi-agent systems.

• Open Replication of Proprietary Systems: Efforts to replicate foundation models and systems that were previously proprietary, ensuring transparency and reproducibility for broader research and development.

2. Submission Site and Important Dates

Submission Site: https://openreview.net/group?id=ICLR.cc/2025/Workshop/SCI-FM

Submission Open Date: January 6, 2025

Submission Deadline: February 10, 2025

Notifications of Acceptance/Rejection: March 5, 2025

The workshop also invites scholars from relevant fields to participate in the reviewing process. Please fill out the form at the link to register: https://forms.office.com/e/SdYw5U75U3?embed=true

3. Some Invited Speakers

Stella Biderman

• Currently working at EleutherAl
  

• Core contributor to several open source efforts (Bloom, Pythia, GPT-NeoX-20B, The Pile, etc.)

  
  

Zhengzhong Liu

• Currently working at Petuum
  
• One of the core members of the full open-source LLM360 models

Luca Soldaini

• Currently working at Allen Al
  

• One of the core members of the full open-source OLMo models

  
  

Wenhao Huang

• Currently working at ByteDance
  
• Former co-founder of 01.AI
  
• One of the core members of the Yi series models

Shayne Longpre

• Ph.D. candidate at MIT
  
• Winner of ACL/NAACL Best Paper Awards
  
• Core member of several open source efforts (Bloom, Aya, Flan-T5)

4. SCI-FM Organizing Committee Member

Jiaheng Liu

• Algorithm expert in large model, Alibaba
  

• One of the co-founders of the open-source large model organization M-A-P

  
  

Riza Batista-Navarro

• Senior Lecturer at the University of Manchester
  
• Research interests include text mining, natural language processing, and other areas
  
• Winner of ACL 2024 Best Paper Award

Qian Liu

• Researcher of TikTok, core contributor to several open-source efforts (StarCoder, OctoPack, OpenCoder, Sailor, LoraHub)

Niklas Muennighoff

• Ph.D. candidate at Stanford University, winner of the NeurIPS 2023 Outstanding Paper Runner-Up Award and ACL 2024 Best Theme Paper Award

Ge Zhang

• Large model researcher of ByteDance, one of the co-founders of the open-source large model organization M-A-P
  
• Core contributor to MAP-Neo, OpenCoder, Yi, and other efforts

Yizhi Li

• Ph.D. candidate at the University of Manchester
  
• One of the co-founders of the open-source large model organization M-A-P

Xinyi Wang

• Ph.D. candidate at the University of California, Santa Barbara

Willie Neiswanger

• Assistant Professor at the University of Southern California

This campaign is sponsored by: ByteDance.

For more details and on-site arrangements, please visit the homepage of the Workshop.