I am a research scientist at NVIDIA Research. I completed my Ph.D. at the Hong Kong University of Science and Technology, advised by Professor Tong Zhang. Previously, I was a visiting scholar at BLENDER LAB@UIUC, working with my amazing host, Professor Heng Ji. I was a research intern at ByteDance AI Lab with Dr. Hang Li and Dr. Xinsong Zhang, and at Sinovation Ventures AI Institute. During my undergraduate study, I interned at WING Group @ NUS and PKU, where I was fortunate to work with Prof. Min-Yen Kan and Prof. Xiaojun Wan. I am passionate about the research in pre-training, efficient-tuning, and alignment of large foundation models.
We are hiring interns for efficient language model pre-training / post-training. Please contact me if you are interested.

News


More News

  • Three papers were accepted by EMNLP 2023 and Findings.
  • Visited BLENDER LAB@UIUC from August 2023 to January 2024.
  • Attended ICML 2023 at Hawaii.
  • One paper was accepted by ICCV 2023.
  • Two papers were accepted by ACL 2023.
  • Attended ICLR 2023 at Kigali, Rwanda.
  • Attended EMNLP 2022 at Abu Dhabi.
  • LMFlow is a framework that allows fine-tuning and deploying personalized LLMs with minimal cost and effort. It has accumulated 7, 000+ stars⭐️ on Github. We envision that LMFlow will enable more creative and diverse applications of LLMs and foster a wider community of LLM enthusiasts!
  • Check out this curated paper list about ChatGPT with the goal of helping everyone learn the techniques behind it.

Publications


  1. Rui Pan, Xiang Liu, Shizhe Diao, Renjie Pi, Jipeng Zhang, Chi Han, Tong Zhang
    LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning [NEW]
    NeurIPS 2024
  2. Tianyang Xu, Shujin Wu, Shizhe Diao, Xiaoze Liu, Xingyao Wang, Yangyi Chen, Jing Gao
    SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales [NEW]
    EMNLP 2024
  3. Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, Hanze Dong, Renjie Pi, Han Zhao, Nan Jiang, Heng Ji, Yuan Yao, Tong Zhang
    Mitigating the Alignment Tax of RLHF [NEW]
    EMNLP 2024
  4. KaShun Shum, Minrui Xu, Jianshu Zhang, Zixin Chen, Shizhe Diao, Hanze Dong, Jipeng Zhang, Muhammad Omer Raza
    FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation [NEW]
    EMNLP 2024
  5. Ruida Wang, Jipeng Zhang, Yizhen Jia, Rui Pan, Shizhe Diao, Renjie Pi, Tong Zhang
    TheoremLlama: Transforming General-Purpose LLMs into Lean4 Experts [NEW]
    EMNLP 2024
  6. Tianyang Han, Qing Lian, Rui Pan, Renjie Pi, Jipeng Zhang, Shizhe Diao, Yong Lin, Tong Zhang
    The Instinctive Bias: Spurious Images lead to Hallucination in MLLMs [NEW]
    EMNLP 2024
  7. Cheng Niu, Yang Guan, Yuanhao Wu, Juno Zhu, Juntong Song, Randy Zhong, Kaihua Zhu, Siliang Xu, Shizhe Diao, Tong Zhang.
    VeraCT Scan: Retrieval-Augmented Fake News Detection with Justifiable Reasoning
    ACL 2024 System Demonstration Track
  8. Shizhe Diao, Pengcheng Wang, Yong Lin, Tong Zhang.
    Active Prompting with Chain-of-Thought for Large Language Models
    ACL 2024
  9. Haoxiang Wang, Yong Lin, Wei Xiong, Rui Yang, Shizhe Diao, Shuang Qiu, Han Zhao, Tong Zhang
    Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
    ACL 2024
  10. Rui Pan*, Shuo Xing*, Shizhe Diao*, Wenhe Sun*, Xiang Liu, Kashun Shum, Renjie Pi, Jipeng Zhang, Tong Zhang
    Plum: Prompt Learning using Metaheuristic
    Findings of ACL 2024
  11. Shizhe Diao*, Rui Pan*, Hanze Dong*, Ka Shun Shum, Jipeng Zhang, Wei Xiong, Tong Zhang
    LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models
    NAACL 2024 Demo Track (Best Demo Paper)
  12. Hanning Zhang*, Shizhe Diao*, Yong Lin*, Yi R. Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, Tong Zhang
    R-Tuning: Teaching Large Language Models to Refuse Unknown Questions
    NAACL 2024 (Outstanding Paper)
  13. Xu Liu, Junfeng Hu, Yuan Li, Shizhe Diao, Yuxuan Liang, Bryan Hooi, Roger Zimmermann
    UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting
    WWW 2024
  14. Quyet V. Do, Tianqing Fang, Shizhe Diao, Zhaowei Wang, Yangqiu Song.
    ConstraintChecker: A Plugin for Large Language Models to Reason on Commonsense Knowledge Bases
    EACL 2024
  15. Hanze Dong*, Wei Xiong*, Deepanshu Goyal, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, Tong Zhang.
    RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
    TMLR
  16. Kashun Shum*, Shizhe Diao*, Tong Zhang.
    Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
    Findings of EMNLP 2023
  17. Renjie Pi*, Jiahui Gao*, Shizhe Diao*, Rui Pan, Hanze Dong, Jipeng Zhang, Lewei Yao, Jianhua Han, Hang Xu, Lingpeng Kong, Tong Zhang
    DetGPT: Detect What You Need via Reasoning
    EMNLP 2023
  18. Shizhe Diao*, Yongyu Lei*, Liangming Pan, Tianqing Fang, Wangchunshu Zhou, Sedrick Scott Keh, Min-Yen Kan, Tong Zhang.
    Doolittle: Benchmarks and Corpora for Academic Writing Formalization
    EMNLP 2023
  19. Zhihong Chen*, Shizhe Diao*, Benyou Wang, Guanbin Li, Xiang Wan.
    Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts
    ICCV 2023
  20. Shizhe Diao*, Tianyang Xu*, Ruijia Xu, Jiawei Wang, Tong Zhang.
    Mixture-of-Domain-Adapters: Decoupling and Injecting Domain Knowledge to Pre-trained Language Models' Memories
    ACL 2023
  21. Zhihong Chen, Guiming Hardy Chen, Shizhe Diao, Xiang Wan, Benyou Wang.
    On the Difference of BERT-style and CLIP-style Text Encoders
    Findings of ACL 2023
  22. Shizhe Diao, Wangchunshu Zhou, Xinsong Zhang, Jiawei Wang.
    Write and Paint: Generative Vision-Language Models are Unified Modal Learners
    ICLR 2023
  23. Shizhe Diao, Zhichao Huang, Ruijia Xu, Xuechun Li, Yong Lin, Xiao Zhou, Tong Zhang.
    Black-box Prompt Learning for Pre-trained Language Models
    TMLR
  24. Shizhe Diao*, Sedrick Scott Keh*, Liangming Pan, Zhiliang Tian, Yan Song, Tong Zhang.
    Hashtag-Guided Low-Resource Tweet Classification
    WWW 2023
  25. Wangchunshu Zhou*, Yan Zeng*, Shizhe Diao*, Xinsong Zhang*.
    VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models
    ICML 2022
  26. Xiao Zhou, Weizhong Zhang, Zonghao Chen, Shizhe Diao, Tong Zhang.
    Efficient Neural Network Training via Forward and Backward Propagation Sparsification
    NeurIPS 2021
  27. Shizhe Diao, Ruijia Xu, Hongjin Su, Yilei Jiang, Yan Song, Tong Zhang.
    Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation
    ACL 2021
  28. Shizhe Diao*, Xinwei Shen*, KaShun SHUM, Yan Song, Tong Zhang.
    TILGAN: Transformer-based Implicit Latent GAN for Diverse and Coherent Text Generation
    Findings of ACL 2021
  29. Shizhe Diao, Jiaxin Bai, Yan Song, Tong Zhang, and Yonggang Wang.
    ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations
    Findings of EMNLP 2020

Preprints


  1. Dylan Zhang, Shizhe Diao, Xueyan Zou, Hao Peng
    PLUM: Preference Learning Plus Test Cases Yields Better Code Language Models [NEW]
  2. Xin Xu, Shizhe Diao, Can Yang, Yang Wang
    Can We Verify Step by Step for Incorrect Answer Detection?
  3. Ziqiang Zheng, Jipeng Zhang, Tuan-Anh Vu, Shizhe Diao, Yue Him Wong Tim, Sai-Kit Yeung
    MarineGPT: Unlocking Secrets of Ocean to the Public
  4. Hanze Dong*, Shizhe Diao*, Weizhong Zhang, Tong Zhang.
    Normalizing Flow with Variational Latent Representation
  5. Rui Pan*, Shizhe Diao*, Jianlin Chen, Tong Zhang.
    ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT
    [Paper] [Code] [Documentation]

Recent Talks




Honors and Awards



Experience


Oct. 2021 - Jul. 2022
Research Intern, ByteDance AI Lab, China
Vision-Language Foundation Models
Advisor: Dr. Hang Li and Dr. Xinsong Zhang

Jun. 2019 - Jan. 2020
Research Intern, Sinovation Ventures AI Institute, China
Pre-trained Language Models
Advisor: Prof. Yan Song

Apr. 2018 - Oct. 2018
Research Intern, National University of Singapore (NUS), Singapore
Semi-supervised End-to-End Dialogue system
Advisor: Prof. Min-Yen Kan and Dr. Wenqiang Lei

Mar. 2017 - Mar. 2019
Research Intern, Peking University (PKU), China
Multimodal Chinese Poem Generation
Advisor: Prof. Xiaojun Wan

Sept. 2017 - Dec. 2017
Exchange Student
The Chinese University of Hong Kong (CUHK), HKSAR

Jul. 2017 - Aug. 2017
Visiting Student, Ben-Gurion University of the Negev (BGU), Israel
Cyber Security and Business Intelligence

Academic Service



Teaching Assistant



Miscellaneous