I am now a final-year Ph.D. candidate at the Hong Kong University of Science and Technology, advised by Professor Tong Zhang. I received my Master of Philosophy degree at HKUST in 2021. Before joining HKUST, I received my Bachelor of Science degree at Beijing Normal University in 2019. I was a research intern at ByteDance AI Lab with Dr. Hang Li and Dr. Xinsong Zhang, and Sinovation Ventures AI Institute. During my undergraduate study, I interned at WING Group @ NUS and PKU, where I was fortunate to work with Prof. Min-Yen Kan and Prof. Xiaojun Wan. My research focuses on pre-training, efficient-tuning, and adaptation of large language models. Check out this curated paper list about ChatGPT with the goal of helping everyone learn the techniques behind it.
Ph.D.
Sept. 2021 - Present
The Hong Kong University of Science and Technology (HKUST), HKSAR Ph.D. student in Computer Science Advisor: Prof. Tong Zhang |
![]() |
M.Phil.
Sept. 2019 - Jun. 2021
The Hong Kong University of Science and Technology (HKUST), HKSAR M.Phil. student in Computer Science Advisor: Prof. Tong Zhang |
![]() |
B.S.
Sept. 2015 - Jun. 2019
Beijing Normal University (BNU), China B.S. in Computer Science and Technology |
![]() |
Publications
- Shizhe Diao, Wangchunshu Zhou, Xinsong Zhang, Jiawei Wang.
Write and Paint: Generative Vision-Language Models are Unified Modal Learners [NEW]
ICLR 2023
- Shizhe Diao, Zhichao Huang, Ruijia Xu, Xuechun Li, Yong Lin, Xiao Zhou, Tong Zhang.
Black-box Prompt Learning for Pre-trained Language Models [Updated Version 2.0]
TMLR
- Shizhe Diao*, Sedrick Scott Keh*, Liangming Pan, Zhiliang Tian, Yan Song, Tong Zhang.
Hashtag-Guided Low-Resource Tweet Classification [NEW]
WWW 2023
- Wangchunshu Zhou*, Yan Zeng*, Shizhe Diao*, Xinsong Zhang*.
VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models
ICML 2022
- Xiao Zhou, Weizhong Zhang, Zonghao Chen, Shizhe Diao, Tong Zhang.
Efficient Neural Network Training via Forward and Backward Propagation Sparsification
NeurIPS 2021
- Shizhe Diao, Ruijia Xu, Hongjin Su, Yilei Jiang, Yan Song, Tong Zhang.
Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation
ACL 2021
- Shizhe Diao*, Xinwei Shen*, KaShun SHUM, Yan Song, Tong Zhang.
TILGAN: Transformer-based Implicit Latent GAN for Diverse and Coherent Text Generation
Findings of ACL 2021
- Shizhe Diao, Jiaxin Bai, Yan Song, Tong Zhang, and Yonggang Wang.
ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations
Findings of EMNLP 2020
- FengKuang Chiang, Shizhe Diao, Haotian Ma, Yujun Wang.
Effects of Hands-on Inquiry-Based Learning Using LEGO Materials on the Learning of Eighth-Grade Physics Students.
International Journal of Engineering Education, vol. 33, ser. 3, 2017, pp. 1098-1103. 3.
- Yunchuan Sun, Mengting Fang, Xinyu Wang, Shizhe Diao.
GubaLex: Guba-oriented sentiment lexicon for big texts in finance
The 13th International Conference on Semantics, Knowledge and Grids
Preprints
- Rui Pan*, Shizhe Diao*, Jianlin Chen, Tong Zhang.
ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT
[Paper] [Code] [Documentation]
arXiv preprint arXiv:2211.17201 (2022)
- Hanze Dong*, Shizhe Diao*, Weizhong Zhang, Tong Zhang.
Normalizing Flow with Variational Latent Representation
arXiv preprint arXiv:2211.11638 (2022)
- Zhihong Chen*, Shizhe Diao*, Benyou Wang, Guanbin Li, Xiang Wan.
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts [NEW]
arXiv preprint arXiv:2302.08958 (2023)
- Shizhe Diao, Pengcheng Wang, Yong Lin, Tong Zhang.
Active Prompting with Chain-of-Thought for Large Language Models [NEW]
arXiv preprint arXiv:2302.12246 (2023)
- Kashun Shum*, Shizhe Diao*, Tong Zhang.
Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data [NEW]
arXiv preprint arXiv:2302.12822 (2023)
Honors and Awards
- HKUST RedBird PhD Scholarship, 2021
- Hong Kong PhD Fellowship, 2021-2024
- EMNLP 2020, SIGIR 2020, ACL 2020 Student Volunteer, 2020
- Merit Student of Beijing (北京市三好学生), 2019
- Outstanding Graduate Student of Beijing (北京市优秀毕业生), 2019
- Top 10 Talent Nomination Award (十佳大学生提名) (Only 20 in ~2000), 2018
- Runner-up, World Robot Olympiad (WRO), New Delhi, India, 2016
- Meritorious Winner, Interdisciplinary Contest in Modeling (ICM), 2018
- First-class Scholarship for Academic Achievement (Top 5% in 64), 2015-2017
- Merit Student of Beijing Normal University (Top 10% in ~2000), 2015-2017
- First Prize, Mathematical Contest in Modelling, Beijing Normal University, 2016
- Outstanding Volunteer Award, Volunteer Teaching Program of UNICEF, 2015
- Excellent Volunteer in BRICS National Education Forum, 2015
Oct. 2021 - Present Research Intern, ByteDance AI Lab, China Vision-Language Foundation Models Advisor: Dr. Hang Li and Dr. Xinsong Zhang |
![]() |
Jun. 2019 - Jan. 2020 Research Intern, Sinovation Ventures AI Institute, China Pre-trained Language Models Advisor: Prof. Yan Song |
![]() |
Apr. 2018 - Oct. 2018 Research Intern, National University of Singapore (NUS), Singapore Semi-supervised End-to-End Dialogue system Advisor: Prof. Min-Yen Kan and Dr. Wenqiang Lei |
![]() |
Mar. 2017 - Mar. 2019 Research Intern, Peking University (PKU), China Multimodal Chinese Poem Generation Advisor: Prof. Xiaojun Wan |
![]() |
Sept. 2017 - Dec. 2017 Exchange Student The Chinese University of Hong Kong (CUHK), HKSAR |
![]() |
Jul. 2017 - Aug. 2017 Visiting Student, Ben-Gurion University of the Negev (BGU), Israel Cyber Security and Business Intelligence |
![]() |
Academic Service
- Journal Reviewer: SIAM Journal on Mathematics of Data Science (SIMODS)
- Conference Reviewer: ACL ARR, ACL (2020 - 2023), EMNLP (2020 - 2022), NAACL (2020 - 2022), NeurIPS (2022), ICML (2022 - 2023), KDD (2023), AAAI (2022), IJCAI (2023), EACL (2022)
- Volunteer: EMNLP 2020, SIGIR 2020, ACL 2020
Teaching Assistant
- COMP2011 Programming with C++ (Spring 2022)
- COMP3711 Design and Analysis of Algorithms (Fall 2020)
- COMP6211E Optimization for Machine Learning (Spring 2020)
Miscellaneous
- I used to be an amateur long-distance runner 🏃. Whenever I am not doing research, I love swimming 🏊, kayaking 🚣, windsurfing 🏄, dinghy sailing ⛵, and stand up paddling!
- As the captain, I organized a team to participate in the World Robot Olympiad (WRO) and won the second place in India.