CV

Education

• Bachelor of Computer Science and Technology, University of Science and Technology of China (USTC), Sept. 2018 - June 2022
• Ph.D student at Institute of Computing Technology, Chinese Academy of Sciences, Sept. 2022 - present

Work experience

• Aug. 2021 - Jan. 2023 & Aug. 2023 - June 2025: Research intern
  • GenAI group, Microsoft Research Asia
  • Supervisor: Dr. Furu Wei and Shuming Ma
• Feb. 2021 - July 2021: Teaching Assistant for Mathematical analysis B2, undergraduate course
  • USTC
  • Supervisor: Professor Yelong Zheng
• Sept. 2020 - Jan. 2021: Teaching Assistant for Mathematical analysis B1, undergraduate course
  • USTC
  • Supervisor: Professor Yelong Zheng

Awards

• Outstanding Teaching Assistant at USTC, 2021
• Silver Award of Outstanding Student Scholarship at USTC, 2020
• Huawei Scholarship, 2020
• Silver Award of Outstanding Student Scholarship at USTC, 2019

Services

• Journal Reviewer: IEEE Transactions on Affective Computing.
• Conference Reviewer: ICLR, ICCV, ECCV, EMNLP, ACL Rolling Review.

Talks

• (05/2024) The Era of 1-bit LLM (BitNet b1 / b1.58 / a4.8 / v2 ) at Xiaomi LLM-Core. Slides.
• (08/2024) The Era of 1-bit LLM at MSR Asia Intern Tech Fest.
• (06/2024) The Era of 1-bit LLM at Cohere for AI, and SIGMA Lab, Tsinghua University.
• (04/2024) BitNet b1.58 at MiraclePlus.
• (08/2023) Magneto at Microsoft Research Asia.

Preprints & Publications

Scalable and Efficient Foundation Model

• MoTE: Mixture of Ternary Experts for Memory-efficient Large Multimodal Models. Hongyu Wang, Jiayu Xu, Ruiping Wang, Yan Feng, Yitao Zhai, Peng Pei, Xunliang Cai, Xilin Chen.
• BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation. Hongyu Wang, Chuyan Xiong, Ruiping Wang, Xilin Chen.
• BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs. Hongyu Wang*, Shuming Ma*, Furu Wei.
• BitNet b1.58 2B4T Technical Report. Shuming Ma*, Hongyu Wang*, Shaohan Huang, Xingxing Zhang, Ying Hu, Ting Song, Yan Xia, Furu Wei.
• BitNet a4.8: 4-bit Activations for 1-bit LLMs. Hongyu Wang*, Shuming Ma*, Furu Wei.
• Bitnet.cpp: Efficient Edge Inference for Ternary LLMs. Jinheng Wang, Hansong Zhou, Ting Song, Shijie Cao, Yan Xia, Ting Cao, Jianyu Wei, Shuming Ma, Hongyu Wang, Furu Wei. ACL 2025.
• Q-Sparse: All Large Language Models can be Fully Sparsely-Activated. Hongyu Wang*, Shuming Ma*, Ruiping Wang, Furu Wei.
• BitNet: 1-bit Pre-training for Large Language Models. Hongyu Wang*, Shuming Ma*, Lingxiao Ma, Lei Wang, Wenhui Wang, Li Dong, Shaohan Huang, Huaijie Wang, Jilong Xue, Ruiping Wang, Yi Wu, Furu Wei. Journal of Machine Learning Research (JMLR 2025) [BitNet b1] [BitNet b1.58]
• DeepNet: Scaling Transformers to 1,000 Layers. Hongyu Wang*, Shuming Ma*, Li Dong, Shaohan Huang, Dongdong Zhang, Furu Wei, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI 2024).
• Magneto: A Foundation Transformer. Hongyu Wang*, Shuming Ma*, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei. International Conference on Machine Learning (ICML), 2023. Slides Poster
• TorchScale: Transformers at Scale. Shuming Ma*, Hongyu Wang*, Shaohan Huang, Wenhui Wang, Zewen Chi, Li Dong, Alon Benhaim, Barun Patra, Vishrav Chaudhary, Xia Song, Furu Wei.

Multimodal and Robotics

• M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models. Hongyu Wang*, Jiayu Xu*, Senwei Xie*, Ruiping Wang, Jialin Li, Zhaojie Xie, Bin Zhang, Chuyan Xiong, Xilin Chen.
• Robotic Programmer: Video Instructed Policy Code Generation for Robotic Manipulation. Senwei Xie*, Hongyu Wang*, Zhanqi Xiao*, Ruiping Wang, Xilin Chen. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025.

Media Reports

Search for 1-bit LLMs

• BitNet v2:
  1. 微软再放LLM量化大招！原生4bit量化，成本暴减，性能几乎0损失 (新智元)
  2. 微软1bit LLM新研究：原生4bit激活值量化，可充分利用新一代GPU对4bit计算的原生支持 (量子位)
• BitNet b1.58 2B4T:
  1. Microsoft researchers say they’ve developed a hyper-efficient AI model that can run on CPUs. (TechCrunch)
  2. Microsoft Fiscal Year 2025 Third Quarter Earnings Conference Call
  3. 仅需0.4GB，参数只有0和±1！微软开源首个原生1 bit模型，CPU轻松跑 (新智元)
  4. 微软开源“原生1bit”三进制LLM：2B参数，0.4GB内存/单CPU就能跑，性能与同规模全精度开源模型相当 (量子位)
  5. Low Precision, High Performance: Researchers at Microsoft and Tsinghua researchers propose 1.58-bit AI model that rivals full-precision competitors (Deeplearning.AI)
• BitNet a4.8:
  1. How Microsoft’s next-gen BitNet architecture is turbocharging LLM efficiency (VentureBeat)
  2. 1-bit大模型还能再突破！新一代BitNet架构启用4位激活值 (新智元)
• BitNet.cpp: 微软开源爆火1.58bit大模型推理框架！千亿参数模型量化后单CPU可跑，速度每秒5-7个token (量子位)
• Q-Sparse:
  1. 完全激活稀疏大模型，Q-Sparse突破LLM推理效能 (MSRA)
  2. 只激活3.8B参数，性能比肩同款7B模型！训练微调都能用，来自微软 (新智元)
• BitNet b1.58:
  1. 1-bit LLMs Could Solve AI’s Energy Demands (IEEE Spectrum)
  2. Small Bits, Big Ideas: The Amazing Rise Of 1-Bit LLMs For Building Faster And Slimmer Generative AI Apps (Forbes)
  3. State of AI report 2024
  4. 微软6页论文爆火：三进制LLM，真香！(量子位)
  5. 微软、国科大开启1Bit时代：大模型转三进制，速度快4倍能耗降至1/41 (机器之心)
  6. BitNet b1.58：开启1-bit大语言模型时代 (MSRA)
  7. 「1ビットLLM」でAIのエネルギー効率1桁改善、Microsoft研究所の幹部が力説 (xTech)
• TorchScale: 让天下没有难训练的大模型，微软亚洲研究院开源TorchScale (MSRA)
• DeepNet:
  1. Transformer深至1000层还能稳定训练，微软实习生一作，LSTM之父转发 (量子位)
  2. 解决训练难题，1000层的Transformer来了，训练代码很快公开 (机器之心)
  3. 千层Transformer问世！多语种机器翻译水准刷新多项SOTA (新智元)