Hongyu Wang

Hongyu Wang

Ph.D student at CAS

Biography

Hongyu Wang is a Ph.D. candidate (expected to graduate in July 2027) at the Institute of Computing Technology, Chinese Academy of Sciences (ICT, CAS), and the University of Chinese Academy of Sciences (UCAS), advised by Professor Xilin Chen. His research interests include LLM pre-training, efficient foundation models, and scalable neural network architectures. He received his B.Eng. in Computer Science and Technology from the University of Science and Technology of China (USTC) in 2022. From August 2021 to June 2025, he was a Research Intern with the General Artificial Intelligence Group at Microsoft Research Asia, where he worked under the supervision of Dr. Furu Wei and Shuming Ma.

His research has been published in leading AI venues, including TPAMI, JMLR, and ICML. He is the lead author of DeepNet, Magneto, BitNet, and BitVLA. DeepNet and Magneto pioneered scaling Transformers beyond 1,000 layers and have been adopted by several influential open-source foundation models. BitNet introduced the first 1-bit large language model architecture and pre-training methodology, while BitNet 2B4T became the first LLM with native 1.58-bit weights. His latest work, BitVLA, enables the first fully 1.58-bit vision-language and vision-language-action models. His publications have received over 1,400 Google Scholar citations.

I have great interest on the following topics:

  1. Scale efficiently! Efficient architecture for the large-scale foundation models
  2. Multimodal reasoning, robotics

Contact: why0711@mail.ustc.edu.cn

I’m currently open to full-time opportunities starting in Summer 2027. If you believe my background could be a good fit for your team, I’d be glad to talk.

News:

  • [06/2025] BitNet is accepted as the regular paper by JMLR 2025!
  • [06/2025] Introducing BitVLA, the first 1-bit VLA model for robotics manipulation and multimodal tasks! Model weights and code are public!
  • [05/2025] Wrote a slides to review our exploration in BitNet series . Feel free to send your questions through e-mail.
  • [04/2025] BitNet v2, native 4-bit activations for 1-bit LLMs.
  • [04/2025] Introducing BitNet b1.58 2B4T, the first native 1-bit LLM trained at scale! Model weights and technical report are public! Cooking larger models now…
  • [11/2024] BitNet a4.8, enabling 4-bit activations for 1-bit LLMs. BitNet a4.8 has only 55% active parameters and further supports 3-bit KV cache without extra training.
  • [10/2024] bitnet.cpp, the official inference framework for BitNet b1.58! Run a 100B BitNet b1.58 model on a single CPU at a human reading speed!
  • [07/2024] Q-Sparse, the fully Sparsely-Activated LLM.
  • [04/2024] DeepNet is accepted as the regular paper by TPAMI 2024.
  • [03/2024] BitNet b1.58: Training Tips, Code and FAQ.
  • [02/2024] BitNet b1.58, the first ternary LLM that matches the performance of FP16 LLM with siginificant reduction of inference cost (latency, memory, throughput, and energy consumption)