网站介绍
微软开源的低成本实现类似ChatGPT的模型训练
官网公开介绍
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
[2026/05] Using Muon Optimizer with DeepSpeed
[2026/05] System DMA (SDMA) for ZeRO-3: offload collectives off compute units on AMD GPUs for better overlap
[2025/12] DeepSpeed Core API updates: PyTorch-style backward and low-precision master states
[2025/10] SuperOffload: Unleashing the Power of Large-Scale LLM Training on Superchips
[2025/10] Study of ZenFlow and ZeRO offload performance with DeepSpeed CPU core binding
Extreme Speed and Scale for DL Training
DeepSpeed enabled the world’s most powerful language models (at the time of this writing) such as MT-530B and BLOOM. DeepSpeed offers a confluence of system innovations, that has made large scale DL training effective, and efficient, greatly improved ease of use, and redefined the DL training landscape in terms of scale that is possible. These innovations include ZeRO, 3D-Parallelism, DeepSpeed-MoE, ZeRO-Infinity, etc.
DeepSpeed Adoption
DeepSpeed has been used to train many different large-scale models. Below is a list of several examples that we are aware of (if you’d like to include your model please submit a PR):
DeepSpeed has been integrated with several different popular open-source DL frameworks such as:
DeepSpeed is an integral part of Microsoft’s AI at Scale initiative to enable next-generation AI capabilities at scale.
Contributing
DeepSpeed welcomes your contributions! Please see our contributing guide for more details on formatting, testing, etc.
Contributor License Agreement
This project welcomes contributions and suggestions. Most contributions require you to agree to a Developer Certificate of Origin (DCO)[https://wiki.linuxfoundation.org/dco] stating that they agree to the terms published at https://developercertificate.org for that particular contribution.
DCOs are per-commit, so each commit needs to be signed off. These can be signed in the commit by adding the -s flag. DCO enforcement can also be signed off in the PR itself by clicking on the DCO enforcement check.
Code of Conduct