Skip to content

Issues: microsoft/DeepSpeed

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
498 Open 1,026 Closed
498 Open 1,026 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[BUG] any clue for MFU drop? bug Something isn't working training
#6727 opened Nov 8, 2024 by SeunghyunSEO
1
[BUG] [ROCm] Fine-tuning DeepSeek-Coder-V2-Lite-Instruct with 8 MI300X GPUs results in c10::DistBackendError bug Something isn't working rocm AMD/ROCm/HIP issues training
#6725 opened Nov 8, 2024 by nikhil-tensorwave
@loadams
2
CUBLAS_STATUS_NOT_SUPPORTED bug Something isn't working training
#6723 opened Nov 7, 2024 by niebowen666
@hwchen2017
5
[BUG] Zero3 for torch.compile with compiled_autograd when running LayerNorm bug Something isn't working training
#6719 opened Nov 6, 2024 by yitingw1
@tohtana
1
[BUG] DeepSpeed accuracy issue for torch.compile if activation checkpoint function not compiler disabled bug Something isn't working training
#6718 opened Nov 6, 2024 by jerrychenhf
@tohtana
[BUG] The problem of using Deepspeed to start training bug Something isn't working training
#6715 opened Nov 5, 2024 by sanxiaojijiaben
@loadams
1
[BUG]Issue with Zero Optimization for Llama-2-7b Fine-Tuning on Intel GPUs bug Something isn't working training
#6713 opened Nov 5, 2024 by molang66
4
[BUG] NCCL Timeout When Pre-traing "ds_train_bert_nvidia_data_bsz32k_seq512". bug Something isn't working training
#6705 opened Nov 3, 2024 by always-H
2
[BUG] Universal Checkpoint Conversion: Resumed Training Behaves as If Model Initialized from Scratch bug Something isn't working training
#6691 opened Oct 30, 2024 by purefall
1
[BUG] While submodule forward process in different gpu is not same, loss.backward get stuck bug Something isn't working training
#6667 opened Oct 25, 2024 by fuzuoyi
@jomayeri
3
[BUG] Deepspeed launcher not picking up virtualenv --system-site-packages bug Something isn't working build Improvements to the build and testing systems.
#6664 opened Oct 24, 2024 by VRehnberg
@loadams
8
[BUG] ZeRO++ sharding small parameter raise IndexError bug Something isn't working training
#6659 opened Oct 23, 2024 by wuxibin89
[BUG] Training batch size is not consistent with train_batch_size bug Something isn't working training
#6657 opened Oct 23, 2024 by tnnandi
1
[BUG] RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1! bug Something isn't working training
#6643 opened Oct 20, 2024 by RickoNoNo3
[BUG] failed to find frozen {param} in named params bug Something isn't working training
#6620 opened Oct 11, 2024 by ssklzx
@jomayeri
4
[BUG] Non-Deterministic Model Responses when the Input Prompt Order Changes bug Something isn't working inference
#6612 opened Oct 8, 2024 by zcakzhuu
@tohtana
[BUG] MOE: Loading experts parameters error when using expert parallel. bug Something isn't working training
#6589 opened Sep 29, 2024 by kakaxi-liu
1
[BUG] DeepSpeed Ulysses zero3 compatibility bug Something isn't working training
#6582 opened Sep 27, 2024 by Xirid
1
[BUG] AttributeError: 'NoneType' object has no attribute 'set_moe' bug Something isn't working inference
#6572 opened Sep 25, 2024 by zhanwenchen
@loadams
1
[BUG] ValueError: Tensors must be contiguous when using deepspeed.initialize bug Something isn't working training
#6571 opened Sep 25, 2024 by shadow150519
2
[BUG] AssertionError: Unable to pre-compile ops without torch installed. Please install torch before attempting to pre-compile ops. bug Something isn't working training
#6568 opened Sep 24, 2024 by umarbutler
3
[BUG] Expert gradient scaling problem with ZeRO optimizer bug Something isn't working training
#6545 opened Sep 17, 2024 by wyooyw
@tohtana
1
[BUG] Distributed Training randomly stuck in trainings loop bug Something isn't working training
#6524 opened Sep 11, 2024 by raeudigerRaeffi
@tohtana
2
[BUG] error ：past_key, past_value = layer_past，how to solve this ? bug Something isn't working deepspeed-chat Related to DeepSpeed-Chat
#6522 opened Sep 11, 2024 by lovychen
@loadams
2
[BUG] RuntimeError: Error building extension 'inference_core_ops' bug Something isn't working build Improvements to the build and testing systems.
#6519 opened Sep 10, 2024 by Chetan3200
@loadams
2
ProTip! no:milestone will show everything without a milestone.