Skip to content

Issues: NVIDIA/TensorRT-LLM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

402 Open 1,827 Closed
402 Open 1,827 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Qwen2VL, 报错 TypeError: sequence item 0: expected str instance, NoneType found
#2805 opened Feb 20, 2025 by dong-168
speculative decoding not work
#2804 opened Feb 20, 2025 by biaochen
Possible bug in Qwen convert_checkpoint.py Investigating Low Precision Issue about lower bit quantization, including int8, int4, fp8 triaged Issue has been triaged by maintainers
#2803 opened Feb 19, 2025 by mathijshenquet
@Barry-Delaney
1
Running Llama2-70B TP2 on L40s (with fp8 quantization)
#2802 opened Feb 19, 2025 by cquil11
1
Open MPI error while converting model checkpoints
#2801 opened Feb 19, 2025 by adityarajsahu
Why replace the GptManager interface with the Executor interface?
#2800 opened Feb 19, 2025 by fyf2016
1
run.py gets stuck when running Speculative Decoding with Draft model bug Something isn't working
#2798 opened Feb 19, 2025 by ValeGian
2 of 4 tasks
2
DeepSeek-R1-Distill-Qwen-1.5B inference bias bug Something isn't working
#2797 opened Feb 19, 2025 by Mitty-ZH
2 of 4 tasks
Multimodal Cross-attention incorrect results in bug Something isn't working
#2796 opened Feb 19, 2025 by mutkach
2 of 4 tasks
2
[Feature Request] TensorRT-LLM libs should work with older versions of GLIBC
#2795 opened Feb 19, 2025 by vihangm
[Model Requests] Support Qwen2.5-VL Architecture
#2794 opened Feb 18, 2025 by mtezgider
deepseek branch link error: undefined reference to `std::ios_base_library_init()' from libfp8_blockscale_gemm.a bug Something isn't working
#2791 opened Feb 18, 2025 by wahaha22
2 of 4 tasks
2
Incompatible buffer sizes when running Speculative decoding with draft/target models on Qwen2 bug Something isn't working
#2789 opened Feb 17, 2025 by gloritygithub11
1 of 4 tasks
When executing this command, many modules are missing, such as luguru, hydra, and llava. Is there any way to install all of these at once? bug Something isn't working triaged Issue has been triaged by maintainers
#2788 opened Feb 17, 2025 by J4S0N666
4 tasks
3
Deepseek-v3 running on 2xH100 nodes getting poor performanc bug Something isn't working triaged Issue has been triaged by maintainers
#2786 opened Feb 14, 2025 by zymy-chen
2 of 4 tasks
1
The performance of Qwen1.5-7B based on the trtllm-bench test was very poor bug Something isn't working
#2785 opened Feb 14, 2025 by ruru5697
3 of 4 tasks
Does anyone try TensorRT-LLM for ComfyUI
#2784 opened Feb 13, 2025 by TuanNT-ZenAI
Bug when loading an engine using LoRA through LLM API bug Something isn't working Investigating LLM API/Workflow triaged Issue has been triaged by maintainers
#2782 opened Feb 13, 2025 by pei0033
2 of 4 tasks
@nv-guomingz
2
User Buffer and Reduce Fusion overwritten to False
#2781 opened Feb 12, 2025 by jolyons123
GPU Utilization drops gradually over time using Executor API bug Something isn't working
#2778 opened Feb 12, 2025 by MahmoudAshraf97
3 of 4 tasks
1
Inconsistent Batch Index Order in Decoupled Mode with trt-llm bug Something isn't working
#2777 opened Feb 12, 2025 by Oldpan
2 of 4 tasks
8
DeepSeek-V3 fp8 tp32 failed to convert chectpoint bug Something isn't working triaged Issue has been triaged by maintainers
#2776 opened Feb 12, 2025 by MtFitzRoy
2 of 4 tasks
1
Processing multi concurrent request by Qwen2-VL is slow. It seems infer in queue.
#2775 opened Feb 12, 2025 by zhaocc1106
Limit max GPU memory used
#2773 opened Feb 11, 2025 by bri25yu
Cannot create checkpoint for llama-3.2 (1B, 3B) bug Something isn't working triaged Issue has been triaged by maintainers
#2772 opened Feb 11, 2025 by falkbene
3 of 4 tasks
1
ProTip! Add no:assignee to see everything that’s not assigned.