Discussions

• 6 You must be logged in to vote

  🙏

  How does llama.cpp achieve CPU-GPU hybrid mode?

  HuangLED asked Jun 26, 2024 in Q&A · Unanswered

  

  

  

  

  10
• 17 You must be logged in to vote

  📣

  repo : upcoming transfer to ggml-org

  ggerganov announced Feb 11, 2025 in Announcements

  

  

  

  

  9
• 6 You must be logged in to vote

  💡

  Inference LLM Deepseek-v3_671B on CPU only.

  jonndoe started Feb 8, 2025 in Ideas

  

  

  

  

  10
• 6 You must be logged in to vote

  💬

  Performance of llama.cpp with Vulkan

  performance Speed related topics need feedback Testing and feedback with results are needed

  netrunnereve started Dec 18, 2024 in General

  

  

  

  

  83
• 1 You must be logged in to vote

  🙏

  what's the replacement of llama_set_rng_seed now?

  shadow3aaa asked Feb 15, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  llama_cli.exe no any response on windows 11 dos command

  DonaldCMLIN asked Feb 15, 2025 in Q&A · Unanswered

  

  

  0
• 6 You must be logged in to vote

  💡

  Using LQER to improve low-bit quants

  compilade started Aug 2, 2024 in Ideas

  

  

  

  

  14
• 1 You must be logged in to vote

  🙏

  Parallel sampling in processing the batch of tokens?

  whitezhang asked Feb 15, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  low GPU utilization for deepseek-r1 1.73 quantized inference in llama.cpp while high CPU utilization

  Jackarry188 asked Feb 15, 2025 in Q&A · Unanswered

  

  

  0
• 10 You must be logged in to vote

  💬

  Performance of llama.cpp on Snapdragon X Elite/Plus

  AndreasKunar started Jul 3, 2024 in General

  

  

  

  

  57
• 1 You must be logged in to vote

  💡

  Quiero aprender c++

  Yohainer started Feb 14, 2025 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  Question about INT8 in AMD MI50

  cdrfvb asked Feb 14, 2025 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  💡

  Throttling GPU usage

  wbruna started Feb 10, 2025 in Ideas

  

  

  1
• 1 You must be logged in to vote

  🙏

  ggml_opencl: plaform IDs not available.

  belog2867 asked Feb 14, 2025 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  Is there anyone run llamacpp on Jetson Orin SoC or other devices, how about the performance?

  adamydwang asked Jan 21, 2024 in Q&A · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  For matrix multiplication operations with quantized models, is the fp32 src1 first quantized to Q8_1?

  BishmoyPaul asked Feb 7, 2025 in Q&A · Unanswered

  

  

  1
• 14 You must be logged in to vote

  💬

  Adjust VRAM/RAM split on Apple Silicon

  dr3murr started Jul 11, 2023 in General

  

  

  

  

  44
• 1 You must be logged in to vote

  🙏

  Failed Vulkan

  okamototk asked Feb 13, 2025 in Q&A · Unanswered

  

  

  4
• 1 You must be logged in to vote

  💬

  What was the last version to support OpenCL

  BlackSwordFIFTY5 started Feb 12, 2025 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  how many rpc-host should I start on remote server

  vino5211 asked Feb 14, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  RPC Failed To Connect

  xuanzhec asked Feb 13, 2025 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Tried the HIP build install instructions and now my Stable Diffusion model doesn't compute

  CurseWave asked Feb 14, 2025 in Q&A · Unanswered

  

  

  0
• 11 You must be logged in to vote

  🙏

  ValueError: Can not map tensor 'model.layers.0.mlp.down_proj.g_idx'

  Frog863 asked Aug 20, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Performance degradation with different GPUs

  JMPSequeira asked Feb 13, 2025 in Q&A · Closed · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  GGML Shape & Strides question for custom kernel/hardware

  akapoor3518 started Feb 12, 2025 in General

  

  

  5