Skip to content

Releases: ggerganov/llama.cpp

b4153

22 Nov 10:41
6dfcfef
Compare
Choose a tag to compare
ci: Update oneAPI runtime dll packaging (#10428)

This is the minimum runtime dll dependencies for oneAPI 2025.0

b4151

22 Nov 07:22
c18610b
Compare
Choose a tag to compare
CANN: Support Ascend310P to accelerate F32 and F16 Model (#10216)

* CANN Support Ascend310P to accelerate F32 and F16 Model

* Add compile option soc type macro ASCEND_310P to ggml-cann lib

* Remove unused code

* Remove the ascend soc_type hard code compile option in CMakelist.txt

b4150

21 Nov 18:27
a5e4759
Compare
Choose a tag to compare
cuda : optimize argmax (#10441)

* cuda : optimize argmax

* remove unused parameter

ggml-ci

* fixup : use full warps

ggml-ci

* Apply suggestions from code review

Co-authored-by: Johannes Gäßler <[email protected]>

* fix ub

* ggml : check ne00 <= INT32_MAX in argmax and argsort

---------

Co-authored-by: Johannes Gäßler <[email protected]>

b4149

21 Nov 09:23
1bb30bf
Compare
Choose a tag to compare
llama : handle KV shift for recurrent models (#10402)

ggml-ci

b4148

21 Nov 08:49
87a533b
Compare
Choose a tag to compare
sync : ggml

b4143

20 Nov 15:17
fab5d30
Compare
Choose a tag to compare
llama : add .clang-format file (#10415)

b4142

20 Nov 09:03
8fd4b7f
Compare
Choose a tag to compare
vulkan: copy iq4_nl LUT into shared memory (#10409)

b4141

20 Nov 08:42
1bacb9f
Compare
Choose a tag to compare
vulkan: further optimize mul_mat_vec using larger loads (#10387)

* vulkan: Use pipeline_robustness to disable robustness in mul_mat_vec.

Add some early returns for nonexistent rows in mul_mat_vec shaders. These
can only be hit when dispatching a 2D grid of workgroups. Fix the logic
for the 2D grid of workgroups to round up.

Enable the pipeline robustness extension if it's available, and use it to
disable robustness for these pipelines. The instructions to do the bounds
checking contend for the same ALU resources as the bit twiddling dequant
instructions.

* vulkan: Add GLSL structure aliases for quant types to allow larger loads

In Vulkan it's not possible to cast pointer types, so instead you have to
declare an aliased binding for the memory with a different type. This
commit adds aliases for the quant formats using 16b ints, and in a few
places where the struct size is a multiple of 4 also using 32b ints.
Currently only q4_k's aliases are used, but others will be used in
subsequent commits.

* vulkan: use larger loads in q5_k and q6_k shaders.

Similar to the optimization I did in q4_k recently, this vectorizes some loads
and reduces the number of bit twiddling instructions.

* vulkan: use larger K step per iteration in mul_mat_vec.

Add vec4 dequantization functions, and use them to do K=8 per iteration in
mul_mat_vec. This uses 16b loads for the quant values and 128b loads for B
which helps reduce the load on the memory system.

The K_PER_ITER==2 logic is still there, just for F16/F32, and really only
because they support unaligned sizes.

Tweak the num_iters/unrolling logic to be simpler and catch a couple missed
unrolling opportunities.

b4139

19 Nov 23:39
3952a22
Compare
Choose a tag to compare
Fix missing file renames in Makefile due to changes in commit ae8de6d…

b4138

19 Nov 21:30
42ae10b
Compare
Choose a tag to compare
add cmake rvv support (#10411)