OLMoE Q4_0 quant does not work

[l3utterfly]

Name and Version

commit hash: a4f011e

Operating systems

Other? (Please let us know in description)

GGML backends

CPU

Hardware

Snapdragon 8 Gen 2

Models

Model is here: https://huggingface.co/allenai/OLMoE-1B-7B-0125-Instruct-GGUF/tree/main

Problem description & steps to reproduce

It is failing with the following error for arrch64:

llama.cpp/ggml/src/ggml-cpu/ggml-cpu-aarch64.cpp:4013: GGML_ASSERT(params->wsize >= (GGML_PAD(nbw3, sizeof(int64_t)) + n_as * sizeof(int64_t) + n_as * ne12 * sizeof(mmid_row_mapping))) failed

Model is here: https://huggingface.co/allenai/OLMoE-1B-7B-0125-Instruct-GGUF/tree/main

Do you know why this error happens? Does the model need to be re-quanted?

First Bad Commit

No response

Relevant log output

llama.cpp/ggml/src/ggml-cpu/ggml-cpu-aarch64.cpp:4013: GGML_ASSERT(params->wsize >= (GGML_PAD(nbw3, sizeof(int64_t)) + n_as * sizeof(int64_t) + n_as * ne12 * sizeof(mmid_row_mapping))) failed

[ggerganov]

Does it work with cmake -DGGML_CPU_AARCH64=OFF ...?