IBM Granite

[eltonjohnfanboy]

Hi!

I am writing to inquire about the future support plans for the Granite 3B and 8B models in the llama-cpp-python library. While attempting to load the small GGUF models for these Granite models using llama-cpp-python, I encountered the following error:
error loading model: done_getting_tensors: wrong number of tensors; expected 578, got 470
I suspect we get this issue because the small Granite models (3B and 8B) are not yet supported by this library. Are there any information on any plans to support these models in the future?

Thanks! :))

[abetlen]

@eltonjohnfanboy should be in the newest release (0.2.77), let me know if you have any issues.

[eltonjohnfanboy]

@abetlen Great, it's working properly! Thanks a lot:))