feat: pass model parameters to HFLocalInvocationLayer via model_kwargs, enabling direct model usage

[vblagoje]

Related Issues

• partially fixes feat: Ensure Agent works with open-source and other models #4557

Problem:

As we continue to see advancements in large language model (LLM) development, there has been a notable shift towards custom torch architectures not yet supported in HuggingFace's transformers. This trend poses a challenge for our existing setup, where we rely heavily on the transformers pipeline approach. Take, for example, MPT open-source models. As custom MPT model architecture is not yet part of the Hugging Face transformers and because it includes many custom architectural approaches ranging from FlashAttention, ALiBi, QK LayerNorm, one needs to create a model for MPT "by hand":

tokenizer = transformers.AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
config = transformers.AutoConfig.from_pretrained(
  'mosaicml/mpt-7b-instruct',
  trust_remote_code=True
)
config.attn_config['attn_impl'] = 'triton'

model = transformers.AutoModelForCausalLM.from_pretrained(
  'mosaicml/mpt-7b-instruct',
  config=config,
  torch_dtype=torch.bfloat16,
  trust_remote_code=True
)
model.to(device='cuda:0')

To complicate things, some models use tokenizers from other existing models, just like MPT models do.
This custom architecture trend will only likely increase, racing ahead of transformers.

Proposed Changes:

This PR proposes directly integrating HuggingFace models within PromptNode via the model_kwargs model parameter, allowing for more flexibility and compatibility with rapidly evolving LLM architectures.
Assuming someone created the above-mentioned model and tokenizer already, our users would simply create a PromptNode like this:

from haystack.nodes import PromptNode
pn = PromptNode("mosaicml/mpt-7b-instruct", stop_words=["<|endoftext|>"], model_kwargs={"model":model, "tokenizer": tokenizer})

Key Benefits:

Enhanced Flexibility: this minor change allows us to incorporate the latest LLMs into the Haystack ecosystem, including those based on custom torch architectures, increasing our capability to stay current with LLM advancements.
Future-proof: as LLM research outpaces transformers, this change ensures PromptNode is prepared to adapt quickly to new developments.

How did you test it?

New unit tests, manual tests, and custom demo colab

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added tests that demonstrate the correct behavior of the change
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[masci]

this should be mocked and made unit

[coveralls]

Pull Request Test Coverage Report for Build 5197464139

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 406 unchanged lines in 11 files lost coverage.
• Overall coverage increased (+0.7%) to 40.74%

---

Files with Coverage Reduction	New Missed Lines	%
nodes/prompt/prompt_template.py	1	89.39%
nodes/prompt/prompt_model.py	4	80.0%
nodes/answer_generator/openai.py	8	87.16%
nodes/audio/whisper_transcriber.py	11	24.32%
nodes/prompt/invocation_layer/hugging_face.py	15	83.2%
nodes/prompt/invocation_layer/chatgpt.py	19	55.74%
nodes/prompt/invocation_layer/open_ai.py	20	62.34%
nodes/prompt/prompt_node.py	30	40.57%
nodes/retriever/dense.py	49	25.98%
modeling/training/base.py	81	14.29%

Totals
Change from base Build 5174989324:	0.7%
Covered Lines:	9138
Relevant Lines:	22430

---

💛 - Coveralls

[masci]

Took another pass

[vblagoje]

Took another pass

Thanks, learned a ton! LMK if there are any additional changes needed.

[masci]

🚢