Large Language Models for Developers - from zero to your first LLM application

• 🍿 Talk - 1st edition - Open Source Day 2024
• 🍿 Talk - 2nd edition - PyCon Italy 2024
• 🧑‍🏫 Slide deck

Material for the homonymous talk.

📝 Abstract

The rise of ChatGPT and Large Language Models has revolutionized the tech landscape, leaving developers overwhelmed by the infinite opportunities and intrigued by the technical challenges posed by their complex nature.

This session provides a developer-centric introduction to LLMs, focused on practical applications. No pre-existing knowledge of LLMs and NLP is required.

You will gain insights into: using closed and open-source models, how to effectively prompt LLMs, vector databases, implementing Retrieval Augmented Generation applications (answer generation based on your data), building more complex applications.

Through a hands-on approach, I will show code examples using open-source tools: Haystack LLM framework, Hugging Face Transformers, Ollama, and more. I will also show how you can switch from proprietary to open models.

📚 Resources and code ‍💻

• Haystack LLM framework
• Start from a proprietary model
• Switch to local open LLMs with Ollama
  • Chat with Mistral
• Prompt Engineering
• RAG
  • Naive RAG
  • Web RAG
• Retrieval
  • Document Stores and Retrievers
  • Keyword-based Retrieval (BM25)
  • Embedding/vector Retrieval
  • Hybrid Retrieval
• From Demo to Production
  • Choose a good Language Model
  • Choose a LLM inference solution
  • Evaluation
  • Deployment
• Beyond RAG...
• There is much more!

Haystack LLM framework

• Website
• GitHub
• Main concepts

Start from a proprietary model

• Code snippet
• OpenAI Generator docs

Switch to local open LLMs with Ollama

• Ollama-Haystack integration
• Code snippets: run Ollama; use Ollama in Haystack

Chat with Mistral

• ChatMessage docs
• Code snippet

Prompt Engineering

• OpenAI guide
• promptingguide.ai

RAG

• Introduction to RAG with code examples

Naive RAG

• Code snippet

Web RAG

• Docs

Retrieval

Document Stores and Retrievers

• Document Store abstraction
• Choosing a Document Store
• Retrievers

Keyword-based Retrieval (BM25)

• Introduction to keyword-based retrieval: Bag of Words and TF-IDF; BM25
• BM25 Indexing Pipeline - code snippet
• BM25 RAG Pipeline - code snippet

Embedding/vector Retrieval

• Introduction to vector retrieval: From sparse representations to Language Models; Dense Passage Retrieval; Sentence Transformers for Dense Retrieval
• Embedding Indexing Pipeline - code snippet
• Embedding RAG Pipeline - code snippet

Hybrid Retrieval

• Retrieve and Re-Rank
• Hybrid Retrieval Tutorial

From Demo to Production

Choose a good Language Model

• LLM Evals and Benchmarking - Great blog post by Omar San Seviero
• Open LLM Leaderboard
• LMSYS Chatbot Arena Leaderboard
• Open ITA LLM Leaderboard

Choose a LLM inference solution

• Docs

Evaluation

• Docs

Deployment

• Serialization of Pipelines
• Hayhooks
• Deployment guides

Beyond RAG...

• Haystack cookbook
• Multilingual RAG from a podcast
• Hacker News summarizer
• Information extraction via LLMs
• AutoQuizzer

There is much more!

• File converters

• Observability: logging and tracing

• Function calling: docs and tutorial

• Conclusion: "The Shift from Models to Compound AI Systems" - blog post