feat: improve embeddings, models and connectivity

[augchan42]

• Add multiple embedding options:
  • BGE-Small local embeddings (~70-80% of OpenAI quality)
  • Ollama embeddings support
  • Configure via env vars:
    • USE_OPENAI_EMBEDDING=true
    • USE_OLLAMA_EMBEDDING=true
    • Both case-insensitive, default to false
• Switch to Claude 3 Haiku from Claude 3.5 Haiku
  • 4x cheaper
  • Better instruction following
  • Improved price/performance ratio
• Enhance Postgres connectivity and reliability
• Fix transcription error handling

Breaking changes:

• Claude 3.5 Haiku replaced with Claude 3 Haiku
• Embedding system now configurable via string env vars (true)

Closes #604

Discord username

hosermage

[augchan42]

Check the .env.example file for this section:

# Feature Flags
IMAGE_GEN=            # Set to TRUE to enable image generation
USE_OPENAI_EMBEDDING= # Set to TRUE for OpenAI/1536, leave blank for local
USE_OLLAMA_EMBEDDING= # Set to TRUE for OLLAMA/1024, leave blank for local

If using OLLAMA, then set USE_OLLAMA_EMBEDDING=true and USE_OPENAI_EMBEDDING blank

If you want to use BGE/384 (local embeddings using BAAI General Embeddings small model)
Then leave both USE_OLLAMA_EMBEDDING and USE_OPENAI_EMBEDDING blank.

These embeddings are for persisting and searching the memories table and are separate and distinct from LLM inference, and can be mixed and matched (I use Anthropic for inference, BGE for embeddings)