Code

Add LLM Pipeline#137

Code

Merged

rickstaa merged 15 commits intolivepeer:mainlivepeer/ai-runner:mainfrom joinhive-ai:llmjoinhive-ai/ai-worker:llmCopy head branch name to clipboardSep 30, 2024

+1,004-58

Commits

Commits on Sep 27, 2024

• runner: add llm-generate route and pipeline

  kyriediculous

  committed
  409ad51

  Copy full SHA for 409ad51

• add llama3.1 8B to downloads

  kyriediculous

  committed
  6e1ed29

  Copy full SHA for 6e1ed29

• worker: add llm-generate container management

  kyriediculous

  committed
  46f4515

  Copy full SHA for 46f4515

• llm: support streamed responses

  kyriediculous

  committed
  8e6dfc6

  Copy full SHA for 8e6dfc6

• Load LLM model distributed over multiple GPUs

  kyriediculous

  committed
  f5f7d66

  Copy full SHA for f5f7d66

• feat: support 8bit and fp16 for llm pipeline

  kyriediculous

  committed
  5df016b

  Copy full SHA for 5df016b

• rename llm pipeline and route from 'llm-generate' to 'llm'

  kyriediculous

  committed
  d577b74

  Copy full SHA for d577b74

• fix: move check_torch_cuda.py script to check versions to 'dev' folder

  kyriediculous

  committed
  653ff2b

  Copy full SHA for 653ff2b

• fix: return container on non-streaming response

  kyriediculous

  committed
  e488aa4

  Copy full SHA for e488aa4

• fix update naming convention 'Llm' -> 'LLM'

  kyriediculous

  committed
  53f326e

  Copy full SHA for 53f326e

• fix: return container after stream completes

  kyriediculous

  committed
  22fbcf0

  Copy full SHA for 22fbcf0

• update naming conventions for SDK and go bindings

  kyriediculous

  committed
  e02ef21

  Copy full SHA for e02ef21

Commits on Sep 30, 2024

• rename LlmGenerateMultipartWriter -> LLMMultipartWriter

  kyriediculous

  committed
  39e082e

  Copy full SHA for 39e082e

• openapi tags

  kyriediculous

  committed
  620f8b0

  Copy full SHA for 620f8b0

• openapi bindings and spec with improved tags and description

  kyriediculous

  committed
  f7521c3

  Copy full SHA for f7521c3