[v0.0.7] Unable to get started #24
Comments
|
@Moep90 are you still seeing this issue ? What is the per process thread limit on your VM ? |
|
@vignesh-makerdojo Yeah its still the same error on the VM. $ cat /proc/sys/kernel/threads-max
2061725
$ ulimit -a
real-time non-blocking time (microseconds, -R) unlimited
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 1030862
max locked memory (kbytes, -l) 33002112
max memory size (kbytes, -m) unlimited
open files (-n) 8192
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 1030862
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
|
|
@Moep90 did you try setting |
|
@ajhai higher values then 1 cause the following: |
|
Reading other similar issues around, it could be related to the docker version you are running. For example, some reported in DDMAL/Rodan#1032 that updating docker version fixed similar issue for them. Give it a shot maybe? Edit: HumanSignal/label-studio#3070 (comment) error looks similar to yours |
|
Other suggestion from the linked thread (HumanSignal/label-studio#3070 (comment)) is to add seccomp to docker-compose file. I think one of them should work. |
|
@ajhai good catch! I can confirm to have it working by now on the Server VM. .env cat .env
# Change the secrets below before running LLMStack
SECRET_KEY='3!4^11rgx0-53)!n#18(1_^)&pj-3n^Afpc#mbm(+!fj4r$rp7ea!s'
CIPHER_KEY_SALT=salt
DATABASE_PASSWORD=llmstack
# Update the location of the persistent volumes to non-temporary locations
POSTGRES_VOLUME=./data/postgres_llmstack
REDIS_VOLUME=./data/redis_llmstack
WEAVIATE_VOLUME=./data/weaviate_llmstack
# LLMStack port
LLMSTACK_PORT=3000
# Platform default keys (optional)
DEFAULT_OPENAI_API_KEY=
DEFAULT_DREAMSTUDIO_API_KEY=
DEFAULT_AZURE_OPENAI_API_KEY=
DEFAULT_COHERE_API_KEY=
DEFAULT_FOREFRONTAI_API_KEY=
DEFAULT_ELEVENLABS_APiI_KEY=
OPENBLAS_NUM_THREADS=<number of cores>
ALLOWED_HOSTS="127.0.0.1,localhost,<server IP>"
PRELOAD_MODELS=[{"url": "github:go-skynet/model-gallery/gpt4all-j.yaml", "name": "gpt-3.5-turbo"}, { "url": "github:go-skynet/model-gallery/bert-embeddings.yaml", "name": "text-embedding-ada-002"}]docker-compose.ymlversion: "3.8"
services:
api:
image: ${REGISTRY:-ghcr.io/trypromptly/}llmstack-api:latest
build:
context: .
cache_from:
- llmstack-api:latest
command: apiserver
links:
- postgres:postgres
expose:
- 9000
env_file:
- .env
cap_add:
- SYS_PTRACE
security_opt:
- seccomp:unconfined
rqworker:
image: ${REGISTRY:-ghcr.io/trypromptly/}llmstack-api:latest
build:
context: .
cache_from:
- llmstack-rqworker:latest
command: rqworker
depends_on:
- redis
- postgres
links:
- redis:redis
- postgres:postgres
env_file:
- .env
security_opt:
- seccomp:unconfined
nginx:
image: ${REGISTRY:-ghcr.io/trypromptly/}llmstack-nginx:latest
build:
context: .
dockerfile: Dockerfile.nginx
cache_from:
- llmstack-nginx:latest
args:
- REGISTRY=${REGISTRY:-ghcr.io/trypromptly/}
ports:
- ${LLMSTACK_PORT:-3000}:80
env_file:
- .env
depends_on:
- api
playwright:
image: ${REGISTRY:-ghcr.io/trypromptly/}llmstack-playwright:latest
build:
context: playwright
dockerfile: Dockerfile
cache_from:
- llmstack-playwright:latest
command: npx --yes playwright launch-server --browser chromium --config /config.json
expose:
- 30000
ipc: host
user: pwuser
security_opt:
- seccomp:playwright/seccomp_profile.json
redis:
image: redis:alpine
command: redis-server
restart: unless-stopped
volumes:
- ${REDIS_VOLUME}:/data
env_file:
- .env
postgres:
image: postgres:15.1-alpine
command: "postgres -c fsync=off -c full_page_writes=off -c synchronous_commit=OFF"
restart: unless-stopped
volumes:
- ${POSTGRES_VOLUME}:/var/lib/postgresql/data/pgdata
environment:
POSTGRES_HOST_AUTH_METHOD: "password"
PGDATA: /var/lib/postgresql/data/pgdata
POSTGRES_USER: ${DATABASE_USERNAME:-llmstack}
POSTGRES_PASSWORD: ${DATABASE_PASSWORD:-llmstack}
env_file:
- .env
weaviate:
image: semitechnologies/weaviate:1.20.5
volumes:
- ${WEAVIATE_VOLUME}:/var/lib/weaviate
environment:
QUERY_DEFAULTS_LIMIT: 20
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: "true"
PERSISTENCE_DATA_PATH: "/var/lib/weaviate"
DEFAULT_VECTORIZER_MODULE: text2vec-openai
ENABLE_MODULES: text2vec-openai
CLUSTER_HOSTNAME: "weaviate-node"
env_file:
- .env
local-api:
image: quay.io/go-skynet/local-ai:v1.25.0
container_name: local-ai.example.com
ports:
- 8080
env_file:
- .env
environment:
- DEBUG=true
- MODELS_PATH=/models
- THREADS=10
- CONTEXT_SIZE=1024
volumes:
- ./models:/models:cached
command: ["/usr/bin/local-ai"]
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
I encounterer several errors when starting the v.0.0.7 initially without changing anything besides the paths of my docker-compose volumes.
NOTE
I assume this might cause the issue. I ran this on a server VM which may or may not have all devices required for this tool to run.
To Reproduce
.envfileExpected behavior
LLM Stack should start without having errors.
Version
v0.0.7
Environment
PRETTY_NAME="Ubuntu 22.04.3 LTS"
Docker version 20.10.7, build f0df350
Docker Compose version v2.20.3
Screenshots
Additional context
I used
./dataso I could easiely reset my setup withrm -rf ./dataI had to add
OPENBLAS_NUM_THREADS=1as I encounteredAfter that, it seams to get further but dies at:
The text was updated successfully, but these errors were encountered: