Issue when PVC is mounted to weaviate container

[shrijit-narkar]

Description

Installation

• Docker installation on kubernetes pod

Weaviate Deployment

• Docker Deployment on kubernetes pod

Details

Have installed Verba and Weaviate containers on a single pod in kubernetes. When I set PERSISTENCE_DATA_PATH to '/var/lib/weaviate' and mount a pvc on that path, Verba is unable to connect to Weaviate. (I am selecting Docker instance to connect to from Verba UI).
When I don't mount a pvc to the weaviate container, everything is working fine. But when I am mounting a PVC, I am getting below error:
Failed to connect to Weaviate object existence! Unexpected status code: 500, with response body: None.

When I checked the weaviate container logs, I can find below,

{"build_git_commit":"64457c2","build_go_version":"go1.22.9","build_image_tag":"v1.27.2","build_wv_version":"1.27.2","error":"init shard "verba_config_8tB2n31NCSSP": init shard "verba_config_8tB2n31NCSSP": init prop "config": value index: init disk segments: init segment segment-1731316996864028994.db**: unexpected error loading segment** "/var/lib/weaviate/verba_config/8tB2n31NCSSP/lsm/property_config/segment-1731316996864028994.db": runtime error: slice bounds out of range [:419430404] with capacity 12748","level":"error","msg":"Unable to load shard 8tB2n31NCSSP: init shard "verba_config_8tB2n31NCSSP": init shard "verba_config_8tB2n31NCSSP": init prop "config": value index: init disk segments: init segment segment-1731316996864028994.db: unexpected error loading segment "/var/lib/weaviate/verba_config/8tB2n31NCSSP/lsm/property_config/segment-1731316996864028994.db": runtime error: slice bounds out of range [:419430404] with capacity 12748","time":"2024-11-12T09:34:21Z"}
{"action":"requests_total","api":"rest","build_git_commit":"64457c2","build_go_version":"go1.22.9","build_image_tag":"v1.27.2","build_wv_version":"1.27.2","class_name":"VERBA_CONFIG","error":"msg:repo.exists code:500 err:get/init local shard "8tB2n31NCSSP", no shutdown: LazyLoadShard::preventShutdown: Unable
to load shard 8tB2n31NCSSP: init shard "verba_config_8tB2n31NCSSP": init shard "verba_config_8tB2n31NCSSP": init prop "config": value index: init disk segments: init segment segment-1731316996864028994.db: unexpected error loading segment "/var/lib/weaviate/verba_config/8tB2n31NCSSP/lsm/property_config/segment-1731316996864028994.db": runtime error: slice bounds out of range [:419430404] with capacity 12748","level":"error","msg":"unexpected error","query_type":"objects","time":"2024-11-12T09:34:21Z"}

[thomashacker]

Hey, thanks for the issue! Can you share a bit more info about your current setup? Are you deploying the Weaviate instance separately through docker or are you deploying the whole verba application with a weaviate instance inside one container?