Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loki 2.6.1 not starting (failed to create memberlist) #52

Open
flouis1 opened this issue Mar 23, 2023 · 1 comment
Open

Loki 2.6.1 not starting (failed to create memberlist) #52

flouis1 opened this issue Mar 23, 2023 · 1 comment

Comments

@flouis1
Copy link

flouis1 commented Mar 23, 2023

Trying to run Loki through the current version of Helm deployment. Startup of the loki pod is failing with the error message:

log

msg=“module failed” module=memberlist-kv error=“invalid service state: Stopping, expected: Running”

Same issue seen here

Here I assume that your current namespace is jfrog-plg
if not

kc config set-context --current --namespace=jfrog-plg

Pull config

hl get values loki  -o yaml > values.loki.yaml

Fix by adding those line to value.yml

loki:
[...]
  config:
    memberlist:
      bind_addr:
        - ${MY_POD_IP}

  ingester:
    extraArgs:
    - -config.expand-env=true
    extraEnv:
      - name: MY_POD_IP
        valueFrom:
          fieldRef:
            fieldPath: status.podIP

Update install

hl upgrade --install "loki" grafana/loki-stack -f values.loki.yaml
@flouis1 flouis1 changed the title Loki 2.6.1 not starting () Loki 2.6.1 not starting (failed to create memberlist) Mar 23, 2023
@tianmiaohu-fis
Copy link

tianmiaohu-fis commented Mar 29, 2024

For loki 2.9.6, under singleBinary node, the settings (structuredConfig under loki and extraArgs and extraEnv under singleBinary) worked for me.

  loki:
    auth_enabled: false
    podSecurityContext:
      fsGroup: null
      runAsGroup: null
      runAsNonRoot: true
      runAsUser: null
    commonConfig:
      replication_factor: 1
    storage:
      type: "filesystem"
    limits_config:
      retention_period: 168h
    compactor:
      working_directory: /var/loki/retention
      compaction_interval: 60m
      retention_enabled: true
      retention_delete_delay: 2h
      retention_delete_worker_count: 150
    image:
      registry: donelfi-docker-dev.docker.fis.dev
      repository: grafana/loki
      tag: 2.9.6
    structuredConfig:
      memberlist:
        bind_addr:
          - ${MY_POD_IP}
  singleBinary:
    replicas: 1
    persistence:
      size: 50Gi
      storageClass: mcp-azure-file-nfs
    resources:
      limits:
        cpu: 4000m
        memory: 4000Mi
      requests:
        cpu: 2000m
        memory: 4000Mi
    extraArgs:
      - -config.expand-env=true
    extraEnv:
      - name: MY_POD_IP
        valueFrom:
          fieldRef:
            fieldPath: status.podIP

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants