Skip to content

Latest commit

 

History

History
 
 

logging

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 

loki

https://github.com/grafana/loki

EFK stack

Elasticsearch index templates

See index templates for details. It appears that it is necessary to manually create index templatess for elasticsearch in order to apply deletion policies.

This is what I have configured for elasticsearch:

New delete-after-30-days ILM policy

PUT _ilm/policy/delete-after-30d
{
  "delete-after-30d" : {
    "policy" : {
      "phases" : {
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "delete" : {
          "min_age" : "30d",
          "actions" : {
            "delete" : { }
          }
        }
      }
    }
  }
}

Index patterns

for logstash-*:

PUT _template/logstash
{
  "index_patterns": ["logstash-*"]
}

for fluentd-syslog-*:

PUT _template/fluentd-syslog
{
  "index_patterns": ["fluentd-syslog-*"]
}

... After creating the index patterns, it is necessary to 'apply' the index patterns to the newly-created ILM deletion policy so that all new indexes created (with the above patterns) will have the policy automatically associated.

remote syslog logging

  • fluentd.yaml - fluentd (deployed as fluentd-syslog) for listening as a syslog server on UDP/5140 for collecting all syslog messages from all hosts external from the cluster

    • Configure the remote rsyslog daemon with something like,
    echo "*.* @10.2.0.104:5140" > /etc/rsyslog.d/10-fluentd.conf && service rsyslog restart