ELK 7.10.0部署-docker-compose

install & config ELK with docker

设置密码并配置安全ssl

set vm.max_map_count

1. modify sysctl.conf

vim /etc/sysctl.conf 
# add content: vm.max_map_count = 262144 

2. 临时生效

sysctl -w vm.max_map_count=262144

1 pull image

image 版本要和.env文件中的版本一致

cd /home
git clone https://github.com/loululin/elk-docker.git
cd elk-docker
./pullImage.sh 7.10.0
chmod 777 elasticsearch  kibana  logstash

2 get file elastic-stack-ca.p12

1. start temp container

docker run --name es -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.10.0

2. generator elastic-stack-ca.p12

docker exec -it es /bin/bash
./bin/elasticsearch-certutil ca
exit

3. copy elastic-stack-ca.p12 to host dir

cd /home/elk-docker
docker cp es:/usr/share/elasticsearch/elastic-stack-ca.p12 .
chmod 777 elastic-stack-ca.p12

4. remove temp container

docker rm -f es

3 generate passwd

1. start container

docker-compose up -d es01 es02 es03
docker-compose logs -f es01

2. enter container

自动生成密码用auto或自己设置用 interactive

docker exec -it es01 /bin/bash
./bin/elasticsearch-setup-passwords interactive
# 123456

3. restart container

docker-compose restart es01 es02 es03

4. verify

access http://ip:9200/_cat/nodes  with elastic/123456
and you will see:
192.168.16.4 54 28 18 0.19 0.31 0.69 cdhilmrstw - es02
192.168.16.2 65 28 18 0.19 0.31 0.69 cdhilmrstw * es03
192.168.16.3 14 28 18 0.19 0.31 0.69 cdhilmrstw - es01

4 confirm kibana and logstash authentication info

如果密码为纯数字，需要加上双引号

logstash/config/logstash.yml 
logstash/pipeline/logstash.conf 
kibana/config/kibana.yml

5 start kibana

docker-compose up -d kibana
docker-compose logs -f kibana

6 start logstash

docker-compose up -d logstash
docker-compose logs -f logstash

7 restart service

docker-compose restart

8 look services status

[root@localhost elk-docker]# docker-compose ps
  Name                Command               State                                               Ports                                             
--------------------------------------------------------------------------------------------------------------------------------------------------
es01       /tini -- /usr/local/bin/do ...   Up      0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp                                                
es02       /tini -- /usr/local/bin/do ...   Up      9200/tcp, 9300/tcp                                                                            
es03       /tini -- /usr/local/bin/do ...   Up      9200/tcp, 9300/tcp                                                                            
kibana     /usr/local/bin/dumb-init - ...   Up      0.0.0.0:5601->5601/tcp                                                                        
logstash   /usr/local/bin/docker-entr ...   Up      0.0.0.0:5000->5000/tcp, 0.0.0.0:5000->5000/udp, 0.0.0.0:5044->5044/tcp, 0.0.0.0:9600->9600/tcp

9 access kibana

http://host-ip:5601/

10 midify passwd

1. enter es container

docker exec -it es01 /bin/bash

2. create temp superadmin user : ryan

至少6位

./bin/elasticsearch-users useradd ryan -r superuser

3. modify elastic's passwd

curl -XPUT -u ryan:ryan123 http://localhost:9200/_xpack/security/user/elastic/_password -H "Content-Type: application/json" -d '
{
  "password": "q5f2qNfUJQyvZPIz57MZ"
}'

11 curator

定期删除ES索引，比如删除3天前的索引

11.1 build image

cd extensions/curator
docker build -t curator:comm .
docker images

11.2 start curator

docker-compose up -d curator

docker-compose ps

docker exec -it curator /bin/bash

11.3 log

cd extensions/curator/logs/

tail -f run.log

log look like:

2020-12-10 10:58:02,358 INFO      Deleting 5 selected indices: ['aos-webmanage-api-2020.12.02', 'aos-webmanage-api-2020.12.07', 'aos-webmanage-api-2020.12.03', 'aos-webmanage-api-2020.12.04', 'aos-webmanage-api-2020.12.05']
2020-12-10 10:58:02,358 INFO      ---deleting index aos-webmanage-api-2020.12.02
2020-12-10 10:58:02,358 INFO      ---deleting index aos-webmanage-api-2020.12.07
2020-12-10 10:58:02,358 INFO      ---deleting index aos-webmanage-api-2020.12.03
2020-12-10 10:58:02,358 INFO      ---deleting index aos-webmanage-api-2020.12.04
2020-12-10 10:58:02,358 INFO      ---deleting index aos-webmanage-api-2020.12.05
2020-12-10 10:58:02,738 INFO      Action ID: 2, "delete_indices" completed.
2020-12-10 10:58:02,738 INFO      Job completed.
2020-12-10 10:59:02,766 INFO      Preparing Action ID: 1, "delete_indices"

11.4 modify delete_log_files_curator.yml-添加删除策略

no need restart curator container

cd extensions/curator/config

vim delete_log_files_curator.yml

cd extensions/curator/logs/

tail -f run.log

11.5 modify crontab-修改定时执行策略

linux crontab online tool:https://tool.lu/crontab

need restart curator container

crontab for example: 0 1 * * *

cd extensions/curator/cron

vim crontab

cd ../../..

docker-compose restart curator

cd extensions/curator/logs/

tail -f run.log