-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Administrator
committed
Dec 22, 2018
1 parent
bc9b70e
commit 7f9e091
Showing
7 changed files
with
469 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
# docker-elasticsearch-kubernetes | ||
|
||
Ready to use, lean Elasticsearch Docker image ready for using within a Kubernetes cluster. | ||
|
||
|
||
## Current software | ||
|
||
* Alpine Linux 3.8 | ||
* IcedTea JRE 8u171 | ||
* Elasticsearch 6.4.2 | ||
|
||
**Note:** `x-pack-ml` module is forcibly disabled as it's not supported on Alpine Linux. | ||
|
||
## Run | ||
|
||
See [kubernetes-elasticsearch-cluster](https://github.com/vekatkriish/elasticsearchonk8s) for instructions on how to run, scale and use Elasticsearch on Kubernetes. | ||
|
||
## Environment variables | ||
|
||
This image can be configured by means of environment variables, that one can set on a `Deployment`. | ||
|
||
|
||
* [DISCOVERY_SERVICE](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#unicast) - the service to be queried for through DNS (default = `elasticsearch-discovery`). | ||
* [MEMORY_LOCK](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#bootstrap.memory_lock) - memory locking control defaults to `false` as Kubernetes requires swap to be disabled. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
FROM dockerkriish/docker-jre:8u171_alpine_3.8.1 | ||
MAINTAINER [email protected] | ||
|
||
# Export HTTP & Transport | ||
EXPOSE 9200 9300 | ||
|
||
ENV ES_VERSION 6.4.2 | ||
|
||
ENV DOWNLOAD_URL "https://artifacts.elastic.co/downloads/elasticsearch" | ||
ENV ES_TARBAL "${DOWNLOAD_URL}/elasticsearch-${ES_VERSION}.tar.gz" | ||
ENV ES_TARBALL_ASC "${DOWNLOAD_URL}/elasticsearch-${ES_VERSION}.tar.gz.asc" | ||
ENV GPG_KEY "46095ACC8548582C1A2699A9D27D666CD88E42B4" | ||
|
||
# Install Elasticsearch. | ||
RUN apk add --no-cache --update bash ca-certificates su-exec util-linux curl | ||
RUN apk add --no-cache -t .build-deps gnupg openssl \ | ||
&& cd /tmp \ | ||
&& echo "===> Install Elasticsearch..." \ | ||
&& curl -o elasticsearch.tar.gz -Lskj "$ES_TARBAL"; \ | ||
if [ "$ES_TARBALL_ASC" ]; then \ | ||
curl -o elasticsearch.tar.gz.asc -Lskj "$ES_TARBALL_ASC"; \ | ||
export GNUPGHOME="$(mktemp -d)"; \ | ||
gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$GPG_KEY"; \ | ||
gpg --batch --verify elasticsearch.tar.gz.asc elasticsearch.tar.gz; \ | ||
rm -r "$GNUPGHOME" elasticsearch.tar.gz.asc; \ | ||
fi; \ | ||
tar -xf elasticsearch.tar.gz \ | ||
&& ls -lah \ | ||
&& mv elasticsearch-$ES_VERSION /elasticsearch \ | ||
&& adduser -DH -s /sbin/nologin elasticsearch \ | ||
&& mkdir -p /elasticsearch/config/scripts /elasticsearch/plugins \ | ||
&& chown -R elasticsearch:elasticsearch /elasticsearch \ | ||
&& rm -rf /tmp/* \ | ||
&& apk del --purge .build-deps | ||
|
||
ENV PATH /elasticsearch/bin:$PATH | ||
|
||
WORKDIR /elasticsearch | ||
|
||
# Copy configuration | ||
COPY config /elasticsearch/config | ||
|
||
# Copy run script | ||
COPY run.sh / | ||
|
||
# Set environment variables defaults | ||
ENV ES_JAVA_OPTS "-Xms512m -Xmx512m" | ||
ENV CLUSTER_NAME elasticsearch-default | ||
ENV NODE_MASTER true | ||
ENV NODE_DATA true | ||
ENV NODE_INGEST true | ||
ENV HTTP_ENABLE true | ||
ENV NETWORK_HOST _site_ | ||
ENV HTTP_CORS_ENABLE true | ||
ENV HTTP_CORS_ALLOW_ORIGIN * | ||
ENV NUMBER_OF_MASTERS 1 | ||
ENV MAX_LOCAL_STORAGE_NODES 1 | ||
ENV SHARD_ALLOCATION_AWARENESS "" | ||
ENV SHARD_ALLOCATION_AWARENESS_ATTR "" | ||
ENV MEMORY_LOCK true | ||
ENV REPO_LOCATIONS "" | ||
|
||
# Volume for Elasticsearch data | ||
VOLUME ["/data"] | ||
|
||
CMD ["/run.sh"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,148 @@ | ||
## Current software | ||
|
||
* Alpine Linux 3.8 | ||
* OpenJDK JRE 8u171 | ||
* Elasticsearch 6.4.2 | ||
|
||
**Note:** `x-pack-ml` module is forcibly disabled as it's not supported on Alpine Linux. | ||
|
||
## Run | ||
|
||
### Attention | ||
|
||
* In order for `bootstrap.mlockall` to work, `ulimit` must be allowed to run in the container. Run with `--privileged` to enable this. | ||
* [Multicast discovery is no longer built-in](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/breaking_20_removed_features.html#_multicast_discovery_is_now_a_plugin) | ||
|
||
Ready to use node for cluster `elasticsearch-default`: | ||
``` | ||
docker run --name elasticsearch \ | ||
--detach \ | ||
--privileged \ | ||
--volume /path/to/data_folder:/data \ | ||
dockerkriish/elasticsearchk8s:6.4.2 | ||
``` | ||
|
||
Ready to use node for cluster `myclustername`: | ||
``` | ||
docker run --name elasticsearch \ | ||
--detach \ | ||
--privileged \ | ||
--volume /path/to/data_folder:/data \ | ||
-e CLUSTER_NAME=myclustername \ | ||
dockerkriish/elasticsearchk8s:6.4.2 | ||
``` | ||
|
||
Ready to use node for cluster `elasticsearch-default`, with 8GB heap allocated to Elasticsearch: | ||
``` | ||
docker run --name elasticsearch \ | ||
--detach \ | ||
--privileged \ | ||
--volume /path/to/data_folder:/data \ | ||
-e ES_JAVA_OPTS="-Xms8g -Xmx8g" \ | ||
dockerkriish/elasticsearchk8s:6.4.2 | ||
``` | ||
|
||
Ready to use node with plugins (x-pack and repository-gcs) pre installed. Already installed plugins are ignored: | ||
``` | ||
docker run --name elasticsearch \ | ||
--detach \ | ||
--privileged \ | ||
--volume /path/to/data_folder:/data \ | ||
-e ES_JAVA_OPTS="-Xms8g -Xmx8g" \ | ||
-e ES_PLUGINS_INSTALL="repository-gcs,x-pack" \ | ||
dockerkriish/elasticsearchk8s:6.4.2 | ||
``` | ||
|
||
**Master-only** node for cluster `elasticsearch-default`: | ||
``` | ||
docker run --name elasticsearch \ | ||
--detach \ | ||
--privileged \ | ||
--volume /path/to/data_folder:/data \ | ||
-e NODE_DATA=false \ | ||
-e HTTP_ENABLE=false \ | ||
dockerkriish/elasticsearchk8s:6.4.2 | ||
``` | ||
|
||
**Data-only** node for cluster `elasticsearch-default`: | ||
``` | ||
docker run --name elasticsearch \ | ||
--detach --volume /path/to/data_folder:/data \ | ||
--privileged \ | ||
-e NODE_MASTER=false \ | ||
-e HTTP_ENABLE=false \ | ||
dockerkriish/elasticsearchk8s:6.4.2 | ||
``` | ||
|
||
**Data-only** node for cluster `elasticsearch-default` with shard allocation awareness: | ||
``` | ||
docker run --name elasticsearch \ | ||
--detach --volume /path/to/data_folder:/data \ | ||
--volume /etc/hostname:/dockerhost \ | ||
--privileged \ | ||
-e NODE_MASTER=false \ | ||
-e HTTP_ENABLE=false \ | ||
-e SHARD_ALLOCATION_AWARENESS=dockerhostname \ | ||
-e SHARD_ALLOCATION_AWARENESS_ATTR="/dockerhost" \ | ||
dockerkriish/elasticsearchk8s:6.4.2 | ||
``` | ||
|
||
**Client-only** node for cluster `elasticsearch-default`: | ||
``` | ||
docker run --name elasticsearch \ | ||
--detach \ | ||
--privileged \ | ||
--volume /path/to/data_folder:/data \ | ||
-e NODE_MASTER=false \ | ||
-e NODE_DATA=false \ | ||
dockerkriish/elasticsearchk8s:6.4.2 | ||
``` | ||
I also make available special images and instructions for [AWS EC2](https://github.com/pires/docker-elasticsearch-aws) and [Kubernetes](https://github.com/pires/docker-elasticsearch-kubernetes). | ||
|
||
### Environment variables | ||
|
||
This image can be configured by means of environment variables, that one can set on a `Deployment`. | ||
|
||
* [CLUSTER_NAME](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#cluster.name) | ||
* [NODE_NAME](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#node.name) | ||
* [NODE_MASTER](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#master-node) | ||
* [NODE_DATA](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#data-node) | ||
* [NETWORK_HOST](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#network-interface-values) | ||
* [HTTP_ENABLE](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html#_settings_2) | ||
* [HTTP_CORS_ENABLE](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html#_settings_2) | ||
* [HTTP_CORS_ALLOW_ORIGIN](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html#_settings_2) | ||
* [NUMBER_OF_MASTERS](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#master-election) | ||
* [MAX_LOCAL_STORAGE_NODES](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#max-local-storage-nodes) | ||
* [ES_JAVA_OPTS](https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html) | ||
* [ES_PLUGINS_INSTALL](https://www.elastic.co/guide/en/elasticsearch/plugins/current/installation.html) - comma separated list of Elasticsearch plugins to be installed. Example: `ES_PLUGINS_INSTALL="repository-gcs,x-pack"` | ||
* [SHARD_ALLOCATION_AWARENESS](https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-awareness.html#CO287-1) | ||
* [SHARD_ALLOCATION_AWARENESS_ATTR](https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-awareness.html#CO287-1) | ||
* [MEMORY_LOCK](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#bootstrap.memory_lock) - memory locking control - enable to prevent swap (default = `true`) . | ||
* [REPO_LOCATIONS](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html#_shared_file_system_repository) - list of registered repository locations. For example `"/backup"` (default = `[]`). The value of REPO_LOCATIONS is automatically wrapped within an `[]` and therefore should not be included in the variable declaration. To specify multiple repository locations simply specify a comma separated string for example `"/backup", "/backup2"`. | ||
* [PROCESSORS](https://github.com/elastic/elasticsearch-definitive-guide/pull/679/files) - allow elasticsearch to optimize for the actual number of available cpus (must be an integer - default = 1) | ||
|
||
### Backup | ||
Mount a shared folder (for example via NFS) to `/backup` and make sure the `elasticsearch` user | ||
has write access. Then, set the `REPO_LOCATIONS` environment variable to `"/backup"` and create | ||
a backup repository: | ||
|
||
`backup_repository.json`: | ||
``` | ||
{ | ||
"type": "fs", | ||
"settings": { | ||
"location": "/backup", | ||
"compress": true | ||
} | ||
} | ||
``` | ||
|
||
```bash | ||
curl -XPOST http://<container_ip>:9200/_snapshot/nas_repository -d @backup_repository.json` | ||
``` | ||
|
||
Now, you can take snapshots using: | ||
```bash | ||
curl -f -XPUT "http://<container_ip>:9200/_snapshot/nas_repository/snapshot_`date --utc +%Y_%m_%dt%H_%M`?wait_for_completion=true" | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
cluster: | ||
name: ${CLUSTER_NAME} | ||
|
||
node: | ||
master: ${NODE_MASTER} | ||
data: ${NODE_DATA} | ||
name: ${NODE_NAME} | ||
ingest: ${NODE_INGEST} | ||
max_local_storage_nodes: ${MAX_LOCAL_STORAGE_NODES} | ||
|
||
processors: ${PROCESSORS:1} | ||
|
||
network.host: ${NETWORK_HOST} | ||
|
||
path: | ||
data: /data/data | ||
logs: /data/log | ||
repo: ${REPO_LOCATIONS} | ||
|
||
bootstrap: | ||
memory_lock: ${MEMORY_LOCK} | ||
|
||
http: | ||
enabled: ${HTTP_ENABLE} | ||
compression: true | ||
cors: | ||
enabled: ${HTTP_CORS_ENABLE} | ||
allow-origin: ${HTTP_CORS_ALLOW_ORIGIN} | ||
|
||
discovery: | ||
zen: | ||
ping.unicast.hosts: ${DISCOVERY_SERVICE} | ||
minimum_master_nodes: ${NUMBER_OF_MASTERS} | ||
|
||
xpack.ml.enabled: false |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
## JVM configuration | ||
|
||
################################################################ | ||
## IMPORTANT: JVM heap size | ||
################################################################ | ||
## | ||
## You should always set the min and max JVM heap | ||
## size to the same value. For example, to set | ||
## the heap to 4 GB, set: | ||
## | ||
## -Xms4g | ||
## -Xmx4g | ||
## | ||
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html | ||
## for more information | ||
## | ||
################################################################ | ||
|
||
# Xms represents the initial size of total heap space | ||
# Xmx represents the maximum size of total heap space | ||
|
||
# Commented out since these are supplied as env vars in the command-line | ||
# for the service. | ||
|
||
#-Xms2g | ||
#-Xmx2g | ||
|
||
################################################################ | ||
## Expert settings | ||
################################################################ | ||
## | ||
## All settings below this section are considered | ||
## expert settings. Don't tamper with them unless | ||
## you understand what you are doing | ||
## | ||
################################################################ | ||
|
||
## GC configuration | ||
-XX:+UseConcMarkSweepGC | ||
-XX:CMSInitiatingOccupancyFraction=75 | ||
-XX:+UseCMSInitiatingOccupancyOnly | ||
|
||
## optimizations | ||
|
||
# disable calls to System#gc | ||
-XX:+DisableExplicitGC | ||
|
||
# pre-touch memory pages used by the JVM during initialization | ||
-XX:+AlwaysPreTouch | ||
|
||
## basic | ||
|
||
# force the server VM (remove on 32-bit client JVMs) | ||
-server | ||
|
||
# explicitly set the stack size (reduce to 320k on 32-bit client JVMs) | ||
-Xss1m | ||
|
||
# set to headless, just in case | ||
-Djava.awt.headless=true | ||
|
||
# ensure UTF-8 encoding by default (e.g. filenames) | ||
-Dfile.encoding=UTF-8 | ||
|
||
# use our provided JNA always versus the system one | ||
-Djna.nosys=true | ||
|
||
# use old-style file permissions on JDK9 | ||
-Djdk.io.permissionsUseCanonicalPath=true | ||
|
||
# flags to keep Netty from being unsafe | ||
-Dio.netty.noUnsafe=true | ||
-Dio.netty.noKeySetOptimization=true | ||
|
||
# log4j 2 | ||
-Dlog4j.shutdownHookEnabled=false | ||
-Dlog4j2.disable.jmx=true | ||
-Dlog4j.skipJansi=true | ||
|
||
## heap dumps | ||
|
||
# generate a heap dump when an allocation from the Java heap fails | ||
# heap dumps are created in the working directory of the JVM | ||
-XX:+HeapDumpOnOutOfMemoryError | ||
|
||
# specify an alternative path for heap dumps | ||
# ensure the directory exists and has sufficient space | ||
#-XX:HeapDumpPath=${heap.dump.path} | ||
|
||
## GC logging | ||
|
||
#-XX:+PrintGCDetails | ||
#-XX:+PrintGCTimeStamps | ||
#-XX:+PrintGCDateStamps | ||
#-XX:+PrintClassHistogram | ||
#-XX:+PrintTenuringDistribution | ||
#-XX:+PrintGCApplicationStoppedTime | ||
|
||
# log GC status to a file with time stamps | ||
# ensure the directory exists | ||
#-Xloggc:${loggc} | ||
|
||
# Elasticsearch 5.0.0 will throw an exception on unquoted field names in JSON. | ||
# If documents were already indexed with unquoted fields in a previous version | ||
# of Elasticsearch, some operations may throw errors. | ||
# | ||
# WARNING: This option will be removed in Elasticsearch 6.0.0 and is provided | ||
# only for migration purposes. | ||
#-Delasticsearch.json.allow_unquoted_field_names=true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
|
||
status = error | ||
|
||
appender.console.type = Console | ||
appender.console.name = console | ||
appender.console.layout.type = PatternLayout | ||
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] %marker%m%n | ||
|
||
rootLogger.level = info | ||
rootLogger.appenderRef.console.ref = console |
Oops, something went wrong.