Skip to content

Latest commit

 

History

History
328 lines (265 loc) · 15 KB

quickstart.asciidoc

File metadata and controls

328 lines (265 loc) · 15 KB

Quickstart

Deploy ECK in your Kubernetes cluster

Things to consider before you start:

  • For this quickstart guide, your Kubernetes cluster is assumed to be already up and running. Before you proceed with the ECK installation, make sure you check the supported versions.

  • If you are using GKE, make sure your user has cluster-admin permissions. For more information, check Prerequisites for using Kubernetes RBAC on GKE.

  • If you are using Amazon EKS, make sure the Kubernetes control plane is allowed to communicate with the Kubernetes nodes on port 443. This is required for communication with the Validating Webhook. For more information, check Recommended inbound traffic.

  • Refer to [{p}-installing-eck] for more information on installation options.

  • Check the upgrade notes if you are attempting to upgrade an existing ECK deployment.

To deploy the ECK operator:

  1. Install custom resource definitions with create:

    kubectl create -f https://download.elastic.co/downloads/eck/{eck_version}/crds.yaml

    This will output similar to the following upon Elastic resources' creation:

    customresourcedefinition.apiextensions.k8s.io/agents.agent.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/apmservers.apm.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/beats.beat.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/elasticmapsservers.maps.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/elasticsearches.elasticsearch.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/enterprisesearches.enterprisesearch.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/kibanas.kibana.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/logstashes.logstash.k8s.elastic.co created

  2. Install the operator with its RBAC rules with apply:

    kubectl apply -f https://download.elastic.co/downloads/eck/{eck_version}/operator.yaml
    Note
    		 The ECK operator runs by default in the elastic-system namespace. It is recommended that you choose a dedicated namespace for your workloads, rather than using the elastic-system or the default namespace.

  3. Monitor the operator’s setup from its logs through logs:

    kubectl -n elastic-system logs -f statefulset.apps/elastic-operator

  4. Once ready, the operator will report as Running as shown with get, replacing default elastic-system with applicable installation namespace as needed: *

$ kubectl get -n elastic-system pods
NAME                 READY   STATUS    RESTARTS   AGE
elastic-operator-0   1/1     Running   0          1m

This completes the quickstart of the ECK operator. We recommend continuing to Deploying an {es} cluster; but for more configuration options as needed, navigate to Operating ECK.

Deploy an {es} cluster

To deploy a simple {es}] cluster specification, with one {es} node:

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/{eck_crd_version}
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: {version}
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
EOF

The operator automatically creates and manages Kubernetes resources to achieve the desired state of the {es} cluster. It may take up to a few minutes until all the resources are created and the cluster is ready for use.

Caution
 Setting node.store.allow_mmap: false has performance implications and should be tuned for production workloads as described in the [{p}-virtual-memory] section.
Note
 If your Kubernetes cluster does not have any Kubernetes nodes with at least 2GiB of free memory, the pod will be stuck in Pending state. Check [{p}-managing-compute-resources] for more information about resource requirements and how to configure them.
Note
 The cluster that you deployed in this quickstart guide only allocates a persistent volume of 1GiB for storage using the default storage class defined for the Kubernetes cluster. You will most likely want to have more control over this for production workloads. Refer to [{p}-volume-claim-templates] for more information.

For a full description of each CustomResourceDefinition (CRD), refer to the [{p}-api-reference] or view the CRD files in the project repository. You can also retrieve information about a CRD from the cluster. For example, describe the {es} CRD specification with describe:

kubectl describe crd elasticsearch

Monitor cluster health and creation progress

Get an overview of the current {es} clusters in the Kubernetes cluster with get, including health, version and number of nodes:

kubectl get elasticsearch

When you first create the Kubernetes cluster, there is no HEALTH status and the PHASE is empty. After the pod and service start-up, the PHASE turns into Ready, and HEALTH becomes green. The HEALTH status comes from {es}'s cluster health API.

NAME          HEALTH    NODES     VERSION   PHASE         AGE
quickstart              1         {version}               1s

While the {es} pod is in the process of being started it will report Pending as checked with get:

kubectl get pods --selector='elasticsearch.k8s.elastic.co/cluster-name=quickstart'

Which will output similar to:

NAME                      READY   STATUS    RESTARTS   AGE
quickstart-es-default-0   0/1     Pending   0          9s

During and after start-up, up that pod’s logs can be accessed:

kubectl logs -f quickstart-es-default-0

Once the pod has finished coming up, our original get request will now report:

NAME          HEALTH    NODES     VERSION   PHASE         AGE
quickstart    green     1         {version}     Ready         1m

Request {es} access

A ClusterIP Service is automatically created for your cluster as checked with get:

kubectl get service quickstart-es-http

Which will output similar to:

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
quickstart-es-http   ClusterIP   10.15.251.145   <none>        9200/TCP   34m

In order to make requests to the {es} API:

  1. Get the credentials.

    By default, a user named elastic is created with the password stored inside a Kubernetes secret. This default user can be disabled if desired, refer to [{p}-users-and-roles] for more information.

    PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')

  2. Request the {es} root API. You can do so from inside the Kubernetes cluster or from your local workstation. For demonstration purposes, certificate verification is disabled using the -k curl flag; however, this is not recommended outside of testing purposes. Refer to [{p}-setting-up-your-own-certificate] for more information.

    • From inside the Kubernetes cluster:

      curl -u "elastic:$PASSWORD" -k "https://quickstart-es-http:9200"

    • From your local workstation:

      1. Use the following command in a separate terminal:

        kubectl port-forward service/quickstart-es-http 9200

      2. Request localhost:

        curl -u "elastic:$PASSWORD" -k "https://localhost:9200"

This completes the quickstart of deploying an {es} cluster. We recommend continuing to Deploy a {kib} instance but for more configuration options as needed, navigate to Running {es} on ECK.

Deploy a {kib} instance

To deploy a simple {kib} specification, with one {kib} instance:

  1. Specify a {kib} instance and associate it with your {es} quickstart cluster created previously under Deploying an {es} cluster:

    cat <<EOF | kubectl apply -f -
apiVersion: kibana.k8s.elastic.co/{eck_crd_version}
kind: Kibana
metadata:
  name: quickstart
spec:
  version: {version}
  count: 1
  elasticsearchRef:
    name: quickstart
EOF

  2. Monitor {kib} health and creation progress.

    Similar to {es}, you can retrieve details about {kib} instances with get:

    kubectl get kibana

    And the associated Pods:

    kubectl get pod --selector='kibana.k8s.elastic.co/name=quickstart'

    {kib} will be status available once get reports green. If it experiences issues starting up, use logs against the pod in order to Troubleshoot {kib} start-up.

  3. Access {kib}.

    A ClusterIP Service is automatically created for {kib}:

    kubectl get service quickstart-kb-http

    Use kubectl port-forward to access {kib} from your local workstation:

    kubectl port-forward service/quickstart-kb-http 5601

    Open https://localhost:5601 in your browser. Your browser will show a warning because the self-signed certificate configured by default is not verified by a known certificate authority and not trusted by your browser. You can temporarily acknowledge the warning for the purposes of this quick start but it is highly recommended that you configure valid certificates for any production deployments.

    Login as the elastic user. The password can be obtained with the following command:

    kubectl get secret quickstart-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode; echo

For a full description of each CustomResourceDefinition (CRD), refer to the [{p}-api-reference] or view the CRD files in the project repository. You can also retrieve information about a CRD from the instance. For example, describe the {kib} CRD specification with describe:

kubectl describe crd kibana

This completes the quickstart of deploying an {kib} instance on top of the ECK operator and deployed {es} cluster. We recommend continuing to updating your deployment. For more {kib} configuration options, refer to Running {kib} on ECK.

Update your deployment

You can add and modify most elements of the original Kubernetes cluster specification provided that they translate to valid transformations of the underlying Kubernetes resources (for example existing volume claims cannot be downsized). The ECK operator will attempt to apply your changes with minimal disruption to the existing cluster. You should ensure that the Kubernetes cluster has sufficient resources to accommodate the changes (extra storage space, sufficient memory and CPU resources to temporarily spin up new pods, and so on).

For example, you can grow the cluster to three {es} nodes from the deployed {es} cluster example by updating the count with apply:

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/{eck_crd_version}
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: {version}
  nodeSets:
  - name: default
    count: 3
    config:
      node.store.allow_mmap: false
EOF

ECK will automatically schedule the requested update. Changes can be monitored with the ECK operator logs, events, and applicable product’s pod logs. These will either report successful application of changes or provide context for further troubleshooting. Kindly note, Kubernetes restricts some changes, for example refer to Updating Volume Claims.

This completes our quickstart guide for deploying an {es} cluster and {kib} instance with our ECK operator. We recommend continuing to Orchestrating Elastic Stack applications for more configuration options