Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Control Plane Logging as a default in all patterns #129

Merged
merged 6 commits into from
Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/logs.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ Then you can view the results of your query:

![logs-results](./patterns/images/mixed-log-results.png)

You can also find control plane logs in CloudWatch with Control Plane logs visible under the `/cluster` log group:

![control-plane-logs](./patterns/images/control-plane-logs.png)

You can then filter by patterns within the logs to find specific events like so:
![control-plane-insights](./patterns/images/control-plane-insights.png)

## Viewing Logs in Grafana

### Using CloudWatch Logs as data source in Grafana
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Objective

This pattern aims to add Observability on top of an existing EKS cluster and adds API server monitoring, with open source managed AWS services.
This pattern aims to add Observability and Control plane logging on top of an existing EKS cluster and adds API server monitoring, with open source managed AWS services.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ This example makes use of CloudWatch, as a metric and log aggregation layer. In

## Objective

This pattern aims to add Observability on top of an existing EKS cluster, with AWS services.
This pattern aims to add Observability and control plane logging on top of an existing EKS cluster, with AWS services.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Utilizing CloudWatch and X-Ray as an aggregation layer allows for a fully-manage

## Objective

This pattern aims to add Observability on top of an existing EKS cluster, with a mixture of AWS native and open source managed AWS services.
This pattern aims to add Observability and control plane logging on top of an existing EKS cluster, with a mixture of AWS native and open source managed AWS services.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The current example deploys the AWS Distro for OpenTelemetry Operator for Amazon

## Objective

This pattern aims to add Observability on top of an existing EKS cluster and NGINX workloads, with open source managed AWS services.
This pattern aims to add Observability and control plane logging on top of an existing EKS cluster and NGINX workloads, with open source managed AWS services.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ please read the [service documentation](https://docs.aws.amazon.com/eks/latest/u
## Objective

Configure the existing Amazon EKS cluster with below Observability components;
- Control Plane logging
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
- AWS Distro For OpenTelemetry Operator and Collector for Metrics and Traces
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit)
- Installs Grafana Operator to add AWS data sources and create Grafana Dashboards to Amazon Managed Grafana.
Expand Down
Binary file added docs/patterns/images/control-plane-insights.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/patterns/images/control-plane-logs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The following figure illustrates the architecture of the pattern we will be depl

## Objective

1. Deploying two production grade Amazon EKS cluster across two AWS Accounts (Prod1, Prod2 account) in two different regions through a Continuous Deployment infrastructure pipeline triggered upon a commit to the repository that holds the pipeline configuration in another AWS account (pipeline account).
1. Deploying two production grade Amazon EKS cluster with control plane logging across two AWS Accounts (Prod1, Prod2 account) in two different regions through a Continuous Deployment infrastructure pipeline triggered upon a commit to the repository that holds the pipeline configuration in another AWS account (pipeline account).
2. Deploying ADOT add-on, AMP add-on to Prod 1 Amazon EKS Cluster to remote-write metrics to AMP workspace in Prod 1 AWS Account.
3. Deploying ADOT add-on, CloudWatch add-on to Prod 2 Amazon EKS Cluster to write metrics to CloudWatch in Prod 2 AWS Account.
4. Configuring GitOps tooling (Argo CD add-on) to support deployment of [ho11y](https://github.com/aws-observability/aws-o11y-recipes/tree/main/sandbox/ho11y) and [yelb](https://github.com/mreferre/yelb) sample applications, in a way that restricts each application to be deployed only into the team namespace, by using Argo CD projects.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Objective

This pattern demonstrates how to use the _New EKS Cluster Open Source Observability Accelerator_ with API Server monitoring.
This pattern demonstrates how to use the _New EKS Cluster Open Source Observability Accelerator_ with API Server monitoring and control plane logging.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ please read the [service documentation](https://docs.aws.amazon.com/eks/latest/u

## Objective

- Deploys one production grade Amazon EKS Fargate cluster.
- Deploys one production grade Amazon EKS Fargate cluster with control plane logging.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
- Logs with CloudWatch Logs
- Enables CloudWatch Container Insights.
- Installs Prometheus Node Exporter and Metrics Server for infrastructure metrics.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ please read the [service documentation](https://docs.aws.amazon.com/eks/latest/u

## Objective

- Deploys one production grade Amazon EKS cluster.
- Deploys one production grade Amazon EKS cluster with control plane logging.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
- AWS Distro For OpenTelemetry Operator and Collector
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit) and CloudWatch Logs
- Enables CloudWatch Container Insights.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Objective

This pattern deploys one production grade Amazon EKS cluster, without any Observability add-on.
This pattern deploys one production grade Amazon EKS cluster with control plane logging, without any Observability add-on.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Amazon Managed Grafana is a managed service for Grafana, a popular open-source a

## Objective

This pattern deploys an Amazon EKS cluster with a node group that includes instance types featuring NVIDIA GPUs.
This pattern deploys an Amazon EKS cluster with control plane logging and a node group that includes instance types featuring NVIDIA GPUs.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

The AMI type of the node group is `AL2_x86_64_GPU AMI`, which uses the [Amazon EKS-optimized Linux AMI with GPU support](https://aws.amazon.com/marketplace/pp/prodview-nwwwodawoxndm). In addition to the standard Amazon EKS-optimized AMI configuration, the GPU AMI includes the NVIDIA drivers.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Visit our [EKS Blueprints docs](https://github.com/aws-quickstart/cdk-eks-bluepr

## Objective

- Deploys one production grade Amazon EKS cluster running on a Graviton3 Processor
- Deploys one production grade Amazon EKS cluster running on a Graviton3 Processor with control plane logging
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
- AWS Distro For OpenTelemetry Operator and Collector for Metrics and Traces
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit)
- Installs Grafana Operator to add AWS data sources and create Grafana Dashboards to Amazon Managed Grafana.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Objective

This pattern demonstrates how to use the _New EKS Cluster Open Source Observability Accelerator_ with Java based workloads.
This pattern demonstrates how to use the _New EKS Cluster Open Source Observability Accelerator_ with Java based workloads and control plane logging.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Utilizing CloudWatch and X-Ray as an aggregation layer allows for a fully-manage

## Objective

- Deploys one production grade Amazon EKS cluster.
- Deploys one production grade Amazon EKS cluster with Control Plane logging.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
- AWS Distro For OpenTelemetry Operator and Collector configured to collect metrics and traces.
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit) and CloudWatch Logs
- Aggregate Metrics in CloudWatch
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Objective

This pattern demonstrates how to use the _New EKS Cluster Open Source Observability Accelerator_ with Nginx based workloads.
This pattern demonstrates how to use the _New EKS Cluster Open Source Observability Accelerator_ with Nginx based workloads and control plane logging.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ please read the [service documentation](https://docs.aws.amazon.com/eks/latest/u

## Objective

- Deploys one production grade Amazon EKS cluster.
- Deploys one production grade Amazon EKS cluster with Control Plane logging.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
- AWS Distro For OpenTelemetry Operator and Collector for Metrics and Traces
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit)
- Installs Grafana Operator to add AWS data sources and create Grafana Dashboards to Amazon Managed Grafana.
Expand Down
1 change: 1 addition & 0 deletions lib/existing-eks-awsnative-observability-pattern/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ export default class ExistingEksAwsNativeObservabilityPattern {
.region(region)
.version('auto')
.enableNativePatternAddOns()
.enableControlPlaneLogging()
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
.clusterProvider(importClusterProvider)
.resourceProvider(blueprints.GlobalResources.Vpc, new blueprints.VpcProvider(vpcId)) // this is required with import cluster provider
.addOns(...addOns)
Expand Down
1 change: 1 addition & 0 deletions lib/existing-eks-mixed-observability-pattern/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ export default class ExistingEksMixedobservabilityPattern {
.region(region)
.version('auto')
.enableMixedPatternAddOns()
.enableControlPlaneLogging()
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
.clusterProvider(importClusterProvider)
.resourceProvider(blueprints.GlobalResources.Vpc, new blueprints.VpcProvider(vpcId)) // this is required with import cluster provider
.addOns(...addOns)
Expand Down
1 change: 1 addition & 0 deletions lib/existing-eks-opensource-observability-pattern/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ export default class ExistingEksOpenSourceobservabilityPattern {
.version('auto')
.withAmpProps(ampAddOnProps)
.enableOpenSourcePatternAddOns()
.enableControlPlaneLogging()
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
.clusterProvider(importClusterProvider)
.resourceProvider(blueprints.GlobalResources.Vpc, new blueprints.VpcProvider(vpcId)) // this is required with import cluster provider
.resourceProvider(ampWorkspaceName, new blueprints.CreateAmpProvider(ampWorkspaceName, ampWorkspaceName))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ export default class SingleNewEksAWSNativeFargateobservabilityConstruct {
.withCertManagerProps(certManagerAddOnProps)
.withCoreDnsProps(coreDnsAddOnProps)
.enableFargatePatternAddOns()
.enableControlPlaneLogging()
.clusterProvider(fargateClusterProvider)
.addOns(...addOns)
.build(scope, stackId);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ export default class SingleNewEksClusterAWSNativeobservabilityPattern {
.region(region)
.version('auto')
.enableNativePatternAddOns()
.enableControlPlaneLogging()
.addOns(...addOns)
.build(scope, stackId);
}
Expand Down
1 change: 1 addition & 0 deletions lib/single-new-eks-cluster-pattern/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export default class SingleNewEksPattern {
.account(account)
.region(region)
.version('auto')
.enableControlPlaneLogging()
.addOns(...addOns)
.build(scope, stackId);
}
Expand Down
1 change: 1 addition & 0 deletions lib/single-new-eks-mixed-observability-pattern/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ export default class SingleNewEksMixedobservabilityPattern {
.region(region)
.version('auto')
.enableMixedPatternAddOns()
.enableControlPlaneLogging()
.addOns(...addOns)
.build(scope, stackId);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ export default class SingleNewEksGpuOpenSourceObservabilityPattern {
.version('auto')
.withAmpProps(ampAddOnProps)
.enableOpenSourcePatternAddOns()
.enableControlPlaneLogging()
.resourceProvider(ampWorkspaceName, new blueprints.CreateAmpProvider(ampWorkspaceName, ampWorkspaceName))
.clusterProvider(
new blueprints.GenericClusterProvider({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ export default class SingleNewEksGravitonOpenSourceObservabilityPattern {
.version('auto')
.withAmpProps(ampAddOnProps)
.enableOpenSourcePatternAddOns()
.enableControlPlaneLogging()
.resourceProvider(ampWorkspaceName, new blueprints.CreateAmpProvider(ampWorkspaceName, ampWorkspaceName))
.clusterProvider(new blueprints.MngClusterProvider(mngProps))
.addOns(...addOns)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ export default class SingleNewEksOpenSourceobservabilityPattern {
.version('auto')
.withAmpProps(ampAddOnProps)
.enableOpenSourcePatternAddOns()
.enableControlPlaneLogging()
.addOns(...addOns)
.build(scope, stackId);
}
Expand Down