Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Control Plane Logging as a default in all patterns #129

Merged
merged 6 commits into from
Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/logs.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ Then you can view the results of your query:

![logs-results](./patterns/images/mixed-log-results.png)

You can also find control plane logs in CloudWatch with Control Plane logs visible under the `/cluster` log group:

![control-plane-logs](./patterns/images/control-plane-logs.png)

You can then filter by patterns within the logs to find specific events like so:
![control-plane-insights](./patterns/images/control-plane-insights.png)

## Viewing Logs in Grafana

### Using CloudWatch Logs as data source in Grafana
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ You will also need:

## Deploying

!!! note If control plane logging is not enabled in the existing cluster, edit `lib/existing-eks-opensource-observability-pattern/index.ts` to include the following code snippet:
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
```typescript
ObservabilityBuilder.builder()
// some properties
.enableControlPlaneLogging()
// other properties
.build(scope, stackId);
```


1. Edit `~/.cdk.json` by setting the name of your existing cluster:

```json
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,15 @@ You will also need:
2. An OpenID Connect (OIDC) provider, associated to the above EKS cluster (Note: Single EKS Cluster Pattern takes care of that for you)

## Deploying
!!! note If control plane logging is not enabled in the existing cluster, edit `lib/existing-eks-awsnative-observability-pattern/index.ts` to include the following code snippet:
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
```typescript
ObservabilityBuilder.builder()
// some properties
.enableControlPlaneLogging()
// other properties
.build(scope, stackId);
```


1. Edit `~/.cdk.json` by setting the name of your existing cluster:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Utilizing CloudWatch and X-Ray as an aggregation layer allows for a fully-manage

## Objective

This pattern aims to add Observability on top of an existing EKS cluster, with a mixture of AWS native and open source managed AWS services.
This pattern aims to add Observability and control plane logging on top of an existing EKS cluster, with a mixture of AWS native and open source managed AWS services.
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
5herlocked marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites:

Expand All @@ -32,6 +32,16 @@ You will also need:

## Deploying

!!! note If control plane logging is not enabled in the existing cluster, edit `lib/existing-eks-mixed-observability-pattern/index.ts` to include the following code snippet:
```typescript
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
ObservabilityBuilder.builder()
// some properties
.enableControlPlaneLogging()
// other properties
.build(scope, stackId);
```


1. Edit `~/.cdk.json` by setting the name of your existing cluster:

```json
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,15 @@ You will also need:

## Deploying

!!! note If control plane logging is not enabled in the existing cluster, edit `lib/existing-eks-opensource-observability-pattern/index.ts` to include the following code snippet:
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
```typescript
ObservabilityBuilder.builder()
// some properties
.enableControlPlaneLogging()
// other properties
.build(scope, stackId);
```

1. Edit `~/.cdk.json` by setting the name of your existing cluster:

```json
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,15 @@ You will also need:

## Deploying

!!! note If control plane logging is not enabled in the existing cluster, edit `lib/existing-eks-opensource-observability-pattern/index.ts` to include the following code snippet:
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
```typescript
ObservabilityBuilder.builder()
// some properties
.enableControlPlaneLogging()
// other properties
.build(scope, stackId);
```

1. Edit `~/.cdk.json` by setting the name of your existing cluster:

```json
Expand Down
Binary file added docs/patterns/images/control-plane-insights.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/patterns/images/control-plane-logs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The following figure illustrates the architecture of the pattern we will be depl

## Objective

1. Deploying two production grade Amazon EKS cluster across two AWS Accounts (Prod1, Prod2 account) in two different regions through a Continuous Deployment infrastructure pipeline triggered upon a commit to the repository that holds the pipeline configuration in another AWS account (pipeline account).
1. Deploying two production grade Amazon EKS cluster with control plane logging across two AWS Accounts (Prod1, Prod2 account) in two different regions through a Continuous Deployment infrastructure pipeline triggered upon a commit to the repository that holds the pipeline configuration in another AWS account (pipeline account).
2. Deploying ADOT add-on, AMP add-on to Prod 1 Amazon EKS Cluster to remote-write metrics to AMP workspace in Prod 1 AWS Account.
3. Deploying ADOT add-on, CloudWatch add-on to Prod 2 Amazon EKS Cluster to write metrics to CloudWatch in Prod 2 AWS Account.
4. Configuring GitOps tooling (Argo CD add-on) to support deployment of [ho11y](https://github.com/aws-observability/aws-o11y-recipes/tree/main/sandbox/ho11y) and [yelb](https://github.com/mreferre/yelb) sample applications, in a way that restricts each application to be deployed only into the team namespace, by using Argo CD projects.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ please read the [service documentation](https://docs.aws.amazon.com/eks/latest/u

## Objective

- Deploys one production grade Amazon EKS Fargate cluster.
- Deploys one production grade Amazon EKS Fargate cluster
- Enables Control Plane Logging.
- Logs with CloudWatch Logs
- Enables CloudWatch Container Insights.
- Installs Prometheus Node Exporter and Metrics Server for infrastructure metrics.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ please read the [service documentation](https://docs.aws.amazon.com/eks/latest/u
## Objective

- Deploys one production grade Amazon EKS cluster.
- Enables Control Plane Logging.
- AWS Distro For OpenTelemetry Operator and Collector
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit) and CloudWatch Logs
- Enables CloudWatch Container Insights.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Amazon Managed Grafana is a managed service for Grafana, a popular open-source a

## Objective

This pattern deploys an Amazon EKS cluster with a node group that includes instance types featuring NVIDIA GPUs.
This pattern deploys an Amazon EKS cluster and a node group that includes instance types featuring NVIDIA GPUs.

The AMI type of the node group is `AL2_x86_64_GPU AMI`, which uses the [Amazon EKS-optimized Linux AMI with GPU support](https://aws.amazon.com/marketplace/pp/prodview-nwwwodawoxndm). In addition to the standard Amazon EKS-optimized AMI configuration, the GPU AMI includes the NVIDIA drivers.

Expand All @@ -25,6 +25,8 @@ Data is visualised in Amazon Managed Grafana by the [NVIDIA DCGM Exporter Dashbo

The rest of the setup to collect and visualise metrics with Amazon Managed Service for Prometheus and Amazon Managed Grafana, is similar to that used in other open-source based patterns included in this repository.

It also enables control plane logging on the EKS cluster for a comprehensive overview of cluster health.

## Prerequisites:

Ensure that you have installed the following tools on your machine.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Visit our [EKS Blueprints docs](https://github.com/aws-quickstart/cdk-eks-bluepr
## Objective

- Deploys one production grade Amazon EKS cluster running on a Graviton3 Processor
- Enables control plane logging
- AWS Distro For OpenTelemetry Operator and Collector for Metrics and Traces
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit)
- Installs Grafana Operator to add AWS data sources and create Grafana Dashboards to Amazon Managed Grafana.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

This pattern demonstrates how to use the _New EKS Cluster Open Source Observability Accelerator_ with Java based workloads.

It also enables control plane logging for comprehensive monitoring on the EKS cluster.

## Prerequisites

Ensure that you have installed the following tools on your machine.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Utilizing CloudWatch and X-Ray as an aggregation layer allows for a fully-manage
## Objective

- Deploys one production grade Amazon EKS cluster.
- Enables Control Plane logging
- AWS Distro For OpenTelemetry Operator and Collector configured to collect metrics and traces.
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit) and CloudWatch Logs
- Aggregate Metrics in CloudWatch
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

This pattern demonstrates how to use the _New EKS Cluster Open Source Observability Accelerator_ with Nginx based workloads.

It also enables control plane logging to provide comprehensive overview of cluster health.

## Prerequisites

Ensure that you have installed the following tools on your machine.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ please read the [service documentation](https://docs.aws.amazon.com/eks/latest/u
## Objective

- Deploys one production grade Amazon EKS cluster.
- Enables Control Plane logging.
- AWS Distro For OpenTelemetry Operator and Collector for Metrics and Traces
- Logs with [AWS for FluentBit](https://github.com/aws/aws-for-fluent-bit)
- Installs Grafana Operator to add AWS data sources and create Grafana Dashboards to Amazon Managed Grafana.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ export default class SingleNewEksAWSNativeFargateobservabilityConstruct {
.withCertManagerProps(certManagerAddOnProps)
.withCoreDnsProps(coreDnsAddOnProps)
.enableFargatePatternAddOns()
.enableControlPlaneLogging()
.clusterProvider(fargateClusterProvider)
.addOns(...addOns)
.build(scope, stackId);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ export default class SingleNewEksClusterAWSNativeobservabilityPattern {
.region(region)
.version('auto')
.enableNativePatternAddOns()
.enableControlPlaneLogging()
.addOns(...addOns)
.build(scope, stackId);
}
Expand Down
1 change: 1 addition & 0 deletions lib/single-new-eks-cluster-pattern/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export default class SingleNewEksPattern {
.account(account)
.region(region)
.version('auto')
.enableControlPlaneLogging()
.addOns(...addOns)
.build(scope, stackId);
}
Expand Down
1 change: 1 addition & 0 deletions lib/single-new-eks-mixed-observability-pattern/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ export default class SingleNewEksMixedobservabilityPattern {
.region(region)
.version('auto')
.enableMixedPatternAddOns()
.enableControlPlaneLogging()
.addOns(...addOns)
.build(scope, stackId);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ export default class SingleNewEksGpuOpenSourceObservabilityPattern {
.version('auto')
.withAmpProps(ampAddOnProps)
.enableOpenSourcePatternAddOns()
.enableControlPlaneLogging()
.resourceProvider(ampWorkspaceName, new blueprints.CreateAmpProvider(ampWorkspaceName, ampWorkspaceName))
.clusterProvider(
new blueprints.GenericClusterProvider({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ export default class SingleNewEksGravitonOpenSourceObservabilityPattern {
.version('auto')
.withAmpProps(ampAddOnProps)
.enableOpenSourcePatternAddOns()
.enableControlPlaneLogging()
.resourceProvider(ampWorkspaceName, new blueprints.CreateAmpProvider(ampWorkspaceName, ampWorkspaceName))
.clusterProvider(new blueprints.MngClusterProvider(mngProps))
.addOns(...addOns)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ export default class SingleNewEksOpenSourceobservabilityPattern {
.version('auto')
.withAmpProps(ampAddOnProps)
.enableOpenSourcePatternAddOns()
.enableControlPlaneLogging()
.addOns(...addOns)
.build(scope, stackId);
}
Expand Down