Multi Account Mixed Observability Observability Accelerator (#110)

* initial commit * AMP only try * AMP try * AMP only try * AMP only * AMP try * AMP Try * AMP try * AMP try * AMP try * AMP try * AMP Try * AMP try * AMP try * AMP try * AMP try * AMP try * AMP try * AMP Try * AMP try * AMP try * AMP try * AMP try * AMP try * AMP * AMP * AMP only * prod2 * prod2 * prod2 * prod2 * prod2 * amg * amg * amg * amg * amg * amg * amg * AMG * AMG * AMG * AMG * AMG * AMG * AMG * AMG and PROD2 * removed ssh from argocd * just credentialsSecretName * added credentialsType * all-in * AMG only * no ssh-key * with github-ssh-key * separated for public and private * try * updated createArgoAddonConfig * with go on mon-account * update * edit * added grafanaOperatorArgoAddonConfig * removed grafanaOperatorArgoAddonConfig * adding flux * new flux * updated * new bootstrapvals * added GrafanaOperatorAddon * new changes * update * resolved * updated * update * update * update * update * added mon * update * update * update * updated * update * update * updated bootstrapvals * updated amgEndpointUrl * updates * updated * updated with context from cdk.json * updates * update * updates * path * try * update * upd * upd * added CW in bootstrapValues * app update * update * upd * update * upd * converted to SSM SecureString * added pipelineSource * updates * update * SSM step * updates * upd * update * GO * more * upd * update * update * UPD * updates * update * update * doc update * more updates * upd * updated * rolechange * new * updated * split pipeline * commit * added * correct sampleStage * with account * update * commented out * upd * updated to prod1 * upd * upd * update * added new * update * removed getAMPInfo * doc updates * doc updates * doc updates * doc update and clean up script * updated doc * updated pipeline.ts * cleanup script and doc update * updated pipeline.ts to single statement * pipeline module * updated multi-account-central-pipeline to multi-account-COA-pipeline removed amp-iam-setup updated doc * doc update, architecture diagram and code updates * cleanup script update * cleanup script update * cleanup script update * update * added cwAssumeRoleName * updated cleanup * md update * updated doc * updated doc * Incorporating review comments removed comments in grafana-operator-construct removed comments in lib/multi-acc-new-eks-mixed-observability-pattern/cloudwatch-monitoring.ts renamed CreateIAMRoleNestedStack.ts to create-iam-role.ts getSSMSecureString.ts to get-ssm-securestring.ts GrafanaOperatorConstruct.ts to grafana-operator-index.ts grafanaoperatorsecretaddon.ts to grafana-operator-secret-addon.ts scripts to helpers * removed ServicePrincipal which is not in use * removed trailing whitespaces * moved policystatements outside of pipeline.ts * updated doc * updated 'scripts' ref to 'helpers' * updated codebuild role * rolled back to old way of creating iam role * fix with pipeline.ts * WORKING VERSION * policydoc outside * all policydocs are outside pipeline.ts * updated doc and some cleanup at pipeline.ts * doc update * fixed undefined assumerole with monitoringEnv * updated argocd app sync * updated sample runme gif * doc update * updated git-ssh-key to pipeline-account * removed git-ssh-key. sample apps are in publi repo * moved github-token creation to script * added categories to doc * removed post-deploy * Revert "removed post-deploy" This reverts commit d238188. * Revert "removed post-deploy" This reverts commit d238188. * Revert "added categories to doc" This reverts commit 3622641. * updated categories * updated command for running argo cd admin ui * updated cleanup script * updated doc * updated to artifacts/argocd-apps ref * updated progress check command * updated argo apps to aws-observability repo * updated broken URLs * fixed broken URLs in doc * removed unused images * added pre-req for tools * main readme with pattern docs, mkdocs, GH fix * updated README and doc for mkdocs * main readme update * moved argocdaddonconfig * removed createArgoAddonConfig * retaining one doc * retaining one doc * renamed in mkdocs
aws-observability · Oct 6, 2023 · 8f07b19 · 8f07b19
1 parent 3740a50
commit 8f07b19
Show file tree

Hide file tree

Showing 33 changed files with 1,877 additions and 70 deletions.
diff --git a/README.md b/README.md
@@ -20,75 +20,81 @@ We provide curated metrics, logs, traces collection, cloudwatch dashboard, alert
 The individual patterns can be found in the [`lib`](https://github.com/aws-observability/cdk-aws-observability-accelerator/tree/main/lib) directory.  Most of the patterns are self-explanatory, for some more complex examples please use this guide and docs/patterns directory for more information.
 
 ## Usage
+
 Before proceeding, make sure [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) is installed on your machine.
 
-To use the eks-blueprints and patterns module, you must have [Node.js](https://nodejs.org/en/) and [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) installed. You will also use `make` and `brew` to simplify build and other common actions. 
+To use the eks-blueprints and patterns module, you must have [Node.js](https://nodejs.org/en/) and [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) installed. You will also use `make` and `brew` to simplify build and other common actions.
 
 ### Ubuntu Setup
+
 Follow the below steps to setup and leverage cdk-aws-observability-accelerator in your Ubuntu Linux machine.
 
-1. **Update the package list** 
+1. **Update the package list**
 
-    Update the package list to ensure you're installing the latest versions.
+Update the package list to ensure you're installing the latest versions.
 
-    ```bash
-    sudo apt update
-    ```
+```bash
+sudo apt update
+```
 
 2. **Install make**
 
-    ```bash
-    sudo apt install make
-    ```
+```bash
+sudo apt install make
+```
 
 3. **Install Node.js and npm**
 
-    Install Node.js and npm using the NodeSource binary distributions.
-    ```bash
-    curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - &&\
-    sudo apt-get install -y nodejs
-    ```
-
-    Note: The Node.js package from NodeSource includes npm
+Install Node.js and npm using the NodeSource binary distributions.
+
+```bash
+curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - &&\
+sudo apt-get install -y nodejs
+```
+
+Note: The Node.js package from NodeSource includes npm
 
 4. **Verify Node.js and npm Installation**
 
-    Check the installed version of Node.js:
+Check the installed version of Node.js:
 
-    ```bash
-    node -v
-    ```
+```bash
+node -v
+```
 
-    The output should be `v20.x.x`.
+The output should be `v20.x.x`.
 
-    Check the installed version of npm:
+Check the installed version of npm:
 
-    ```bash
-    npm -v
-    ```
+```bash
+npm -v
+```
 
-    The output should be a version greater than `9.7.x`.
+The output should be a version greater than `9.7.x`.
 
-    If your npm version is not `9.7.x` or above, update npm with the following command:
+If your npm version is not `9.7.x` or above, update npm with the following command:
 
-    ```bash
-    sudo npm install -g npm@latest
-    ```
+```bash
+sudo npm install -g npm@latest
+```
 
-    Verify the installed version by running `npm -v`.
+Verify the installed version by running `npm -v`.
 
 5. Install brew on ubuntu by following instructions as detailed in [docs.brew.sh](https://docs.brew.sh/Homebrew-on-Linux)
-   ```bash
-    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
-   ```
-   Add Homebrew to your PATH
-   ```
-   test -d ~/.linuxbrew && eval "$(~/.linuxbrew/bin/brew shellenv)"
-   test -d /home/linuxbrew/.linuxbrew && eval "$(/home/linuxbrew/.linux  brew/bin/brew shellenv)"
-   test -r ~/.bash_profile && echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> ~/.bash_profile
-   echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> ~/.profile
-
-   ```
+
+```bash
+ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
+```
+
+Add Homebrew to your PATH
+
+```sh
+test -d ~/.linuxbrew && eval "$(~/.linuxbrew/bin/brew shellenv)"
+test -d /home/linuxbrew/.linuxbrew && eval "$(/home/linuxbrew/.linux  brew/bin/brew shellenv)"
+test -r ~/.bash_profile && echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> ~/.bash_profile
+echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> ~/.profile
+
+```
 
 Post completing the above, continue from Step: [Repo setup](#repo-setup)
 
@@ -98,14 +104,14 @@ Follow the below steps to setup and leverage `cdk-aws-observability-accelerator`
 
 1. Install `make` and `node` using brew
 
-```
+```sh
 brew install make
 brew install node
 ```
 
 2. Install `npm`
 
-```
+```sh
 sudo npm install -g n
 sudo n stable
 ```
@@ -121,7 +127,7 @@ v20.3.1
 
 Update (provided Node version manager is installed): `n stable`. May require `sudo`.
 
--  NPM version must be 8.4 or above:
+- NPM version must be 8.4 or above:
 
 ```bash
 $ npm -v
@@ -130,67 +136,72 @@ $ npm -v
 
 Updating npm: `sudo n stable` where stable can also be a specific version above 8.4. May require `sudo`.
 
-
 ### Repo setup
+
 1. Clone the `cdk-aws-observability-accelerator` repository
 
-```
+```sh
 git clone https://github.com/aws-observability/cdk-aws-observability-accelerator.git
-``` 
+```
 
 PS: If you are contributing to this repo, please make sure to fork the repo, add your changes and create a PR against it.
 
 2. Once you have cloned the repo, you can open it using your favourite IDE and run the below commands to install the dependencies and build the existing patterns.
 
 - Install project dependencies.
 
-```
+```text
 make deps
 ```
 
 - To view patterns that are available to be deployed, execute the following:
 
-```
+```sh
 make build
 ```
 
 - To list the existing CDK AWS Observability Accelerator Patterns
 
-```
+```text
 make list
 ```
 
 Note: Some patterns have a hard dependency on AWS Secrets (for example GitHub access tokens). Initially you will see errors complaining about lack of the required secrets. It is normal. At the bottom, it will show the list of patterns which can be deployed, in case the pattern you are looking for is not available, it is due to the hard dependency which can be fixed by following the docs specific to those patterns.
 
-```
+```ps1
 To work with patterns use:
 	$ make pattern <pattern-name> <list | deploy | synth | destroy>
 Example:
 	$ make pattern single-new-eks-opensource-observability deploy
 
 Patterns:
 
+	single-new-eks-cluster
 	existing-eks-mixed-observability
-	existing-eks-opensource-observability
+	single-new-eks-opensource-observability
 	single-new-eks-awsnative-observability
-	single-new-eks-cluster
+	single-new-eks-gpu-opensource-observability
 	single-new-eks-graviton-opensource-observability
 	single-new-eks-mixed-observability
-	single-new-eks-opensource-observability
+	existing-eks-opensource-observability
+	existing-eks-awsnative-observability
+	multi-acc-new-eks-mixed-observability
 ```
 
 - Bootstrap your CDK environment.
 
-```
+```sh
 npx cdk bootstrap
 ```
 
 - You can then deploy a specific pattern with the following:
 
-```
+```sh
 make pattern single-new-eks-opensource-observability deploy
 ```
 
+- To access instructions for individual patterns check documentation in `docs/patterns` directory.
+
 # Developer Flow
 
 ## Modifications
@@ -201,17 +212,18 @@ All files are compiled to the dist folder including `lib` and `bin` directories.
 make compile
 ```
 
-The `compile` command is optimized to build only modified files and is fast. 
+The `compile` command is optimized to build only modified files and is fast.
 
 ## New Patterns
 
 To create a new pattern, please follow these steps:
 
-1. Under lib create a folder for your pattern, such as `<pattern-name>-pattern`. If you plan to create a set of patterns that represent a particular subdomain, e.g. `security` or `hardening`, please create an issue to discuss it first. If approved, you will be able to create a folder with your subdomain name and group your pattern constructs under it. 
-2. Blueprints generally don't require a specific class, however we use a convention of wrapping each pattern in a plain class like `<Pattern-Name>Pattern`. This class is generally placed in `index.ts` under your pattern folder. 
+1. Under lib create a folder for your pattern, such as `<pattern-name>-pattern`. If you plan to create a set of patterns that represent a particular subdomain, e.g. `security` or `hardening`, please create an issue to discuss it first. If approved, you will be able to create a folder with your subdomain name and group your pattern constructs under it.
+2. Blueprints generally don't require a specific class, however we use a convention of wrapping each pattern in a plain class like `<Pattern-Name>Pattern`. This class is generally placed in `index.ts` under your pattern folder.
 3. Once the pattern implementation is ready, you need to include it in the list of the patterns by creating a file `bin/<pattern-name>.ts`. The implementation of this file is very light, and it is done to allow patterns to run independently.
 
 Example simple synchronous pattern:
+
 ```typescript
 import SingleNewEksOpenSourceobservabilityPattern from '../lib/single-new-eks-opensource-observability-pattern';
 import { configureApp } from '../lib/common/construct-utils';

diff --git a/bin/multi-acc-new-eks-mixed-observability.ts b/bin/multi-acc-new-eks-mixed-observability.ts
@@ -0,0 +1,15 @@
+
+import { configureApp, errorHandler } from '../lib/common/construct-utils';
+import { PipelineMultiEnvMonitoring } from '../lib/multi-acc-new-eks-mixed-observability-pattern';
+
+const app = configureApp();
+
+/*
+CDK Observability Accelerator pattern for Multiple EKS clusters in multiple AWS accounts
+*/
+
+new PipelineMultiEnvMonitoring()
+    .buildAsync(app)
+    .catch((e) => {
+        errorHandler(app, "Multi Account Multi EKS CDK Observability pattern setup errored. Ensure SSM SecureString Parameter /cdk-accelerator/cdk-context exists in COA_PIPELINE_REGION of pipeline account. See documentation for details.", e);
+    });
diff --git a/docs/patterns/images/multi-acc-new-eks-mixed-observability-pattern-amg-amp1.png b/docs/patterns/images/multi-acc-new-eks-mixed-observability-pattern-amg-amp1.png
diff --git a/docs/patterns/images/multi-acc-new-eks-mixed-observability-pattern-amg-cw1.png b/docs/patterns/images/multi-acc-new-eks-mixed-observability-pattern-amg-cw1.png
diff --git a/docs/patterns/images/multi-acc-new-eks-mixed-observability-pattern-amg-xray.png b/docs/patterns/images/multi-acc-new-eks-mixed-observability-pattern-amg-xray.png
diff --git a/...s/images/multi-acc-new-eks-mixed-observability-pattern-architecture-dark-v2.gif b/...s/images/multi-acc-new-eks-mixed-observability-pattern-architecture-dark-v2.gif
diff --git a/docs/patterns/images/multi-acc-new-eks-mixed-observability-pattern-runme-v2.gif b/docs/patterns/images/multi-acc-new-eks-mixed-observability-pattern-runme-v2.gif