Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve linter. Fix broken links, spelling errors, enable markdown table formatter #507

Merged
merged 6 commits into from
Aug 31, 2022
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .markdown-link-check.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"httpHeaders": [
{
"urls": [
"https://docs.github.com/",
"https://help.github.com/"
],
"headers": {
"Accept-Encoding": "zstd, br, gzip, deflate"
}
}
]
}
7 changes: 6 additions & 1 deletion .mega-linter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,15 @@ APPLY_FIXES: none

# If you use ENABLE_LINTERS variable, all other linters will be disabled by default
ENABLE_LINTERS:
- BASH_EXEC
- CLOUDFORMATION_CFN_LINT
- DOCKERFILE_HADOLINT
- JSON_JSONLINT
- JSON_PRETTIER
- JSON_V8R
- JAVASCRIPT_STANDARD
- MARKDOWN_MARKDOWN_LINK_CHECK
- MARKDOWN_MARKDOWN_TABLE_FORMATTER
- SPELL_MISSPELL
- TERRAFORM_TFLINT
- YAML_YAMLLINT
Expand All @@ -31,4 +34,6 @@ JSON_PRETTIER_PRE_COMMANDS:
- command: npm install [email protected]
cwd: "workspace"

CLOUDFORMATION_CFN_LINT_FILE_EXTENSIONS: [".yml", ".yaml"]
CLOUDFORMATION_CFN_LINT_FILE_EXTENSIONS: [".yml", ".yaml"]

MARKDOWN_MARKDOWN_LINK_CHECK_ARGUMENTS: '-q'
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,6 @@ Please do **not** create a public GitHub issue.

## Licensing

See the [LICENSE](https://github.com/awslabs/aws-deployment-framework/blob/master/LICENSE)
See the [LICENSE](LICENSE.txt)
file for our project's licensing. We will ask you to confirm the licensing of
your contribution.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ Launch ADF via the
[Serverless Application Repository](https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:112893979820:applications/aws-deployment-framework)
within the AWS Console.

- Refer to the [Installation Guide](/docs/installation-guide.md) for
- Refer to the [Installation Guide](docs/installation-guide.md) for
Installation steps.
- Refer to the [User Guide](/docs/user-guide.md) for using ADF once it is setup.
- Refer to the [Samples Guide](/docs/samples-guide.md) for a detailed walk
- Refer to the [User Guide](docs/user-guide.md) for using ADF once it is setup.
- Refer to the [Samples Guide](docs/samples-guide.md) for a detailed walk
through of the provided samples.
78 changes: 41 additions & 37 deletions docs/admin-guide.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions docs/pipeline-types-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ in when updating ADF versions.

### Adding a new pipeline type

A pipeline can either be added manually into the [cdk_stacks](src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks)
folder as a separate python file or installed via [requirements.txt](src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/requirements.txt)
A pipeline can either be added manually into the [cdk_stacks](../src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks)
folder as a separate python file or installed via [requirements.txt](../src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/requirements.txt)
in the adf-build folder.

#### Source Code
Expand Down
51 changes: 33 additions & 18 deletions docs/providers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,37 @@ Providers and Actions.

## Index

- [Source](#source)
- [CodeCommit](#codecommit)
- [GitHub](#github)
- [S3](#s3)
- [CodeStar](#codestar)
- [Build](#build)
- [CodeBuild](#codebuild)
- [Jenkins](#jenkins)
- [Deploy](#deploy)
- [Approval](#approval)
- [CodeBuild](#codebuild-1)
- [CodeDeploy](#codedeploy)
- [CloudFormation](#cloudformation)
- [Lambda](#lambda)
- [Service Catalog](#service-catalog)
- [S3](#s3-1)
- [Providers Guide](#providers-guide)
- [Index](#index)
- [Source](#source)
- [CodeCommit](#codecommit)
- [Properties](#properties)
- [GitHub](#github)
- [Properties](#properties-1)
- [S3](#s3)
- [Properties](#properties-2)
- [CodeStar](#codestar)
- [Properties](#properties-3)
- [Build](#build)
- [CodeBuild](#codebuild)
- [Properties](#properties-4)
- [Jenkins](#jenkins)
- [Properties](#properties-5)
- [Deploy](#deploy)
- [Approval](#approval)
- [Properties](#properties-6)
- [CodeBuild](#codebuild-1)
- [Properties](#properties-7)
- [CodeDeploy](#codedeploy)
- [Properties](#properties-8)
- [CloudFormation](#cloudformation)
- [Properties](#properties-9)
- [Lambda](#lambda)
- [Properties](#properties-10)
- [Service Catalog](#service-catalog)
- [Properties](#properties-11)
- [S3](#s3-1)
- [Properties](#properties-12)

## Source

Expand Down Expand Up @@ -234,7 +249,7 @@ Provider type: `codebuild`.
> pipeline to consume a custom image if required.
> Along with `repository_arn`, we also support a `tag` key which can be used
> to define which image should be used (defaults to `latest`).
> An example of this setup is provided [here](https://github.com/awslabs/aws-deployment-framework/blob/master/docs/user-guide.md#custom-build-images).
> An example of this setup is provided [here](user-guide.md#custom-build-images).
>
> Image can also take an object that contains a reference to a
> public docker hub image with a prefix of `docker-hub://`, such as
Expand Down Expand Up @@ -299,7 +314,7 @@ Jenkins can be configured as the build provider, where it will be triggered
as part of the CodePipeline deployed by ADF.

To use Jenkins as a Build provider, you will need to install the
[Jenkins Plugin as documented here](https://wiki.jenkins.io/display/JENKINS/AWS+CodePipeline+Plugin).
[Jenkins Plugin as documented here](https://plugins.jenkins.io/aws-codepipeline).

Provider type: `jenkins`.

Expand Down
8 changes: 4 additions & 4 deletions docs/technical-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@ This document is intended to give insight into how the AWS Deployment Framework
## High Level Overview - AWS Deployment Framework Bootstrap Repository
The AWS Deployment Framework Bootstrap Repository aka "Bootstrap Repo" is where the source code used by ADF lives. The bootstrap repo is also where your accounts, OU layout and base templates are defined.
The flow below is a high level overview of what happens when a change is committed to this repository.
![bootstrap-repo-overview](./images/TechnicalGuide-BootstrapRepo.png)
![bootstrap-repo-overview](images/TechnicalGuide-BootstrapRepo.drawio.png)

### Account Management State Machine
The Account Managment State Machine is triggered by S3 PUT events to the ADF Accounts bucket.
The Account Management State Machine is triggered by S3 PUT events to the ADF Accounts bucket.
Below is a diagram detailing the components of the standard state machine. This state machine is defined in `src/account_processing.yml` and the lambda functions code is location in `src/lambda_codebase/account_processing`
![account-management-state-machine](./images/TechnicalGuide-AccountManagementStateMachine.drawio.png)
![account-management-state-machine](images/TechnicalGuide-AccountManagementStateMachine.drawio.png)


## High Level Overview - AWS Deployment Framework Pipeline Repository
The AWS Deployment Framework Pipeline Repository aka "Pipeline Rep" is where the deployment map definitions live. It typically exists in CodeCommit within your Deployment Account(s).
The diagram below details what happens when a commit is pushed to this repository.
![pipeline-repo-overview](./images/TechnicalGuide-PipelineRepo.drawio.png)
![pipeline-repo-overview](images/adf-pipeline-high-level.png)

### Pipeline Management State Machine
The Pipeline Management State machine is triggered by S3 PUT events to the ADF Pipelines bucket. This state machine is responsible for expanding the deployment map, resolving the targets, creating pipeline definitions (JSON objects that detail the source(s) and stages involved and the targets) and then generating CDK stacks off of the definitions.
Expand Down
48 changes: 28 additions & 20 deletions docs/user-guide.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,31 @@
# User Guide

- [Deployment Map](#deployment-map)
- [Providers](#providers)
- [Targets Syntax](#targets-syntax)
- [Params](#params)
- [Repositories](#repositories)
- [Completion Triggers](#completion-triggers)
- [Additional Triggers](#additional-triggers)
- [Additional Deployment Maps](#additional-deployment-maps)
- [Removing Pipelines](#removing-pipelines)
- [Deploying via Pipelines](#deploying-via-pipelines)
- [BuildSpec](#buildspec)
- [Parameters and Tagging](#cloudformation-parameters-and-tagging)
- [Serverless Transforms](#serverless-transforms)
- [Parameter Injection](#parameter-injection)
- [Nested Stacks](#nested-cloudformation-stacks)
- [Deploying Serverless Applications with SAM](#deploying-serverless-applications-with-sam)
- [Using Anchors and Alias](#using-anchors-and-alias)
- [One to many Relationships](#one-to-many-relationships)
- [User Guide](#user-guide)
- [Deployment Map](#deployment-map)
- [Targeting via Tags](#targeting-via-tags)
- [Important Notes](#important-notes)
- [Zero-prefixed AWS Account Ids](#zero-prefixed-aws-account-ids)
- [Providers](#providers)
- [Targets Syntax](#targets-syntax)
- [Params](#params)
- [Completion Triggers](#completion-triggers)
- [Additional Triggers](#additional-triggers)
- [Additional Deployment Maps](#additional-deployment-maps)
- [Repositories](#repositories)
- [Removing Pipelines](#removing-pipelines)
- [Deploying via Pipelines](#deploying-via-pipelines)
- [BuildSpec](#buildspec)
- [Custom Build Images](#custom-build-images)
- [CloudFormation Parameters and Tagging](#cloudformation-parameters-and-tagging)
- [Serverless Transforms](#serverless-transforms)
- [Parameter Injection](#parameter-injection)
- [Retrieving parameter values](#retrieving-parameter-values)
- [Importing output values](#importing-output-values)
- [Uploading assets](#uploading-assets)
- [Nested CloudFormation Stacks](#nested-cloudformation-stacks)
- [Deploying Serverless Applications with SAM](#deploying-serverless-applications-with-sam)
- [Using Anchors and Alias](#using-anchors-and-alias)
- [One to many relationships](#one-to-many-relationships)

## Deployment Map

Expand Down Expand Up @@ -71,7 +79,7 @@ pipelines:
name: fancy-name #Optional way to pass a name for this stage in the pipeline
```

In the above example we are creating two pipelines with AWS CodePipeline. The first one will deploy from a repository named **iam** that lives in the account `111111111111`. This CodeCommit Repository will automatically be created by default in the `111111111111` AWS Account if it does not exist. The automatic repository creation occurs if you enable `'auto-create-repositories'` (which is enabled by default). The `iam` pipeline will use AWS CodeCommit as its source and deploy in 3 steps. The first stage of the deployment will occur against all AWS Accounts that are in the `/security` Organization unit and be targeted to the `eu-west-1` region. After that, there is a manual approval phase which is denoted by the keyword `approval`. The next step will be targeted to the accounts within the `/banking/testing` OU *(in your default deployment account region)* region. By providing a simple path without a region definition it will default to the region chosen as the deployment account region in your [adfconfig](./admin-guide/adfconfig.yml). Any failure during the pipeline will cause it to halt.
In the above example we are creating two pipelines with AWS CodePipeline. The first one will deploy from a repository named **iam** that lives in the account `111111111111`. This CodeCommit Repository will automatically be created by default in the `111111111111` AWS Account if it does not exist. The automatic repository creation occurs if you enable `'auto-create-repositories'` (which is enabled by default). The `iam` pipeline will use AWS CodeCommit as its source and deploy in 3 steps. The first stage of the deployment will occur against all AWS Accounts that are in the `/security` Organization unit and be targeted to the `eu-west-1` region. After that, there is a manual approval phase which is denoted by the keyword `approval`. The next step will be targeted to the accounts within the `/banking/testing` OU *(in your default deployment account region)* region. By providing a simple path without a region definition it will default to the region chosen as the deployment account region in your [adfconfig](admin-guide.md#adfconfig). Any failure during the pipeline will cause it to halt.

The second pipeline (*vpc*) example deploys to an OU path `/banking/testing`. You can choose between an absolute path in your AWS Organization, AWS Account ID or an array of OUs or IDs. This pipeline also uses Github as a source rather than AWS CodeCommit. When generating the pipeline, ADF expects [GitHub Token](https://help.github.com/en/articles/creating-a-personal-access-token-for-the-command-line) to be placed in AWS Secrets Manager in a path prefixed with `/adf/`.

Expand Down Expand Up @@ -195,7 +203,7 @@ targets:
regions: [eu-central-1, us-west-1]
name: another_step
wave:
size: 30 # (Optional) This forces the pipeline to split this OU into seperate stages, each stage containing up to X accounts
size: 30 # (Optional) This forces the pipeline to split this OU into separate stages, each stage containing up to X accounts
exclude:
- 9999999999 # (Optional) List of accounts to exclude from this target. Currently only supports account Ids
properties: ...
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ function execute(){
chmod +x ./install

if ./install auto; then
echo "Instalation completed"
echo "Installation completed"
if ! ${AUTOUPDATE}; then
echo "Disabling Auto Update"
sed -i '/@reboot/d' /etc/cron.d/codedeploy-agent-update
Expand All @@ -49,7 +49,7 @@ function execute(){
fi
exit 0
else
echo "Instalation script failed, please investigate"
echo "Installation script failed, please investigate"
rm -f /tmp/install
exit 1
fi
Expand All @@ -60,7 +60,7 @@ function execute(){
chmod +x ./install

if ./install auto; then
echo "Instalation completed"
echo "Installation completed"
if ! ${AUTOUPDATE}; then
echo "Disabling auto update"
sed -i '/@reboot/d' /etc/cron.d/codedeploy-agent-update
Expand All @@ -69,7 +69,7 @@ function execute(){
fi
exit 0
else
echo "Instalation script failed, please investigate"
echo "Installation script failed, please investigate"
rm -f /tmp/install
exit 1
fi
Expand Down
18 changes: 9 additions & 9 deletions samples/sample-rdk-rules/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,14 @@ Sample pipeline definition looks like below:

## Development setup
After you clone the repo following file/folder structure will be there;
| Name | Purpose |
|---|---|
| config-rules | This folder will contain all the custom config rules created by `rdk create ...`. Make sure to setup correct details in the `parameters.json` file(ex: SourceRuntime) |
| params | Contains parameters we need for the generated CloudFormation templates. You must set the account id of the Compliance account in `LambdaAccountId` and Target accounts Config role arn as a pattern in `ConfigRoleArnToAssume`. These will be used as parameters when it deploys config-rule into Target accounts to refer Lambda function from the Compliance account. [Refer this link](https://github.com/awslabs/aws-deployment-framework/blob/master/docs/user-guide.md#cloudformation-parameters-and-tagging) |
| templates | This folder contains all the cloudformation template pieces that required to build cfn template for the lambda function deployment. |
| buildspec.yml | Buildspec file to generate Cloudformation templates for the Lambda and Custom Config rules |
| lambda_helper.py | This is the helper file that pack and upload the lambda code recursively in the config-rules folder |
| requirements.txt| Requirements for the lambda_helper.py script.|
| Name | Purpose |
|------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| config-rules | This folder will contain all the custom config rules created by `rdk create ...`. Make sure to setup correct details in the `parameters.json` file(ex: SourceRuntime) |
| params | Contains parameters we need for the generated CloudFormation templates. You must set the account id of the Compliance account in `LambdaAccountId` and Target accounts Config role arn as a pattern in `ConfigRoleArnToAssume`. These will be used as parameters when it deploys config-rule into Target accounts to refer Lambda function from the Compliance account. [Refer this link](../../docs/user-guide.md#cloudformation-parameters-and-tagging) |
| templates | This folder contains all the cloudformation template pieces that required to build cfn template for the lambda function deployment. |
| buildspec.yml | Buildspec file to generate Cloudformation templates for the Lambda and Custom Config rules |
| lambda_helper.py | This is the helper file that pack and upload the lambda code recursively in the config-rules folder |
| requirements.txt | Requirements for the lambda_helper.py script. |

## Lambda function implementation requirements
In Lambda functions when you want to refer boto3 client or resource make sure
Expand Down Expand Up @@ -84,7 +84,7 @@ These methods use STS and config payload to assume the IAM role in the target ac

## Prerequisites/ Important bits
- This solution does not setup config or config recorder.
- When this solution deploys the config-rule to a target account; it expectes config is enabled in the target account.
- When this solution deploys the config-rule to a target account; it expects config is enabled in the target account.
- Each target account's config role should be able assume by `<account-that-has-the-lambda-function>` to put evaluations into each target account's config. AKA config role in target account(2222222222) should have the lambda-function-account-id(1111111111) as trusted entity as below.

![Trusted entiry](./meta/lambda-account-id-trusted-entiry.png)
Loading