Merge pull request #162 from globaldatanet/4.0.0

4.0.0
globaldatanet · Sep 1, 2023 · f05bec2 · f05bec2
2 parents d2dfdc8 + fd46799
commit f05bec2
Show file tree

Hide file tree

Showing 33 changed files with 4,278 additions and 4,622 deletions.
diff --git a/.gitignore b/.gitignore
@@ -2,7 +2,8 @@
 !jest.config.js
 *.d.ts
 node_modules
-
+*.DS_Store
+*.tgz
 # CDK asset staging directory
 .cdk.staging
 cdk.out

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,29 @@
 
 ## Released
 
+## 4.0.0
+### Added
+- A custom resource to retrieve the latest version of the ManagedRuleGroup and check if the specified version is valid.
+- Typescript configuration files for WAF configurations - now it is easier to write custom rules because of the types for [rule statements](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_wafv2.CfnWebACL.RuleProperty.html).
+- A function to convert CdkRule to SdkRule - with the introduction of Typescript configuration and CDK interfaces, we now need to convert every CDK rule to an SDK rule to be able to use the [CheckCapacity API call](https://docs.aws.amazon.com/waf/latest/APIReference/API_CheckCapacity.html).
+- ManagedRuleGroupVersions for CloudFormation Output
+- Example Configurations
+    1.  Example WAF configuration against: [OWASP Top Ten](https://owasp.org/www-project-top-ten/)
+    2.  Example configuration for prerequisite stack
+- Added TOOL_KIT_STACKNAME to the TaskFile - to specify the name of the bootstrap stack ([see Bootstrapping your AWS environment](https://docs.aws.amazon.com/cdk/v2/guide/cli.html#cli-bootstrap)).
+- Migrate script to migrate from json to ts config (./values/migrate.ts)
+  - ts node ./values/migrate.ts YOURJSON.json
+- You now need to set the priority for your custom rules. If you want to learn more about processing order of rules and rule groups in a web ACL, check out this [link](https://docs.aws.amazon.com/waf/latest/developerguide/web-acl-processing-order.html).
+
+### Fixed
+- Allow sub-statements of IPSetReferenceStatements -> Allow IPSetReferenceStatement.ARN entries that reference an aws-firewall-factory controlled ipset (i.e. the name of the ipset) within AND, OR and NOT statements (as sub-statements).
+- Adjusted WAF Config skeleton generation function for Typescript configuration.
+- Updated dependencies to the latest version
+
+### Removed
+- Json config files for WAF configurations
+- DeployHash generation for new configs - legacy functionality - we will now use Prefix, Stage & FirewallName to create unique WAF and CloudFormation StackNames.
+
 ## 3.3.1
 ### Fixed
 - example Json Files
@@ -44,7 +67,6 @@
   - community-user-agent testcases
   - improve owasp testcases
 - Update Testing bin Version
-
 ## 3.2.3
 ### Fixed
 - Bump eslint from 8.41.0 to 8.42.0
@@ -60,15 +82,13 @@
 
 ### Added
 - Added Linting command `lint` to npm scripts which can be run via `npm run lint`◊
-
 ## 3.2.2
 ### Fixed
 - Bump @aws-sdk/client-pricing from 3.332.0 to 3.341.0
 - Bump eslint from 8.40.0 to 8.41.0
 - Bump @aws-sdk/client-cloudwatch from 3.321.1 to 3.341.0
 - Bump @aws-sdk/client-service-quotas from 3.321.1 to 3.342.0
 - Bump @aws-sdk/client-fms from 3.332.0 to 3.342.0
-
 ## 3.2.1
 ### Fixed
 - Bump aws-cdk from 2.74.0 to 2.79.1 

diff --git a/README.md b/README.md
@@ -1,4 +1,5 @@
 
+<a href="https://github.com/globaldatanet/aws-firewall-factory/actions/workflows/linting.yml"><img alt="github-badge" src="https://github.com/globaldatanet/aws-firewall-factory/actions/workflows/linting.yml/badge.svg"></a>
 [![Mentioned in Awesome CDK](https://awesome.re/mentioned-badge.svg)](https://github.com/kolomied/awesome-cdk)
 [![License: Apache2](https://img.shields.io/badge/license-Apache%202-lightgrey.svg)](http://www.apache.org/licenses/) [![cdk](https://img.shields.io/badge/aws_cdk-v2-orange.svg)](https://docs.aws.amazon.com/cdk/v2/guide/home.html)
 [![latest](https://img.shields.io/badge/latest-release-yellow.svg)](https://github.com/globaldatanet/aws-firewall-factory/releases)
@@ -7,7 +8,6 @@
 [![Tweet](https://img.shields.io/twitter/url/http/shields.io.svg?style=social)](https://twitter.com/intent/tweet?text=AWS%20FIREWALL%20FACTORY%20-%20Deploy%2C%20update%2C%20and%20stage%20your%20WAFs%20while%20managing%20them%20centrally%20via%20FMS&url=https://github.com/globaldatanet/aws-firewall-factory&hashtags=aws,security,waf)
 [![roadmap](https://img.shields.io/badge/public-roadmap-yellow.svg)](https://github.com/orgs/globaldatanet/projects/1)
 
-
 **[🚧 Feature request](https://github.com/globaldatanet/aws-firewall-factory/issues/new?assignees=&labels=feature-request%2C+enhancement&template=feature_request.md&title=)** | **[🐛 Bug Report](https://github.com/globaldatanet/aws-firewall-factory/issues/new?assignees=&labels=bug%2C+triage&template=bug_report.md&title=)**
 
 <img src="https://socialify.git.ci/globaldatanet/aws-firewall-factory/image?description=1&font=Raleway&issues=1&logo=https://raw.githubusercontent.com/globaldatanet/aws-firewall-factory/master/static/icon/firewallfactory.svg&pattern=Solid&stargazers=1&theme=Dark" alt="AWSFirewallFactory" width="900" height="320"/>
@@ -83,38 +83,37 @@ If you want to learn more about the AWS Firewall Factory feel free to look at th
 
 6. Checking of the softlimit quota for [WCU](https://docs.aws.amazon.com/waf/latest/developerguide/how-aws-waf-works.html) set in the AWS account (stop deployment if calculated WCU is above the quota)
 
-7. Easy configuration of WAF rules trough JSON file.
-
-8. Deployment hash to deploy same WAF more than once for testing and/or blue/green deployments.
-
-9. Stopping deployment if soft limit will be exceeded:  **Firewall Manager policies per organization per Region (L-0B28E140)** - **Maximum number of web ACL capacity units in a web ACL in WAF for regional (L-D9F31E8A)**
+7. Easy configuration of WAF rules trough Typescript file.
+
+8. Deploy same WAF more than once for testing and/or blue/green deployments.
 
-10. **RegexMatchStatement** and **IPSetReferenceStatement** is working now 🚀
+9.  Stopping deployment if soft limit will be exceeded:  **Firewall Manager policies per organization per Region (L-0B28E140)** - **Maximum number of web ACL capacity units in a web ACL in WAF for regional (L-D9F31E8A)**
 
-11. You can name your rules. If you define a name in your RulesArray, the name + a Base36 timestamp will be used for the creation of your rule - otherwise a name will be generated. This will help you to query your logs in Athena. The same rule name also applies to the metric by adding "-metric" to the name.
+10. You can name your rules. If you define a name in your RulesArray, the name + a Base36 timestamp will be used for the creation of your rule - otherwise a name will be generated. This will help you to query your logs in Athena.
 
-12. Support for Captcha - You can add Captcha as an action to your WAFs. This helps you block unwanted bot traffic by requiring users to successfully complete challenges before their web request are allowed to reach AWS WAF protected resources. AWS WAF Captcha is available in the US East (N. Virginia), US West (Oregon), Europe (Frankfurt), South America (Sao Paulo), and Asia Pacific (Singapore) AWS Regions and supports Application Load Balancer, Amazon API Gateway, and AWS AppSync resources.
+11. Support for Captcha - You can add Captcha as an action to your WAFs. This helps you block unwanted bot traffic by requiring users to successfully complete challenges before their web request are allowed to reach AWS WAF protected resources. AWS WAF Captcha is available in the US East (N. Virginia), US West (Oregon), Europe (Frankfurt), South America (Sao Paulo), and Asia Pacific (Singapore) AWS Regions and supports Application Load Balancer, Amazon API Gateway, and AWS AppSync resources.
 
-13. Added S3LoggingBucketName to JSON. You need to specify the S3 Bucket where logs should be placed in. We also added a prefix for the logs to be AWS conform (Prefix: AWSLogs/*AWS_ACCOUNTID*/FirewallManager/*AWS_REGION*/).
+12. Added S3LoggingBucketName to Configuration. You need to specify the S3 Bucket where logs should be placed in. We also added a prefix for the logs to be AWS conform (Prefix: AWSLogs/*AWS_ACCOUNTID*/FirewallManager/*AWS_REGION*/).
 
-14. Added testing your WAF with [GoTestWAF](https://github.com/wallarm/gotestwaf). To be able to check your WAF we introduced the **SecuredDomain** parameter in the JSON (which should be your domain) which will be checked using the WAF tool.
+13. Added testing your WAF with [GoTestWAF](https://github.com/wallarm/gotestwaf). To be able to check your WAF we introduced the **SecuredDomain** parameter in the Configuration (which should be your domain) which will be checked using the WAF tool.
 
-15. TaskFileParameters:
+14. TaskFileParameters:
 
     |     Parameter      |                                           Value                                              |
     |--------------------|----------------------------------------------------------------------------------------------|
     | SKIP_QUOTA_CHECK   | true (Stop deployment if calculated WCU is above the quota) </br> false (Skipping WCU Check) |
     | WAF_TEST           | true (testing your waf with GoTestWAF) </br> false (Skipping WAF testing)                    |
     | CREATE_DIAGRAM     | true (generating a diagram using draw.io) </br> false (Skipping diagram generation)          |
-    | CDK_DIFF           | true (generating a cdk before invoking cdk deploy) </br> false (Skipping cdk diff)           |
+    | PREQUISITES        | true (deploys Prerequisites Stack) </br> false (deployment of WAF)           |
+    | TOOL_KIT_STACKNAME        | To Specify The name of the bootstrap stack ([see Bootstrapping your AWS environment](https://docs.aws.amazon.com/cdk/v2/guide/cli.html#cli-bootstrap))       |
 
-16. Validation of your ConfigFile using schema validation - if you miss a required parameter in your config file the deployment will stop automatically and show you the missing path.
+15. Validation of your ConfigFile using schema validation - if you miss a required parameter in your config file the deployment will stop automatically and show you the missing path.
 
-17. PreProcess- and PostProcessRuleGroups - you can decide now where the Custom or ManagedRules should be added to.
+16. PreProcess- and PostProcessRuleGroups - you can decide now where the Custom or ManagedRules should be added to.
 
-    - New Structure see [example json](./values/example-waf.json).
+    - New Structure see [example Configuration](./values/examples).
 
-18. RuleLabels - A label is a string made up of a prefix, optional namespaces and a name. The components of a label are delimited with a colon. Labels have the following requirements and characteristics:
+17. RuleLabels - A label is a string made up of a prefix, optional namespaces and a name. The components of a label are delimited with a colon. Labels have the following requirements and characteristics:
 
     - Labels are case-sensitive.
 
@@ -124,9 +123,9 @@ If you want to learn more about the AWS Firewall Factory feel free to look at th
 
     - Components of a label are separated by a colon ( : ).
 
-19. While Deployment the Price for your WAF will be calculated using the Pricing API
+18. While Deployment the Price for your WAF will be calculated using the Pricing API
 
-20. Dashboard - The Firewall Factory is able to provision a CloudWatch Dashboard per Firewall.
+19. Dashboard - The Firewall Factory is able to provision a CloudWatch Dashboard per Firewall.
   The Dashboard shows:
     - Where the WAF is deployed to [AWS Region and Account(s)]
     - Which resource type you are securing
@@ -141,7 +140,12 @@ If you want to learn more about the AWS Firewall Factory feel free to look at th
 See example:
 ![FirewallDashboard](./static/FirewallDashboard.jpg)
 
-21.   Centralized IPSets management -  No more we'll have to be manually updating ipsets across multiple AWS accounts, it can be defined in code and replicated for use by WAF rules everywhere its needed. Check the examples for defining ipsets and using them in the WebACLs on `values/ip-sets-managed.json`.
+20. Example Configurations
+    1.  Example WAF Configuration againts: [OWASP Top Ten](https://owasp.org/www-project-top-ten/)
+    2.  Example Configuration for Prerequisite Stack
+    3.  Function to generate Skeleton for WAF Configuration
+
+21.   Centralized IPSets management -  No more we'll have to be manually updating ipsets across multiple AWS accounts, it can be defined in code and replicated for use by WAF rules everywhere its needed. Check the examples for defining ipsets and using them in the WebACLs on `values/examples/ip-sets-managed-test.ts`.
 
 </div>
 </details>
@@ -186,12 +190,10 @@ See example:
 
 ### 🏁 Deployment via Taskfile
 
-1. Create new json file for you WAF and configure Rules in the JSON (see [owasptopten.json](values/owasptopten.json) to see structure)
-
-⚠️ When deploying a firewall for the first time, the **DeployHash** must be empty. The AWS Firewall Factory takes care of creating a DeployHash. The DeployHash is used to identify which CloudFormation stack is associated with which configuration file.
+1. Create new ts file for you WAF and configure Rules in the Configuration (see [owasptopten.ts](values/examples/owasptop10.ts) to see structure) or use enter `task generate-waf-skeleton`
 
-2. Assume AWS Profile `awsume PROFILENAME`
-3. (Optional) Enter `task generateconfig`
+2. Assume AWS Profile `awsume / assume PROFILENAME`
+3. (Optional) Enter `task generate-waf-skeleton`
 4. Enter `task deploy config=NAMEOFYOURCONFIGFILE`
 
 ## 🦸🏼‍♀️ Contributors

diff --git a/Taskfile.yml b/Taskfile.yml
@@ -3,8 +3,8 @@ version: '3'
 env:
   SKIP_QUOTA_CHECK: true
   WAF_TEST: false
-  CREATE_DIAGRAM: true
-  CDK_DIFF: false
+  CREATE_DIAGRAM: false
+  TOOL_KIT_STACKNAME: CDKToolkit
   LASTEST_FIREWALLFACTORY_VERSION:
     sh: curl --silent "https://api.github.com/repos/globaldatanet/aws-firewall-factory/releases/latest" | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/'
 vars:
@@ -18,60 +18,39 @@ tasks:
   deploy:
     desc: Deploy Stack
     cmds:
-      - task: clean
-      - task: cdkdiff
       - task: cdkdeploy
       - task: creatediagram
-      - task: testwaf
   synth:
     desc: synth Stack
     cmds:
-      - task: clean
       - cdk synth
     vars:
       ACCOUNT:
         sh: aws sts get-caller-identity |jq -r .Account
       TAGS:
         sh: cat tags/tags.json | jq -j '.[]|"--tags " + (.Key)+"="+(.Value)+" "'
     env:
-      PROCESS_PARAMETERS: values/{{.config}}.json  
-  clean:
-    desc: Clean CDK Out
-    cmds:
-      - rm -rf ./cdk.out
-      - rm -rf ./node-modules
-    silent: true
-  cdkdiff:
+      PROCESS_PARAMETERS: .config
+  diff:
     desc: CDK Diff
     cmds:
-      - if [[ {{.CDK_DIFF}} = true ]];then cdk diff; echo -n "Continue  (y/n)?"; read CONT; if [ "$CONT" = "n" ]; then echo "NO" | exit 1; else echo "YES"; fi; else exit 0;fi;
+      - cdk diff
     silent: true
     interactive: true
     env:
-      PROCESS_PARAMETERS: values/{{.config}}.json
+      PROCESS_PARAMETERS: "{{.config}}"
       PREREQUISITE: "{{.prerequisite}}"
-    preconditions:
-      - sh: "test {{.config}}"
-        msg: "config Variable was not set"
-      - sh: "test -f values/{{.config}}.json"
-        msg: "Values file not found: values/{{.config}}.json"
   cdkdeploy:
     desc: CDK Deploy
     cmds:
-      - cdk deploy --require-approval never {{.TAGS}}
-      - rm -f $(find ./cdk.out/*assets.json -type f) &> /dev/null
+      - cdk deploy --require-approval never {{.TAGS}} --toolkit-stack-name {{.TOOL_KIT_STACKNAME}}
     vars:
       ACCOUNT:
         sh: aws sts get-caller-identity |jq -r .Account
       TAGS:
         sh: cat tags/tags.json | jq -j '.[]|"--tags " + (.Key)+"="+(.Value)+" "'
-    preconditions:
-      - sh: "test {{.config}}"
-        msg: "config Variable was not set"
-      - sh: "test -f values/{{.config}}.json"
-        msg: "Values file not found: values/{{.config}}.json"
     env:
-      PROCESS_PARAMETERS: values/{{.config}}.json
+      PROCESS_PARAMETERS: "{{.config}}"
       PREREQUISITE: "{{.prerequisite}}"
     silent: true
     interactive: true
@@ -85,43 +64,19 @@ tasks:
   testwaf:
     desc: Test of your waf using GoTestWAF
     cmds:
+      - echo 🧪  Testing of your new 🔥 WAF using GoTestWAF
       - |
-        if [[ {{.WAF_TEST}} == true ]]; then
-          echo 🧪  Testing of your new 🔥 WAF using GoTestWAF
-
-          items=$(cat values/{{.config}}.json | jq -r '.[] | .SecuredDomain[]?')
-          for item in ${items[@]}; do
-              echo "Using fqdn in 🖥 url : $item"
-              ./gotestwaf/gotestwaf --url https://$item --workers 50 --blockConnReset  --wafName="$(cat values/{{.config}}.json | jq -r '.General.Prefix')-$(cat values/{{.config}}.json | jq -r '.WebAcl.Name')-$(cat values/{{.config}}.json | jq -r '.General.Stage')-$(cat values/{{.config}}.json | jq -r '.General.DeployHash')" --configPath=./gotestwaf/config.yaml --testCasesPath=./gotestwaf/testcases --skipWAFBlockCheck --reportPath "./waf-evaluation-report/$(date '+%Y-%m-%d')" --reportName "$(cat values/{{.config}}.json | jq -r '.General.Prefix')-$(cat values/{{.config}}.json | jq -r '.WebAcl.Name')-$(cat values/{{.config}}.json | jq -r '.General.Stage')-$(cat values/{{.config}}.json | jq -r '.General.DeployHash')"
-          done
-        else
-          echo "⏭ Skipping WAF Testing 🧪"
-        fi
+        items=$(cat values/{{.config}}.json | jq -r '.[] | .SecuredDomain[]?')
+        for item in ${items[@]}; do
+            echo "Using fqdn in 🖥 url : $item"
+            ./gotestwaf/gotestwaf --url https://$item --workers 50 --blockConnReset  --wafName="$(cat values/{{.config}}.json | jq -r '.General.Prefix')-$(cat values/{{.config}}.json | jq -r '.WebAcl.Name')-$(cat values/{{.config}}.json | jq -r '.General.Stage')-$(cat values/{{.config}}.json | jq -r '.General.DeployHash')" --configPath=./gotestwaf/config.yaml --testCasesPath=./gotestwaf/testcases --skipWAFBlockCheck --reportPath "./waf-evaluation-report/$(date '+%Y-%m-%d')" --reportName "$(cat values/{{.config}}.json | jq -r '.General.Prefix')-$(cat values/{{.config}}.json | jq -r '.WebAcl.Name')-$(cat values/{{.config}}.json | jq -r '.General.Stage')-$(cat values/{{.config}}.json | jq -r '.General.DeployHash')"
+        done
     silent: true
-  validateconfig:
-    desc: Validates the current config
-    cmds:
-      - ts-node test/config-loader.ts
-    silent: true
-    env:
-      PROCESS_PARAMETERS: values/{{.config}}.json
     preconditions:
-      - sh: "test {{.config}}"
-        msg: "config Variable was not set"
-      - sh: "test -f values/{{.config}}.json"
-        msg: "Values file not found: values/{{.config}}.json"
-  generateconfig:
-    desc: Generate skeleton for a waf configuration file
-    cmds:
-      - if [[ "{{.config}}" ]] ; then ts-node lib/tools/generate-skeleton.ts >> values/{{.config}}.json ; else ts-node lib/tools/generate-skeleton.ts ; fi
-    silent: true
-  generateowasptoptenconfig:
-    desc: Generate skeleton for a waf configuration file with owasp top 10
-    cmds:
-      - if [[ "{{.config}}" ]] ; then ts-node lib/tools/get-owasp-top10-waf.ts >> values/{{.config}}.json ; else ts-node lib/tools/get-owasp-top10-waf.ts ; fi
+      - sh: "[ '{{.WAF_TEST}}' != 'true' ]"
+        msg: ⏭  Skipping WAF Testing 🧪
+  generate-waf-skeleton:
+    desc: Generate Skeleton
     silent: true
-  generateprerequisitesconfig:
-    desc: Generate skeleton for a prerequisites configuration
     cmds:
-      - if [[ "{{.config}}" ]] ; then ts-node lib/tools/generate-prerequisites-skeleton.ts >> values/{{.config}}.json ; else ts-node lib/tools/generate-prerequisites-skeleton.ts ; fi
-    silent: true
+      - ts-node ./lib/tools/generate-skeleton.ts