-
Notifications
You must be signed in to change notification settings - Fork 26
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Began building YAML validation - Introduced a new pre-commit hook script `validate-yaml.sh` to validate all committed YAML files against a predefined schema using mage's `validateallyamlfiles` command. - Updated `.pre-commit-config.yaml` to include the new `validate-yaml-files` hook. - Added a new schema file `ttpforge-spec.yaml` under the `docs/` directory to define the structure and properties for the TTP YAML files. - Enhanced the `magefile.go` to include the new function `ValidateAllYAMLFiles` which uses the schema for validation. - Dependencies were updated in `magefiles/go.mod` and `magefiles/go.sum`. * Finished YAML validation pre-commit hook - Streamlined the YAML validation process by consolidating it into a shared function. - Improved error messaging for enhanced clarity. - Established a new function, loadSchema, to facilitate schema loading and unmarshalling from a YAML file. - Enhanced ValidateYAML to accommodate schema path input and extended support for directory-wide YAML file validation. - Revised the validateAllYAML function to furnish comprehensive error messages and enhanced logging. - Elevated error handling for YAML validation by providing informative error messages. - Augmented the inspectAndValidate function to adeptly manage nested SubTTPSteps and validate referenced files. - Implemented custom validation to ascertain the presence of the "name" property when "ttp" is declared in a step. This commit amplifies the effectiveness of YAML file validation, ensuring strict adherence to the schema. Introduces a pre-commit hook mechanism to preemptively thwart the commitment of non-conforming YAML files. * Bug fixes and QOL improvements - Fixed concurrency and infinite recursion bugs - Resolved issue with having set -e set - Optimized output to be useful * Fix incorrectly marked regex type in TTPForge schema * Refactor YAML validation logic from magefile.go to a new file schemaValidator.go. - Fixed failures to catch problems in editsteps and args * Fix incorrectly required item for EditStep
- Loading branch information
Showing
6 changed files
with
514 additions
and
61 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| #!/bin/bash | ||
| # This script is a pre-commit hook that checks if the mage command is | ||
| # installed and if not, prompts the user to install it. If mage is | ||
| # installed, the script changes to the repository root and runs the | ||
| # `mage validateyaml` command for each staged YAML file. This command | ||
| # validates all committed YAML files against a predefined schema. If any | ||
| # validation fails, the commit is stopped and an error message is shown. | ||
|
|
||
| # Change to the repository root | ||
| cd "$(git rev-parse --show-toplevel)" || exit 1 | ||
|
|
||
| # Determine the location of the mage binary | ||
| mage_bin=$(go env GOPATH)/bin/mage | ||
|
|
||
| # Check if mage is installed | ||
| if [[ ! -x "${mage_bin}" ]]; then | ||
| echo -e "mage is not installed\n" | ||
| echo -e "Please install mage by running the following command:\n" | ||
| echo -e "go install github.com/magefile/mage@latest\n" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Get the list of staged files under ttps directory and ending with .yaml | ||
| staged_files=$(git diff --cached --name-only --diff-filter=AM | grep '^ttps/.*\.yaml$') | ||
|
|
||
| if [[ -z "$staged_files" ]]; then | ||
| echo "This commit has no TTPs that need to be validated." | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "Files to validate: $staged_files" | ||
|
|
||
| # Iterate over each staged file and validate it | ||
| for file in $staged_files; do | ||
| # Run the mage validateyaml command for the staged YAML file | ||
| "${mage_bin}" validateyaml docs/ttpforge-spec.yaml "$file" || { | ||
| echo "Failed validation for: $file" | ||
| exit 1 | ||
| } | ||
|
|
||
| # Catch the exit code of the last command | ||
| exit_status=$? | ||
|
|
||
| # If the exit code is not zero (i.e., the command failed), | ||
| # then stop the commit and show an error message | ||
| if [ $exit_status -ne 0 ]; then | ||
| echo "Failed to validate YAML file '$file' against the schema." | ||
| exit 1 | ||
| fi | ||
| done |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| --- | ||
| definitions: | ||
| TTP: | ||
| type: object | ||
| properties: | ||
| name: | ||
| type: string | ||
| description: | ||
| type: string | ||
| mitre: | ||
| $ref: "#/definitions/Mitre" | ||
| steps: | ||
| type: array | ||
| items: | ||
| oneOf: | ||
| - $ref: "#/definitions/SubTTPStep" | ||
| - $ref: "#/definitions/BasicStep" | ||
| - $ref: "#/definitions/EditStep" | ||
| args: | ||
| type: array | ||
| items: | ||
| $ref: "#/definitions/Spec" | ||
| required: | ||
| - name | ||
| - description | ||
| - steps | ||
|
|
||
| Mitre: | ||
| type: object | ||
| properties: | ||
| tactics: | ||
| type: array | ||
| items: | ||
| type: string | ||
| techniques: | ||
| type: array | ||
| items: | ||
| type: string | ||
| subtechniques: | ||
| type: array | ||
| items: | ||
| type: string | ||
|
|
||
| BasicStep: | ||
| type: object | ||
| properties: | ||
| name: | ||
| type: string | ||
| inline: | ||
| type: string | ||
| cleanup: | ||
| $ref: "#/definitions/CleanupAct" | ||
| args: | ||
| type: array | ||
| items: | ||
| type: string | ||
| required: | ||
| - name | ||
| - inline | ||
|
|
||
| CleanupAct: | ||
| type: object | ||
| properties: | ||
| inline: | ||
| type: string | ||
| required: | ||
| - inline | ||
|
|
||
| SubTTPStep: | ||
| type: object | ||
| properties: | ||
| name: | ||
| type: string | ||
| ttp: | ||
| type: string | ||
| args: | ||
| type: object | ||
| additionalProperties: | ||
| type: string | ||
| required: | ||
| - name | ||
| - ttp | ||
|
|
||
| EditStep: | ||
| type: object | ||
| properties: | ||
| name: | ||
| type: string | ||
| edit_file: | ||
| type: string | ||
| backup_file: | ||
| type: string | ||
| edits: | ||
| type: array | ||
| items: | ||
| $ref: "#/definitions/Edit" | ||
| required: | ||
| - name | ||
| - edit_file | ||
| - backup_file | ||
| - edits | ||
|
|
||
| Edit: | ||
| type: object | ||
| properties: | ||
| old: | ||
| type: string | ||
| new: | ||
| type: string | ||
| regexp: | ||
| type: boolean | ||
| required: | ||
| - old | ||
| - new | ||
|
|
||
| Spec: | ||
| type: object | ||
| properties: | ||
| name: | ||
| type: string | ||
| type: | ||
| type: string | ||
| default: | ||
| type: string | ||
| description: | ||
| type: string | ||
| required: | ||
| - name | ||
| - type |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.