Skip to content

Commit

Permalink
chore(oscal): begin integration of composed oscal with validations (#496
Browse files Browse the repository at this point in the history
)

## Description

This change proposes the integration of
[Lula](https://github.com/defenseunicorns/lula) with uds-core for the
purposes of compliance assessment on Pull-Request. This Proposal
includes:
- all proposed future changes meet the minimum threshold for compliance
- framework established for iterating control body-of-evidence
- process for consuming uds-core compliance for downstream utilization

### How does it work?
Each component that comprises `uds-core` contains an
[OSCAL](https://pages.nist.gov/OSCAL/about/) file in the form of the
`component-definition` model. This captures 1 -> N controls for 1 -> N
standards (control implementations) that pertain to a specific
component.

When controls are identified as implemented by a component, we can
create or update a `component-definition` artifact through:
```bash
lula generate component -c <catalog-source> -r <controls-comma-delimited> --component "component name>
```

#### Validation
Lula operates on a model where controls are marked as satisfied when
presented with programmatic and repeatable processes to to produce
evidence that the control is satisfied.

These are often developed on a control-by-control basis and then
composed into a single OSCAL component-definition in a schema-compliant
addition in the back-matter. Reducing required network connectivity and
making the compliance information portable.

One control may require evidence from many `validations`. One
`validation` is the collection of data from some source and the
measurement of that data for required adherence.

The `lula validate` operation is an objective assessment - determining
control `satisfied/not-satisfied` state and writing the results to the
`assessment-results` artifact.

#### Evaluation

The last step for determining greater, equal, or lesser compliance than
the previous state is to execute a `lula evaluate`. This consumes the
latest `result` in the `assessment-results` artifact and compares it
against the `threshold` result.

The `threshold` result is identified as the `result` containing the
following prop:
```yaml
- name: threshold
   ns: https://docs.lula.dev/ns
   value: "true"
```

If the compliance of this latest result is worse than the threshold -
the execution fails with a non-zero exit code - allowing for pipeline
operations to fail.

If the compliance is equal - the command passes without any modification
required.

If the compliance is greater - Lula will automatically move the
threshold identifier to the applicable result item. (this would mean
that the updated `assessment-results` artifact should be downloaded and
submitted for update to the repository).

#### Naming

All files are titled with "oscal-" prepended to the OSCAL model type:
- oscal-component.yaml -> component definition
- oscal-assessment-results.yaml -> assessment results

This is primarily to allow for identification for linting workflows to
target oscal files for Lula to Lint.

### Metrics

The following are derived from the FedRAMP Moderate baseline:
```
24 controls of *329 total* mapped in OSCAL from Istio (7.29%)
24 controls of *161 total Technical* mapped in OSCAL from Istio (14.90%)
75 Unique controls captured in OSCAL of *329 total* (22.79%)
10 controls marked satisfied in OSCAL of *329 total* (3.03%)
75 Unique controls captured in OSCAL of *161 technical* (46.58%)
10 controls marked satisfied in OSCAL of *161 technical* (6.21%)
```

> [!NOTE]  
> DISA has recently published a new revision to the Cloud Computing
Security Requirements Guide which may result in future iterations to a
small subset of controls when reconciled.

[DISA link](https://public.cyber.mil/dccs/dccs-documents/)

### Review

The Lula team is actively working to make the review process better.
Composed OSCAL has it's benefits for portability but that comes with the
drawbacks of review.

To assist with the review process:
- Ensure the pipeline additions are sound
- Review the `src/istio/oscal-component.yaml` file. This will map
controls to the current state of validations.
- Understand the connection of `compliance/oscal-component.yaml` to the
individual components under `src/`

### Troubleshooting

If the pipeline should begin failing in the future - notify the
@defenseunicorns/lula-dev team.

It should be expected that any major revisions to applications could
break known assumptions about configuration required for compliance. The
Lula team can help navigate failures.

## Related Issue

Resolves #458 

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Other (security config, docs update, etc)

## Checklist before merging

- [ ] Test, docs, adr added or updated as needed
- [ ] [Contributor Guide
Steps](https://github.com/defenseunicorns/uds-template-capability/blob/main/CONTRIBUTING.md)(https://github.com/defenseunicorns/uds-template-capability/blob/main/CONTRIBUTING.md#submitting-a-pull-request)
followed

---------

Co-authored-by: Andy Mills <[email protected]>
Co-authored-by: Andy Mills <[email protected]>
Co-authored-by: Micah Nagel <[email protected]>
  • Loading branch information
4 people authored Jul 19, 2024
1 parent 30cbae4 commit 047fd30
Show file tree
Hide file tree
Showing 12 changed files with 2,944 additions and 645 deletions.
51 changes: 51 additions & 0 deletions .github/actions/notify-lula/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Notify Lula
description: "Comment on PR to notify Lula Team"


inputs:
state:
description: 'state of the comment update'
required: true
default: ''
flavor:
description: 'flavor of the comment update'
required: true
default: ''
ghToken:
description: 'GITHUB_TOKEN'
required: true

runs:
using: composite
steps:
- name: Find Comment
uses: peter-evans/find-comment@3eae4d37986fb5a8592848f6a574fdf654e61f9e # v3.1.0
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: Compliance ${{ inputs.flavor }} Evaluation
token: ${{ inputs.ghToken }}

- name: Create comment
if: ${{ steps.fc.outputs.comment-id == '' && inputs.state == 'failure'}}
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
with:
issue-number: ${{ github.event.pull_request.number }}
token: ${{ inputs.ghToken }}
body: |
Compliance ${{ inputs.flavor }} Evaluation: ${{ inputs.state }}
CC: @defenseunicorns/lula-dev
- name: Update comment
if: ${{ steps.fc.outputs.comment-id != '' }}
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
token: ${{ inputs.ghToken }}
edit-mode: replace
body: |
Compliance ${{ inputs.flavor }} Evaluation: ${{ inputs.state }}
CC: @defenseunicorns/lula-dev
6 changes: 6 additions & 0 deletions .github/actions/setup/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ runs:
# renovate: datasource=github-tags depName=defenseunicorns/uds-cli versioning=semver
run: brew install defenseunicorns/tap/[email protected]

- name: Install Lula
uses: defenseunicorns/lula-action/setup@095636b7880051e11b05f10a582fdd911526161c
with:
# renovate: datasource=github-tags depName=defenseunicorns/lula versioning=semver-coerced
version: v0.4.1

- name: Iron Bank Login
if: ${{ inputs.registry1Username != '' }}
env:
Expand Down
75 changes: 75 additions & 0 deletions .github/workflows/compliance.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Compliance Evaluation

on:
# Manual trigger
workflow_dispatch:
inputs:
flavor:
type: string
description: "Flavor of the source package to test"
required: true
# Triggered by pull-request-conditionals.yaml
workflow_call:
inputs:
flavor:
type: string
description: "Flavor of the source package to test"
required: true

permissions:
contents: read
pull-requests: write

jobs:
evaluate:
runs-on: ubuntu-latest
name: Evaluate
continue-on-error: true
# env:
# UDS_PKG: ${{ inputs.package }}
steps:
# Used to execute the uds run command
- name: Checkout repository
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- name: Environment setup
uses: ./.github/actions/setup

- name: review compliance directory
run: ls -al ./compliance/
shell: bash

- name: remove overlapping file
run: rm ./compliance/oscal-assessment-results.yaml
shell: bash

- name: Download assessment
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: ${{ inputs.flavor }}-assessment-results
path: ./compliance

- name: review compliance directory again
run: ls -al ./compliance/
shell: bash

- name: Evaluate compliance
id: compliance-evaluation
run: uds run test-compliance-evaluate --no-progress

# steps in this action only run when there has been a previous failure - will indicate success thereafter
# need to think about how much noise this could create - noise currently = good
- name: Notify Lula Team of Compliance Assessment Results
if: ${{ always() }}
uses: ./.github/actions/notify-lula
with:
state: ${{ steps.compliance-evaluation.outcome }}
flavor: ${{ inputs.flavor }}
ghToken: ${{ secrets.GITHUB_TOKEN }}

- name: Upload Evaluated Assessment
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: ${{ inputs.flavor }}-assessment-results
path: ./compliance/oscal-assessment-results.yaml
overwrite: true
56 changes: 56 additions & 0 deletions .github/workflows/lint-oscal.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Lint OSCAL Files

on:
pull_request:
# milestoned is added here as a workaround for release-please not triggering PR workflows (PRs should be added to a milestone to trigger the workflow).
types: [milestoned, opened, reopened, synchronize]
paths:
- '**/*oscal*.yaml'

permissions:
contents: read

jobs:

check-oscal-paths:
runs-on: ubuntu-latest
name: OSCAL Change Detection
outputs:
oscal: ${{ steps.path-filter.outputs.oscal }}
oscal_files: ${{ steps.path-filter.outputs.oscal_files }}
steps:
- name: Checkout the code
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

# Uses a custom action to filter paths for source packages.
- name: Check src paths
id: path-filter
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3
with:
filters: |
oscal:
- added|modified: "**/*oscal*.yaml"
list-files: shell

lint-oscal:
needs: check-oscal-paths
if: ${{ needs.check-oscal-paths.outputs.oscal == 'true' }}
runs-on: ubuntu-latest
steps:
# filter the files to remove not oscal files (such as those titles oscal-* under ./.github)
- name: Identify changed OSCAL files
id: find_changed_files
run: |
CHANGED_FILES=$(echo "${{ needs.check-oscal-paths.outputs.oscal_files }}" | tr ' ' '\n' | grep -v ".github*" | tr '\n' ',' | sed 's/.$//' || true)
echo "Changed OSCAL files: $CHANGED_FILES"
echo "oscal_files=$CHANGED_FILES" >> "$GITHUB_OUTPUT"
shell: bash
# checkout for access to the oscal files targeted for linting
- name: Checkout the code
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Environment setup
uses: ./.github/actions/setup
# lint the oscal files
- name: lint-oscal
run: uds run lint-oscal --set OSCALFILES=${{ steps.find_changed_files.outputs.oscal_files }} --no-progress
shell: bash
14 changes: 13 additions & 1 deletion .github/workflows/pull-request-conditionals.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:
permissions:
id-token: write # Needed for OIDC-related operations.
contents: read # Allows reading the content of the repository.
pull-requests: read # Allows reading pull request metadata.
pull-requests: write # Allows writing pull request metadata.
packages: read # Allows reading the published GHCR packages

# Default settings for all run commands in the workflow jobs.
Expand Down Expand Up @@ -86,3 +86,15 @@ jobs:
flavor: ${{ matrix.flavor }}
test_type: ${{ matrix.test_type }}
secrets: inherit # Inherits all secrets from the parent workflow.

evaluate-package-compliance:
needs: run-package-test
name: Compliance Evaluation
strategy:
matrix:
flavor: [upstream, registry1, unicorn]
fail-fast: false
uses: ./.github/workflows/compliance.yaml
with:
flavor: ${{ matrix.flavor }}
secrets: inherit # Inherits all secrets from the parent workflow.
11 changes: 11 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,17 @@ jobs:
if: ${{ inputs.package == 'all' && inputs.test_type == 'install' }}
run: uds run test-uds-core --set FLAVOR=${{ inputs.flavor }} --no-progress

- name: Validate UDS Core Compliance
if: ${{ inputs.package == 'all' && inputs.test_type == 'install' }}
run: uds run test-compliance-validate --no-progress

- name: Upload Assessment
if: ${{ inputs.package == 'all' && inputs.test_type == 'install' }}
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: ${{ inputs.flavor }}-assessment-results
path: ./compliance/oscal-assessment-results.yaml

- name: Test UDS Core Upgrade
if: ${{ inputs.package == 'all' && inputs.test_type == 'upgrade' }}
run: uds run test-uds-core-upgrade --set FLAVOR=${{ inputs.flavor }} --no-progress
Expand Down
Loading

0 comments on commit 047fd30

Please sign in to comment.