Skip to content

Commit

Permalink
Introduce Checkstyle integration test
Browse files Browse the repository at this point in the history
The new `integration-tests/checkstyle-10.12.4.sh` script can be executed
locally or by adding a comment containing the string `/integration-test`
to a pull request.

The integration test comprises a shell script that:
1. Checks out a Checkstyle release tag.
2. Applies a small patch to enable Error Prone Support.
3. Runs the build in Error Prone patch mode, until no further changes
   are applied.
4. Validates that:
   - The build (including tests) still passes.
   - The set of changes matches a pre-recorded diff.
   - The set of Error Prone Support-emitted warnings matches a
     pre-recorded list.

The script also has a `--sync` mode using which the expected diff and
warnings can be updated. Combined, the script makes it easy to assess
and track the impact of changes to Error Prone Support in relation to
the Checkstyle code base.

Over time this setup will be generalized to include integration tests
against other code bases.
  • Loading branch information
Stephan202 committed Oct 29, 2023
1 parent 8886080 commit 3b2ebf6
Show file tree
Hide file tree
Showing 6 changed files with 64,299 additions and 0 deletions.
32 changes: 32 additions & 0 deletions .github/workflows/run-integration-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# If requested by means of a pull request comment, runs integration tests
# against the project, using the code found on the pull request branch.
# XXX: Generalize this to a matrix build of multiple integration tests,
# possibly using multiple JDK or OS versions.
name: "Integration tests"
on:
issue_comment:
types: [ created ]
permissions:
contents: read
jobs:
run-integration-tests:
name: On-demand integration test
if: |
github.event.issue.pull_request && contains(github.event.comment.body, '/integration-test')
runs-on: ubuntu-22.04
steps:
- name: Check out code
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
persist-credentials: false
ref: refs/pull/${{ github.event.issue.number }}/head
- name: Set up JDK
uses: actions/setup-java@0ab4596768b603586c0de567f2430c30f5b0d2b0 # v3.13.0
with:
java-version: 17.0.8
distribution: temurin
cache: maven
- name: Install project to local Maven repository
run: mvn -T1C install -DskipTests -Dverification.skip
- name: Run integration test
run: xvfb-run ./integration-tests/checkstyle-10.12.4.sh
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,6 @@
target
*.iml
*.swp

# The Git repositories checked out by the integration test framework.
integration-tests/.repos
Loading

0 comments on commit 3b2ebf6

Please sign in to comment.