Introduce generic `run-integration-test.sh` script #1141

rickie · 2024-04-17T06:09:21Z

This PR aims to split out some of the changes from #894 to minimize the diff of that PR.
It's introducing too many changes to review at once. After we merged this PR, it'll be easier to review the other.

Credits to @oxkitsune for these changes!

rickie · 2024-04-17T06:09:42Z

/integration-test

github-actions · 2024-04-17T06:11:32Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-04-17T06:11:41Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

sonarqubecloud · 2024-04-17T06:11:44Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

mohamedsamehsalah

Clean 🧹

github-actions · 2024-05-24T11:26:56Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-06-11T10:14:56Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

sonarqubecloud · 2024-06-11T10:15:07Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

github-actions · 2024-07-14T12:14:20Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Stephan202

Rebased and added a commit. One question remaining!

Stephan202 · 2024-08-04T13:33:51Z

integration-tests/run-integration-test.sh

+    || (
+         git -C "${project_root}" fetch --depth 1 "${repository}" "${revision}" \
+           && git -C "${project_root}" checkout --force FETCH_HEAD \
+           && (git -C "${project_root}" tag "${revision}" --message "${revision}" || true)


Do we understand why --message "${revision}" is now necessary here/is there an easy way for me to see what breaks if this is omitted? 👀

(Looking at git tag --help, I suspect this has to do with a tag.gpgSign user setting.)

Yeah I have the gpgSign setup. Before adding this, I was required to pass in a message (via vim in the commandline) myself before it would continue with running. But instead we can also say --no-sign perhaps?

Hmm, I ran it again without the --message "{revision}" and it works for me... I have a new PC setup since the last time I worked on this, so maybe it's slightly different now?

Will drop it.

Stephan202 · 2024-08-04T13:41:19Z

integration-tests/run-integration-test.sh

+# are behavior preserving. Some tests are skipped:
+# - The `metadataFilesGenerationAllFiles` test is skipped because it makes line
+#   number assertions that will fail when the code is formatted or patched.
+# - The `allCheckSectionJavaDocs` test is skipped because is validates that
+#   Javadoc has certain closing tags that are removed by Google Java Format.


This documentation must now live elsewhere.

Stephan202 · 2024-08-04T13:45:38Z

integration-tests/run-integration-test.sh

+mvn ${shared_build_flags} \
+      clean package \
+      -Derror-prone.configuration-args="${error_prone_validation_flags}" \
+      ${validation_mvn_flags} \


Above we use shared_build_flags, so I'd say that this property should then be names validation_build_flags.

Stephan202 · 2024-08-04T13:58:51Z

integration-tests/run-integration-test.sh

+    ;;
+esac
+
+# XXX: Configure Renovate to manage the AssertJ version declared here.


Suggested change

# XXX: Configure Renovate to manage the AssertJ version declared here.

Stephan202 · 2024-08-04T14:10:50Z

integration-tests/run-integration-test.sh

+repos_root="${integration_test_root}/.repos"
+
+if [ "${#}" -ne 11 ]; then
+  >&2 echo "Usage $(basename "${0}") [TestName] [Project] [Repository] [Revision] [BuildFlags] [AdditionalSourceDirectories] [PatchFlags] [ValidationEpFlags] [ValidationMvnFlags] [DoSync] [ReportDirectory]"


Colon missing after "Usage". 👀

The parameters aren't optional, so we should use angled brackets, rather than square brackets.

Stephan202 · 2024-08-04T14:11:26Z

integration-tests/run-integration-test.sh

+test_name="${1}"
+project="${2}"
+repository="${3}"
+revision="${4}"
+build_flags="${5}"
+additional_src_directories="${6}"
+patch_flags="${7}"
+validation_ep_flags="${8}"
+validation_mvn_flags="${9}"
+do_sync="${10}"
+report_directory="${11}"


The naming here isn't very consistent; will propose some things.

Stephan202 · 2024-08-04T14:15:05Z

integration-tests/run-integration-test.sh

+   find "${error_prone_support_root}" \
+      -path "*/META-INF/services/com.google.errorprone.bugpatterns.BugChecker" \
+      -not -path "*/error-prone-experimental/*" \
+      -not -path "*/error-prone-guidelines/*" \
+      -print0 \


Indentation partially corrected here, but not fully. Should do the same below.

Stephan202 · 2024-08-04T15:59:33Z

integration-tests/checkstyle.sh

 do_sync="$([ "${#}" = 0 ] || [ "${1:-}" != '--sync' ] || echo 1)"
 report_directory="$([ "${#}" = 0 ] || ([ -z "${do_sync}" ] && echo "${1}") || ([ "${#}" = 1 ] || echo "${2}"))"


We can partially deduplicate this ^ logic too; will propose something.

github-actions · 2024-08-04T16:20:17Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Stephan202 · 2024-08-04T16:20:21Z

Suggested commit message:

Introduce generic `run-integration-test.sh` script (#1141)

This new script contains reusable logic extracted from
`integration-tests/checkstyle.sh`, facilitating the introduction of additional
integration tests.

github-actions · 2024-08-07T08:01:42Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Stephan202

Let's go! 🚀

sonarqubecloud · 2024-08-07T18:09:28Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2024-08-07T18:09:38Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

rickie added the improvement label Apr 17, 2024

rickie added this to the 0.17.0 milestone Apr 17, 2024

rickie requested review from Stephan202 and mohamedsamehsalah April 17, 2024 06:09

rickie force-pushed the rossendrijver/extract_running_integration_test branch from b61b4b5 to 5d15247 Compare April 17, 2024 06:09

mohamedsamehsalah approved these changes Apr 22, 2024

View reviewed changes

rickie force-pushed the rossendrijver/extract_running_integration_test branch from 5d15247 to be79b3b Compare May 24, 2024 11:24

rickie force-pushed the rossendrijver/extract_running_integration_test branch from be79b3b to 0c9e563 Compare June 11, 2024 10:12

rickie force-pushed the rossendrijver/extract_running_integration_test branch from 0c9e563 to ee1b4b4 Compare July 14, 2024 12:12

Stephan202 modified the milestones: 0.17.0, 0.18.0 Jul 20, 2024

Stephan202 reviewed Aug 4, 2024

View reviewed changes

Stephan202 force-pushed the rossendrijver/extract_running_integration_test branch from ee1b4b4 to 03d1a70 Compare August 4, 2024 16:18

Stephan202 added chore A task not related to code (build, formatting, process, ...) and removed improvement labels Aug 4, 2024

rickie requested a review from Stephan202 August 7, 2024 13:42

Stephan202 approved these changes Aug 7, 2024

View reviewed changes

rickie added 2 commits August 7, 2024 20:07

Introduce script to run integration test

46955e0

Improve docs

85103de

Stephan202 and others added 2 commits August 7, 2024 20:07

Suggestions

5264ede

Drop message arg for tagging

342bf12

Stephan202 force-pushed the rossendrijver/extract_running_integration_test branch from b837751 to 342bf12 Compare August 7, 2024 18:07

Stephan202 merged commit c322ea1 into master Aug 7, 2024
15 checks passed

Stephan202 deleted the rossendrijver/extract_running_integration_test branch August 7, 2024 21:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce generic `run-integration-test.sh` script #1141

Introduce generic `run-integration-test.sh` script #1141

rickie commented Apr 17, 2024

rickie commented Apr 17, 2024

github-actions bot commented Apr 17, 2024

github-actions bot commented Apr 17, 2024

sonarqubecloud bot commented Apr 17, 2024

mohamedsamehsalah left a comment

github-actions bot commented May 24, 2024

github-actions bot commented Jun 11, 2024

sonarqubecloud bot commented Jun 11, 2024

github-actions bot commented Jul 14, 2024

Stephan202 left a comment

Stephan202 Aug 4, 2024

rickie Aug 7, 2024

Stephan202 Aug 4, 2024

Stephan202 Aug 4, 2024

Stephan202 Aug 4, 2024

Stephan202 Aug 4, 2024

Stephan202 Aug 4, 2024

Stephan202 Aug 4, 2024

Stephan202 Aug 4, 2024

github-actions bot commented Aug 4, 2024

Stephan202 commented Aug 4, 2024

github-actions bot commented Aug 7, 2024

Stephan202 left a comment

sonarqubecloud bot commented Aug 7, 2024

github-actions bot commented Aug 7, 2024

		do_sync="$([ "${#}" = 0 ] \|\| [ "${1:-}" != '--sync' ] \|\| echo 1)"
		report_directory="$([ "${#}" = 0 ] \|\| ([ -z "${do_sync}" ] && echo "${1}") \|\| ([ "${#}" = 1 ] \|\| echo "${2}"))"

Introduce generic run-integration-test.sh script #1141

Introduce generic run-integration-test.sh script #1141

Conversation

rickie commented Apr 17, 2024

rickie commented Apr 17, 2024

github-actions bot commented Apr 17, 2024

github-actions bot commented Apr 17, 2024

sonarqubecloud bot commented Apr 17, 2024

Quality Gate passed

mohamedsamehsalah left a comment

Choose a reason for hiding this comment

github-actions bot commented May 24, 2024

github-actions bot commented Jun 11, 2024

sonarqubecloud bot commented Jun 11, 2024

Quality Gate passed

github-actions bot commented Jul 14, 2024

Stephan202 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Aug 4, 2024

Stephan202 commented Aug 4, 2024

github-actions bot commented Aug 7, 2024

Stephan202 left a comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Aug 7, 2024

Quality Gate passed

github-actions bot commented Aug 7, 2024

Introduce generic `run-integration-test.sh` script #1141

Introduce generic `run-integration-test.sh` script #1141