Skip to content

Commit

Permalink
fix: Fix server side skip test cases (#36572)
Browse files Browse the repository at this point in the history
## Description
> [!TIP]  
> _Add a TL;DR when the description is longer than 500 words or
extremely technical (helps the content, marketing, and DevRel team)._
>
> _Please also include relevant motivation and context. List any
dependencies that are required for this change. Add links to Notion,
Figma or any other documents that might be relevant to the PR._


Fixes #`Issue Number`  
_or_  
Fixes `Issue URL`
> [!WARNING]  
> _If no issue exists, please create an issue first, and check with the
maintainers if the issue is valid._

## Automation

/ok-to-test tags="@tag.IDE"

### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results  -->
> [!TIP]
> 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/11370534880>
> Commit: dea75ff
> <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11370534880&attempt=1"
target="_blank">Cypress dashboard</a>.
> Tags: `@tag.IDE`
> Spec:
> <hr>Wed, 16 Oct 2024 17:40:56 UTC
<!-- end of auto-generated comment: Cypress test results  -->


## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [x] No


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Bug Fixes**
- Improved handling and reporting of test failures during the build
process.
	- Enhanced logging for better clarity on test results.

- **New Features**
- Introduced detailed output for failed and skipped tests, now available
in the GitHub step summary and as comments on pull requests.
- Added new input parameter `is-pg-build` for enhanced workflow control.
- Updated default value for `skip-tests` to "false" to enforce test
execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
  • Loading branch information
sagar-qa007 authored Oct 17, 2024
1 parent 7ff703c commit 81b2146
Showing 1 changed file with 95 additions and 30 deletions.
125 changes: 95 additions & 30 deletions .github/workflows/server-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -178,39 +178,104 @@ jobs:
APPSMITH_ENVFILE_PATH: /tmp/dummy.env
APPSMITH_VERBOSE_LOGGING_ENABLED: false
run: |
if [[ "${{ inputs.is-pg-build }}" == "true" ]]; then
export APPSMITH_DB_URL="postgresql://postgres:password@localhost:5432/postgres"
else
export APPSMITH_DB_URL="mongodb://localhost:27017/mobtools"
fi
args=()
if [[ "${{ steps.run_result.outputs.run_result }}" == "failedtest" ]]; then
failed_tests="${{ steps.failed_tests.outputs.tests }}"
args+=("-DfailIfNoTests=false" "-Dsurefire.failIfNoSpecifiedTests=false" "-Dtest=${failed_tests}")
fi
if ! mvn test "${args[@]}"; then
echo "Generating failed test list:"
failed_tests_file="$PWD/failed-server-tests.txt"
if [[ "${{ inputs.is-pg-build }}" == "true" ]]; then
export APPSMITH_DB_URL="postgresql://postgres:password@localhost:5432/postgres"
else
export APPSMITH_DB_URL="mongodb://localhost:27017/mobtools"
fi
args=()
if [[ "${{ steps.run_result.outputs.run_result }}" == "failedtest" ]]; then
failed_tests="${{ steps.failed_tests.outputs.tests }}"
args+=("-DfailIfNoTests=false" "-Dsurefire.failIfNoSpecifiedTests=false" "-Dtest=${failed_tests}")
fi
# Run tests and capture logs
mvn test "${args[@]}" | tee mvn_test.log
# Check for "BUILD FAILURE" in the mvn_test.log
if grep -q "BUILD FAILURE" mvn_test.log; then
test_result="failed"
else
test_result="passed"
fi
echo "test_result variable value: ${test_result}"
# Prepare output file for failed tests and ensure a fresh file is created
OUTPUT_FILE="failed-server-tests.txt"
rm -f "$OUTPUT_FILE"
touch "$OUTPUT_FILE"
failed_modules=()
skipped_modules=()
# Process mvn_test.log for FAILURE and SKIPPED statuses
while IFS= read -r line; do
if [[ $line == *"FAILURE"* ]]; then
module_name=$(echo "$line" | awk '{print $2}')
failed_modules+=("$module_name")
elif [[ $line == *"SKIPPED"* ]]; then
module_name=$(echo "$line" | awk '{print $2}')
skipped_modules+=("$module_name")
fi
done < mvn_test.log
echo "Failed Modules: ${failed_modules[*]}"
echo "Skipped Modules: ${skipped_modules[*]}"
# Handle older approach for reading failed tests from XML files
failed_tests_from_xml="$PWD/failed-tests-from-xml.txt"
gawk -F\" '/<testcase / {cur_test = $4 "#" $2} /<(failure|error) / {print cur_test}' $(find . -type f -name 'TEST-*.xml') \
| sort -u \
| tee "$failed_tests_file"
echo "Saved to $failed_tests_file"
if [[ -s $failed_tests_file ]]; then
content="$(
echo "## Failed server tests"
echo
sed 's/^/- /' "$failed_tests_file"
)"
echo "$content" >> "$GITHUB_STEP_SUMMARY"
# Add a comment to the PR with the list of failed tests.
curl --silent --show-error \
--header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
--data "$(jq -n --arg body "$content" '$ARGS.named')" \
"https://api.github.com/repos/$GITHUB_REPOSITORY/issues/${{ inputs.pr }}/comments" \
> /dev/null
| tee "$failed_tests_from_xml"
# Filter out failed modules and add only relevant tests to the final failed list
for module in "${failed_modules[@]}"; do
grep -v "$module" "$failed_tests_from_xml" > temp_file && mv temp_file "$failed_tests_from_xml"
done
# Include all skipped module test files in the final list
for module in "${skipped_modules[@]}"; do
module_directories=$(find . -path "*/${module}*/src/test/java/*" -type f -name "*Test.java" -exec dirname {} \; | sort -u)
for module_directory in $module_directories; do
test_classes=$(find "$module_directory" -type f -name "*Test.java" | sed 's|.*/src/test/java/||; s|\.java$||; s|/|.|g')
for class_name in $test_classes; do
if [[ ${#class_name} -le 240 ]] && ! grep -Fxq "$class_name#" "$OUTPUT_FILE"; then
echo "${class_name}#" >> "$OUTPUT_FILE"
fi
done
done
done
# Combine the XML file test cases and skipped module test files into the final output file
cat "$failed_tests_from_xml" >> "$OUTPUT_FILE"
# Print the final output
cat "$OUTPUT_FILE"
if [[ -s $OUTPUT_FILE ]]; then
content="$(
echo "## Failed server tests"
echo
sed 's/^/- /' "$OUTPUT_FILE"
)"
echo "$content" >> "$GITHUB_STEP_SUMMARY"
# Post a comment to the PR
curl --silent --show-error \
--header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
--data "$(jq -n --arg body "$content" '$ARGS.named')" \
"https://api.github.com/repos/$GITHUB_REPOSITORY/issues/${{ inputs.pr }}/comments" \
> /dev/null
fi
exit 1
fi
# Fail the script if tests did not pass
if [[ "$test_result" == "failed" ]]; then
echo "Tests failed, exiting with status 1."
exit 1
fi
# Set status = failedtest
- name: Set fail if there are test failures
Expand Down

0 comments on commit 81b2146

Please sign in to comment.