Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backport 2.28: Record the outcome of each test case in compat.sh #7897

Merged
merged 4 commits into from
Jul 10, 2023
Merged

Backport 2.28: Record the outcome of each test case in compat.sh #7897

merged 4 commits into from
Jul 10, 2023

Conversation

tgonzalezorlandoarm
Copy link
Contributor

@tgonzalezorlandoarm tgonzalezorlandoarm commented Jul 7, 2023 •
edited by yanrayw
Loading

Description

Backport of PR #7136

PR checklist

Please tick as appropriate and edit the reasons (e.g.: "backport: not needed because this is a new feature")

Notes for the submitter

Please refer to the contributing guidelines, especially the
checklist for PR contributors.

Yanray Wang added 3 commits July 7, 2023 11:34
@tgonzalezorlandoarm
compat.sh: call record_fail if test case fails
0f386b7 
Signed-off-by: Yanray Wang <[email protected]>
@tgonzalezorlandoarm
compat.sh: add support to record outcome of test cases
44ba969 
If the environment variable MBEDTLS_TEST_OUTCOME_FILE is set,
the test outcome file records each test case in a single line
with the format of

 PLATFORM;CONFIGURATION;compat;TEST CASE DESCRIPTION;RESULT;[CAUSE]

- CONFIGURATION comes from MBEDTLS_TEST_CONFIGURATION to record
  configuration of each test case
- PLATFORM is either set by users or calculated from test
  platform
- RESULT is one of PASS, FAIL or SKIP. If test case fails,
  srv_out/cli_out follows as FAILURE CAUSE.

Signed-off-by: Yanray Wang <[email protected]>
@tgonzalezorlandoarm
test-framework.md: document compat.sh
31d5046 
Signed-off-by: Yanray Wang <[email protected]>
@tgonzalezorlandoarm tgonzalezorlandoarm self-assigned this Jul 7, 2023
@tgonzalezorlandoarm
Copy link
Contributor Author

@yanrayw This is the requested backport. In 2.28 mbedtls_config.h does not exist. I hope I dealt with this correctly, please let me know otherwise.

@daverodgman
Copy link
Contributor

I hope I dealt with this correctly

Please remember to set labels on your PRs - see https://confluence.arm.com/pages/viewpage.action?pageId=562698953

@tgonzalezorlandoarm tgonzalezorlandoarm added needs-review Every commit must be reviewed by at least two team members, size-xs Estimated task size: extra small (a few hours at most) priority-medium Medium priority - this can be reviewed as time permits labels Jul 7, 2023
@tgonzalezorlandoarm tgonzalezorlandoarm changed the title Backport PR 7136 to mbedtls-2.28 Backport 2.28: Record the outcome of each test case in compat.sh Jul 7, 2023
@yanrayw yanrayw marked this pull request as ready for review July 10, 2023 02:53
@yanrayw yanrayw mentioned this pull request Jul 10, 2023
Merged
3 tasks
yanrayw
yanrayw suggested changes Jul 10, 2023
View reviewed changes
Copy link

@yanrayw yanrayw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me except a minor issue.

tests/ssl-opt.sh Outdated Show resolved Hide resolved
@tgonzalezorlandoarm
ssl-opt.sh: support to parse --outcome-file
5659981 
Adjust where to set absolute path for MBEDTLS_TEST_OUTCOME_FILE
as it's supposed to set its absolute path after all possible
value assignment.

Signed-off-by: Yanray Wang <[email protected]>
yanrayw
yanrayw approved these changes Jul 10, 2023
View reviewed changes
Copy link

@yanrayw yanrayw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for doing the backport.

@lpy4105
Copy link
Contributor

lpy4105 commented Jul 10, 2023

LGTM, thanks!

tom-daubney-arm
tom-daubney-arm approved these changes Jul 10, 2023
View reviewed changes
Copy link
Contributor

@tom-daubney-arm tom-daubney-arm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Faithful backport, LGTM.

@tom-daubney-arm
Copy link
Contributor

tom-daubney-arm commented Jul 10, 2023 •
edited
Loading

@lpy4105 It looks like you have just left a comment rather than submitting it as a review officially. I am not sure if @yanrayw is technically allowed to review given he authored the original PR, so could you please officially submit as a review, and then we can mark it approved. (Also, please let me know if I am wrong about this...)

I am taking the CI as green here. It looks like the internal CI is being a bit slow but the equivalent components on OpenCI have passed, so I think we're all good. Edit: CI now all passed.

@daverodgman daverodgman merged commit 814a14a into Mbed-TLS:mbedtls-2.28 Jul 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yanrayw yanrayw yanrayw approved these changes

@tom-daubney-arm tom-daubney-arm tom-daubney-arm approved these changes

@gilles-peskine-arm gilles-peskine-arm Awaiting requested review from gilles-peskine-arm

@lpy4105 lpy4105 Awaiting requested review from lpy4105

Assignees

@tgonzalezorlandoarm tgonzalezorlandoarm

Labels
needs-review Every commit must be reviewed by at least two team members, priority-medium Medium priority - this can be reviewed as time permits size-xs Estimated task size: extra small (a few hours at most)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@tgonzalezorlandoarm @daverodgman @lpy4105 @tom-daubney-arm @yanrayw