Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate of smart-flank-gcs-path to not override different results type #1057

Closed
adamfilipow92 opened this issue Aug 26, 2020 · 3 comments · Fixed by #1056
Closed

Validate of smart-flank-gcs-path to not override different results type #1057

adamfilipow92 opened this issue Aug 26, 2020 · 3 comments · Fixed by #1056
Assignees

Comments

@adamfilipow92
Copy link
Contributor

Author the user story for this feature

As a flank user, I want to be informed when flank can override JUnitResult configured in smart-flank-gcs-path with a different type in the current run so I can easily avoid situations like that

Is your feature request related to a problem? Please describe.

If the test is flaky, flank removing stack traces from JUnitResult, FullJunitResult keeps stack traces.

Describe the solution you'd like

Fast fail when Flank can potentially override JUnitResult with different report type.

@pawelpasterz
Copy link
Contributor

pawelpasterz commented Aug 27, 2020

Just for my understanding. Now flank will upload to a bucket only if types match? (by types I understand full and normal results)
example:
I run my test suite with flank and received JUnitResult. But I am not glad about the info I received and want to have FullJUnitResult. So now, I need to manually remove a file from the bucket and after that, I am able to run flank with the new config?

Not valid question anymore, see #1057 (comment)

@pawelpasterz
Copy link
Contributor

Sorry, it's just too many sources of information for this issue 😄
Based on #936 and implementation in #1056 I am trying to understand the actual case.

For now, we can run smart flank without set --full-junit-result and --legacy-junit-result but if we use FullJUnitResult in gcs path and current run don't have set flag --full-junit-result flank override full junit result by junit result it's can be little confusing

If I get it correctly it means that currently flank will try to upload FullJUnitResult to thew bucket but actually this file won't be produced because --full-junit-result was not used and JUnitResult is created instead. So flank will upload JUnitResult but rename it to FullJUnitResult Am I correct?

If user set different XML name in smart-flank-gcs-path flank shouldn't validate type.

Basically means - ah, yeah, you changed the name so probably you know what you are doing (no validation)

Thanks for understanding and patience :)

@adamfilipow92
Copy link
Contributor Author

So 😄

  1. If we set smart-flank-gcs-path to .../FullJUnitResult.xml and we have turned on flag --full-junit-result file from smart-flank-gcs-path path will be overridden by JUnitResult.xml. We want to avoid that case.
    There is a small exception mentioned by you on my pr, about turn off validation when --smart-flank-disable-upload set. Thanks!

  2. About different XML names - right, if the name is changed by the user we don't validate the type. The user probably knows what he doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants