Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix http tests #1451

Merged
merged 2 commits into from
Jun 26, 2022
Merged

Fix http tests #1451

merged 2 commits into from
Jun 26, 2022

Conversation

mlaily
Copy link
Contributor

@mlaily mlaily commented Jun 20, 2022

The CI for #1447 already failed two times, so let's see if this PR improves the situation...

I initially thought the problem was simply that using an external (internet) endpoint for the tests was unreliable, but the tests were still failing a bit randomly after replacing the remote endpoints with a local http test server.

After investigation, I believe there is a race condition between the http stack timeout and the Async.StartChild timeout. (in getResponse in Http.fs)
If the http stack timeout is triggered first, the inner exception is null. Since the tests were asserting that the inner exception should be a TimeoutException, they were failing in this case.

In all cases, the WebException status should indicate a timeout, so I changed the assertion accordingly...

@mlaily mlaily force-pushed the fix-flaky-tests branch from b0c1aaa to f1c485d Compare June 26, 2022 09:35
@mlaily mlaily changed the title Use a local http test server instead of calling internet servers Fix http tests Jun 26, 2022
Copy link
Collaborator

@cartermp cartermp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good overall, just some questions

Comment on lines 73 to 74
let localServer = startHttpLocalServer()
using localServer <| fun _ ->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason not to do this?

use localServer = startHttplocalServer()
//...

Copy link
Contributor Author

@mlaily mlaily Jun 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started with an implementation where the IDisposable returned by startHttpLocalServer() didn't contain anything useful beside Dispose() and I was afraid Dispose() would be called too early if the value wasn't otherwise used. (I'm not sure when it goes out of scope in this situation)

This is less true now that the BaseAddress is used to create the http request.

Do you want me to change it? (and if you know about when Dispose() is called with seemingly unused use bindings, I'd be interested to know :))

Copy link
Collaborator

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh ok! So nothing too fancy going on. Thanks for the demo! 🙏
I should use sharplab more often...

(I'll update the commit)

let exc = Assert.Throws<WebException> (fun () ->
Http.Request(localServer.BaseAddress + "/200?sleep=1000", customizeHttpRequest = (fun req -> req.Timeout <- 1; req)) |> ignore)

Assert.That(exc.Status, Is.EqualTo(WebExceptionStatus.Timeout))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why Assert.AreEqual can't still be used?

Copy link
Contributor Author

@mlaily mlaily Jun 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. I find this syntax more easy to read, that's all (There is less ambiguity which value is the "expected" value and which one is the "actual" value).

I can change it to Assert.AreEqual if you feel like consistency with the rest of the tests is more important.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, please change it so that it's consistent with the rest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. (actually, the rest of the file uses ... |> should equal ... so I'm going to do the same)

mlaily added 2 commits June 26, 2022 17:39
I believe there is a race condition between the http stack timeout
and the async timeout.
If the http stack timeout is triggered first, the inner exception is null.

In both cases, the WebException status should indicate a timeout.
@mlaily mlaily force-pushed the fix-flaky-tests branch from f1c485d to c5f1a77 Compare June 26, 2022 15:40
Copy link
Collaborator

@cartermp cartermp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@cartermp cartermp merged commit b311949 into fsprojects:main Jun 26, 2022
@mlaily
Copy link
Contributor Author

mlaily commented Jun 26, 2022

Thank you too!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants