Investigate flaky test-dgram-udp6-send-default-host on OS X

[Trott]

• Version: master branch
• Platform: OS X
• Subsystem: test? dgram?

Failure in CI on test of unrelated change: https://ci.nodejs.org/job/node-test-commit-osx/3210/nodes=osx1010/tapTestReport/test.tap-213/

not ok 213 test-dgram-udp6-send-default-host.js
# 
# assert.js:90
#   throw new assert.AssertionError({
#   ^
# AssertionError: message was received correctly
#     at Socket.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1010/test/parallel/test-dgram-udp6-send-default-host.js:29:10)
#     at emitTwo (events.js:106:13)
#     at Socket.emit (events.js:191:7)
#     at UDP.onMessage (dgram.js:532:8)
  ---
  duration_ms: 0.145

/cc @nodejs/testing @addaleax

[Trott]

/cc @mcollina

[Trott]

Stress test: https://ci.nodejs.org/job/node-stress-single-test/687/nodes=osx1010/

[santigimeno]

Have seen this failure locally a couple of times though I cannot reproduce it consistently :/.
Maybe use assert.strictEqual here would help to find out what's going on.

[Trott]

@santigimeno Yup, I had the same thought and am putting in the PR for that right now...

[Trott]

#6581

[mcollina]

This is fairly unexpected, as the ipv4 equivalent of this test is not flaky.

Is this flaky only on Mac? Hopefully #6581 should tell what's happening. I have never seen it fail here.

[santigimeno]

It looks like the problem is that sometimes the messages are not received in order, which makes sense being UDP. See output from https://ci.nodejs.org/job/node-stress-single-test/nodes=osx1010/690 :

not ok 1 test-dgram-udp6-send-default-host.js
# 
# assert.js:90
#   throw new assert.AssertionError({
#   ^
# AssertionError: 'hello' === 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
#     at Socket.<anonymous> (/Users/iojs/build/workspace/node-stress-single-test/nodes/osx1010/test/parallel/test-dgram-udp6-send-default-host.js:29:10)
#     at emitTwo (events.js:106:13)
#     at Socket.emit (events.js:191:7)
#     at UDP.onMessage (dgram.js:532:8)
  ---
  duration_ms: 0.139

[addaleax]

Could it be that the kernel decides to reorder the packets here for some reason? That’s allowed, although I can’t see why it would. But it would explain why the test is flaky and why this is an OS-specific issue.

[santigimeno]

@addaleax Some comments here like this suggest that could be the reason

[addaleax]

👍 I guess the IPv4 test has a chance of being flaky too, then?

[santigimeno]

Is there an equivalent IPv4 test? I can't find it.

[addaleax]

@santigimeno test/parallel/test-dgram-send-default-host.js

[santigimeno]

@addaleax Thanks! I guess it should also be rewritten