x-pack/packetbeat: System tests failure on Windows

[sharbuz]

Failed Test

• Test Name: TestDevices

• Link:

  beats/x-pack/packetbeat/tests/system/app_test.go

  Lines 55 to 70 in 7b041ba

  	func TestDevices(t *testing.T) {
  	stdout, stderr, err := runPacketbeat(t, "devices")
  	require.NoError(t, err, stderr)
  	t.Log("Output:\n", stdout)
  	ifcs, err := net.Interfaces()
  	require.NoError(t, err)
  	var expected []string
  	for _, ifc := range ifcs {
  	expected = append(expected, fmt.Sprintf("%d:%s:%s", ifc.Index, ifc.Name, ifc.Flags))
  	}
  	t.Log("Expect interfaces:\n", expected)
  	for _, ifc := range ifcs {
  	assert.Contains(t, stdout, ifc.Name)
  	}

• Branch: main: Enable tests that were muted during migration #40387

• Artifact Link: beats-xpack-packetbeat_build_5706_windows-x-pack-slash-packetbeat-win-2022-system-tests.log

• Notes: Additional details about the test. e.g. theory as to failure cause
  NB! tests should be enabled back, since those were muted while migrating from Jenkins to Buildkite. Please uncomment:
  

  beats/.buildkite/x-pack/pipeline.xpack.packetbeat.yml

  Line 163 in 7b041ba

  skip: "skipping due to elastic/beats#38142"

  
  

  beats/.buildkite/x-pack/pipeline.xpack.packetbeat.yml

  Line 253 in 7b041ba

  skip: "skipping due to elastic/beats#38142"

Buildkite builds: https://buildkite.com/elastic/beats-xpack-packetbeat/builds/5706#_

Stack Trace

=== FAIL: x-pack/packetbeat/tests/system TestDevices (0.21s)
--
  | app_test.go:58: Output:
  | 0: \Device\NPF_{8058D1FD-C1F5-4A20-A848-0188AB321A97} (WAN Miniport (Network Monitor)) (Not assigned ip address)
  | 1: \Device\NPF_{4265A7ED-129C-497F-B1AC-A60DB44D0DC7} (WAN Miniport (IPv6)) (Not assigned ip address)
  | 2: \Device\NPF_{608080BA-7C74-441B-918A-AEA32DD991B4} (WAN Miniport (IP)) (Not assigned ip address)
  | 3: \Device\NPF_{5F3CCF6B-3314-4F07-9284-5AE0B33202A8} (Google VirtIO Ethernet Adapter) (10.224.0.241)
  | 4: \Device\NPF_Loopback (Adapter for loopback traffic capture) (::1 127.0.0.1)
  | PASS
  | coverage: 43.9% of statements in ./...
  | app_test.go:66: Expect interfaces:
  | [7:Ethernet 2:up\|broadcast\|multicast\|running 1:Loopback Pseudo-Interface 1:up\|loopback\|multicast\|running]
  | app_test.go:69:
  | Error Trace:	C:/buildkite-agent/builds/bk-agent-prod-gcp-1723456393706198410/elastic/beats-xpack-packetbeat/x-pack/packetbeat/tests/system/app_test.go:69
  | Error:      	"0: \\Device\\NPF_{8058D1FD-C1F5-4A20-A848-0188AB321A97} (WAN Miniport (Network Monitor)) (Not assigned ip address)\n1: \\Device\\NPF_{4265A7ED-129C-497F-B1AC-A60DB44D0DC7} (WAN Miniport (IPv6)) (Not assigned ip address)\n2: \\Device\\NPF_{608080BA-7C74-441B-918A-AEA32DD991B4} (WAN Miniport (IP)) (Not assigned ip address)\n3: \\Device\\NPF_{5F3CCF6B-3314-4F07-9284-5AE0B33202A8} (Google VirtIO Ethernet Adapter) (10.224.0.241)\n4: \\Device\\NPF_Loopback (Adapter for loopback traffic capture) (::1 127.0.0.1)\nPASS\ncoverage: 43.9% of statements in ./..." does not contain "Ethernet 2"
  | Test:       	TestDevices
  | app_test.go:69:
  | Error Trace:	C:/buildkite-agent/builds/bk-agent-prod-gcp-1723456393706198410/elastic/beats-xpack-packetbeat/x-pack/packetbeat/tests/system/app_test.go:69
  | Error:      	"0: \\Device\\NPF_{8058D1FD-C1F5-4A20-A848-0188AB321A97} (WAN Miniport (Network Monitor)) (Not assigned ip address)\n1: \\Device\\NPF_{4265A7ED-129C-497F-B1AC-A60DB44D0DC7} (WAN Miniport (IPv6)) (Not assigned ip address)\n2: \\Device\\NPF_{608080BA-7C74-441B-918A-AEA32DD991B4} (WAN Miniport (IP)) (Not assigned ip address)\n3: \\Device\\NPF_{5F3CCF6B-3314-4F07-9284-5AE0B33202A8} (Google VirtIO Ethernet Adapter) (10.224.0.241)\n4: \\Device\\NPF_Loopback (Adapter for loopback traffic capture) (::1 127.0.0.1)\nPASS\ncoverage: 43.9% of statements in ./..." does not contain "Loopback Pseudo-Interface 1"
  | Test:       	TestDevices

[elasticmachine]

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

[rowlandgeoff]

During the migration of beats-ci from Jenkins to Buildkite, a number of tests were failing consistently due to issues unrelated to the migration. Those tests were disabled to stabilize the CI, with the intent to revisit them post-migration. @oakrizan has reviewed them all in her draft PRs linked above in the description, and has opened tickets such as this one to highlight to the product teams the tests that are currently still disabled and could use some attention.

[ycombinator]

@pierrehilbert @jlind23 Can we prioritize this issue (re-enabling the test mentioned in this issue as it is currently skipped) in an upcoming sprint, please? See #41909 (comment) for the rationale.

[jlind23]

@ycombinator looks like this is owned by @elastic/sec-linux-platform.
@nfritts is there anyone on your end able to pick this up?

[nfritts]

@jlind23 I'm not sure. I'm doing some research to see what all is required. Even if we picked it up though, I think having this fixed by the next feature freeze is unlikely looking at the PTO calendar.

[jlind23]

I think having this fixed by the next feature freeze is unlikely looking at the PTO calendar.

It has been disabled for quite some time so I don't think we need to rush that hard to get it back on track but the sooner the better.