Fail test run when QEMU fails too quickly

[kostyanf14]

No description provided.

[akihikodaki]

Use Process#success?.
Also, can't we just always raise an error when the exit status is not 0?

[kostyanf14]

Also, can't we just always raise an error when the exit status is not 0?

No. QEMU can crash, but after a restart, HLK tests will continue working.
If QEMU crashes once during hours, more likely just one test will fail.
For example DISK Stress (LOGO) crashes QEMU when an IDE disk is provided.
This is bad, but all other results are present. As we run testing Windows insider preview
versions, we can see strange crashes.

[akihikodaki]

What about only checking the elapsed time then?
Something certainly went wrong when the elapsed time was too short.
Also, use: Process.clock_gettime(Process::CLOCK_MONOTONIC)

[akihikodaki]

I added comments for a few things I missed in the last review.

[akihikodaki]

Why don't you raise immediately after qemu = nil?
You can also @fails_quickly turn into a local variable by doing so. It is better to have fewer states (instance variables).

It is also better to use . instead of &.; AutoHCK::CmdRun::wait_no_fail returns nil if the process does not terminate, but such a situation should not happen since Process::WNOHANG is not passed to AutoHCK::CmdRun::wait_no_fail. If AutoHCK::CmdRun::wait_no_fail returns nil despite that, it only implies the program has a bug. Assuming the process either succeeded or failed is wrong in that case, and we should raise an error instead.

[akihikodaki]

Suggested change

	end
	if @first_fail_time.nil?
	elsif @first_fail_time.nil?

[akihikodaki]

Suggested change

	(Process.clock_gettime(Process::CLOCK_MONOTONIC) - @first_fail_time) <= 10
	Process.clock_gettime(Process::CLOCK_MONOTONIC) - @first_fail_time <= 10