How to integrate with xgo?

[Envek]

Hey, thanks for the Ginkgo, it is nice to use, love it.

I would like to use xgo mocks to easily test various failure modes of underlying libraries, but it doesn't seem to work.

For example, I want to convert following go test (NB: it must be executed with xgo test to work):

patch_test.go

package patch_test

import (
	"testing"
	"github.com/xhd2015/xgo/runtime/mock"
)

func greet(s string) string {
	return "hello " + s
}

func TestPatchFunc(t *testing.T) {
	mock.Patch(greet, func(s string) string {
		return "mock " + s
	})

	res := greet("world")
	if res != "mock world" {
		t.Fatalf("expect patched result to be %q, actual: %q", "mock world", res)
	}
}

Run results

$ xgo test .
ok  	patch	0.004s

to Ginkgo (after bootstrapping):

patch_suite_test.go

package patch_test

import (
	"testing"

	. "github.com/onsi/ginkgo/v2"
	. "github.com/onsi/gomega"
)

func TestMock(t *testing.T) {
	RegisterFailHandler(Fail)
	RunSpecs(t, "Mock Suite")
}

patch_test.go

package patch_test

import (
	. "github.com/onsi/ginkgo/v2"
	. "github.com/onsi/gomega"
	"github.com/xhd2015/xgo/runtime/mock"
)

func greet(s string) string {
	return "hello " + s
}

var _ = Describe("Mocking with Xgo from Ginkgo", func() {
	It("works", func(ctx SpecContext) {
		unpatch := mock.Patch(greet, func(s string) string {
			return "mock " + s
		})
		defer unpatch()

		Expect(greet("world")).To(Equal("mock world"))
	})
})

Run results

$ xgo exec ginkgo ./... 

Running Suite: Mock Suite - /home/envek/Code/mock
=========================================================
Random Seed: 1714532009

Will run 1 of 1 specs
------------------------------
• [PANICKED] [0.000 seconds]
Mocking with Xgo from Ginkgo [It] works
/home/envek/Code/mock/mock_test.go:60

  [PANICKED] Test Panicked
  In [It] at: /home/envek/go/pkg/mod/github.com/xhd2015/xgo/[email protected]/mock/mock.go:54 @ 05/01/24 11:53:30.098

  failed to setup mock for: mock_test.greet

  Full Stack Trace
    github.com/xhd2015/xgo/runtime/mock.getFunc({0x82d060, 0x9081a8})
    	/home/envek/go/pkg/mod/github.com/xhd2015/xgo/[email protected]/mock/mock.go:54 +0xe7
    github.com/xhd2015/xgo/runtime/mock.Patch({0x82d060, 0x9081a8}, {0x82d060, 0x9081c0})
    	/home/envek/go/pkg/mod/github.com/xhd2015/xgo/[email protected]/mock/patch.go:51 +0x2af
    mock_test.init.func1.1({0x0?, 0x0?})
    	/home/envek/Code/mock/mock_test.go:61 +0x3b
------------------------------

Summarizing 1 Failure:
  [PANICKED!] Mocking with Xgo from Ginkgo [It] works
  /home/envek/go/pkg/mod/github.com/xhd2015/xgo/[email protected]/mock/mock.go:54

Ran 1 of 1 Specs in 0.000 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: TestMock (0.00s)
FAIL

Ginkgo ran 1 suite in 354.453564ms

Test Suite Failed
exit status 1

But as it can be seen that when test run either via xgo test . or via xgo exec ginkgo ./..., xgo loses its under-the-hood tweaks. I suppose that ginkgo somehow re-executes tests in a new process maybe (didn't dive into ginkgo internals yet)

Is it possible to use xgo with ginkgo? How to properly set it up?

[onsi]

hey there - I've never used xgo (first time I'm hearing about it actually!) ginkgo calls go to compile your tests and so xgo exec ginkgo ./... will end up just calling go under the hood.

But ginkgo does allow you to pass in a precompiled binary as a test suite. So you could try:

cd ./path/to/package
xgo test -c 
ginkgo package.test

to see if that works. If it does we'll need to teach the Ginkgo cli how to choose which go to use:
https://github.com/onsi/ginkgo/blob/master/ginkgo/internal/compile.go#L44

I could imagine a hidden GINKGO_GOBINARY="xgo" environment variable that reconfigures that line.

[Envek]

But ginkgo does allow you to pass in a precompiled binary as a test suite.

Nope, that doesn't work either.

[onsi]

odd. what happens if you go to your path_suite and run xgo test (don't even use ginkgo - jus use the usual test runner. you'll lose some of the cli features but the tests will run fine)

[Envek]

Oh, pardon my confusion, I messed up with xgo setup in my terminal (xgo requires that its binary is directly available in the PATH without any intermediate shims from asdf or similar version managers, and I lost this part when reopened my terminal).

Both xgo test -c && ginkgo package.test and xgo test are working, sorry again.

[onsi]

Ok sounds good. Sounds like the last thing we need to do is allow you to override which Go binary ginkgo uses. Would you be up for submitting a PR?

We'd want to add some code to:
https://github.com/onsi/ginkgo/blob/master/ginkgo/internal/compile.go#L44

to look for a GINKGO_GOBINARY environment variable and use that instead of "go" if it is defined.

We'd then need to update the docs:
https://github.com/onsi/ginkgo/blob/master/docs/index.md?plain=1#L5298

to mention the environment variable.

WDYT?

[xhd2015]

Ok sounds good. Sounds like the last thing we need to do is allow you to override which Go binary ginkgo uses. Would you be up for submitting a PR?

We'd want to add some code to: https://github.com/onsi/ginkgo/blob/master/ginkgo/internal/compile.go#L44

to look for a GINKGO_GOBINARY environment variable and use that instead of "go" if it is defined.

We'd then need to update the docs: https://github.com/onsi/ginkgo/blob/master/docs/index.md?plain=1#L5298

to mention the environment variable.

WDYT?

Maybe able to specify the whole building command is more feasible? Like GINKGO_BUILDTEST_COMMAND?
Or something like go's -toolexec flag, which passes the whole command and args to the target binary.

[onsi]

hey sorry for the delay. my understanding of toolexec is that it chains commands before running the canonical go command, not for changing the go command itself. i also don't want folks to be adding their own flags as that could interfere with Ginkgo's build settings. I do think the simple/easy approach here is to simply override the one go invocation that i describe above.