x/mobile: gomobile build fails for iOS targets #32963

dlo · 2019-07-06T17:45:28Z

What version of Go are you using (`go version`)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (`go env`)?

macOS Mojave 10.14.5 (18F132)

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dan/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/dan/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/go-build908091193=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ go get golang.org/x/mobile/cmd/gomobile
$ go get -d golang.org/x/mobile/example/basic
$ gomobile build -bundleid basic.app -target=ios golang.org/x/mobile/example/basic

Note: the wiki instructions leave out the -bundleid parameter even though it appears to be required.

Output

gomobile: go build -tags ios -ldflags=-w -o=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/gomobile-work-749904680/arm golang.org/x/mobile/example/basic failed: exit status 2
# golang.org/x/mobile/app
darwin_ios.m:139:77: warning: null passed to a callee that requires a non-null argument [-Wnonnull]
# golang.org/x/mobile/example/basic
/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation

What did you expect to see?

See https://github.com/golang/go/wiki/Mobile#building-and-deploying-to-ios.

The build command will build an application bundle, named basic.app.

What did you see instead?

gomobile: go build -tags ios -ldflags=-w -o=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/gomobile-work-749904680/arm golang.org/x/mobile/example/basic failed: exit status 2
# golang.org/x/mobile/app
darwin_ios.m:139:77: warning: null passed to a callee that requires a non-null argument [-Wnonnull]
# golang.org/x/mobile/example/basic
/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation

The text was updated successfully, but these errors were encountered:

bcmills · 2019-07-08T18:54:25Z

CC @steeve @hyangah

dlo · 2019-07-23T14:36:01Z

@bcmills Should I go ahead and see if I can figure out what's going on here? Happy to chip in.

andydotxyz · 2019-07-28T15:18:25Z

I have the same problem on Mojave with XCode 10.2 and 10.3

andydotxyz · 2019-07-28T15:56:31Z

The warning can be fixed with this patch, but it does not fix the build error...

index 08bb72f..e17b150 100644
--- a/vendor/golang.org/x/mobile/app/darwin_ios.m
+++ b/vendor/golang.org/x/mobile/app/darwin_ios.m
@@ -135,8 +135,9 @@ - (void)touchesCanceled:(NSSet*)touches withEvent:(UIEvent*)event {
 @end
 
 void runApp(void) {
+       char * argv[] = {};
        @autoreleasepool {
-               UIApplicationMain(0, nil, nil, NSStringFromClass([GoAppAppDelegate class]));
+               UIApplicationMain(0, argv, nil, NSStringFromClass([GoAppAppDelegate class]));
        }
 }

andydotxyz · 2019-07-28T17:39:02Z

Aha, the problem seems to be that the go tools we are using force -headerpad and that bitcode is now enabled by default and these conflict. The following patch to gomobile fixed it for me 😎.

diff --git a/cmd/gomobile/build_iosapp.go b/cmd/gomobile/build_iosapp.go
index 9cc22ab..79b2c3f 100644
--- a/cmd/gomobile/build_iosapp.go
+++ b/cmd/gomobile/build_iosapp.go
@@ -426,6 +426,7 @@ const projPbxproj = `// !$*UTF8*$!
         SDKROOT = iphoneos;
         TARGETED_DEVICE_FAMILY = "1,2";
         VALIDATE_PRODUCT = YES;
+        ENABLE_BITCODE = NO;
       };
       name = Release;
     };
diff --git a/cmd/gomobile/env.go b/cmd/gomobile/env.go
index cb42d1c..a707965 100644
--- a/cmd/gomobile/env.go
+++ b/cmd/gomobile/env.go
@@ -140,7 +140,6 @@ func envInit() (err error) {
                default:
                        panic(fmt.Errorf("unknown GOARCH: %q", arch))
                }
-               cflags += " -fembed-bitcode"
                if err != nil {
                        return err
                }

andydotxyz · 2019-07-28T17:48:03Z

I don't know how to create a PR for gomobile, so here is a consolidated patch file instead...

gomobile.diff.txt

rob-deutsch · 2019-09-04T11:40:11Z

I just ran into the exact same problem. @andydotxyz solution fixed it!

@andydotxyz why did you decide to turn off bitcode instead of turning off headerpad?

I ask because I understand that Apple seems to prefer that bitcode is turned on.

andydotxyz · 2019-09-04T22:16:31Z

That is a great question @rob-deutsch - it was not my preferred approach either. Unfortunately headerpad is coded into the Go tools which gomobile delegates to (as far as I can tell) - this means that the patch would have to go upstream to the main project which seemed much harder!

rob-deutsch · 2019-09-05T00:08:42Z

Ha, that's a great reason! Thanks for the info @andydotxyz .

I know its unlikely, but would you happen to remember where exactly headerpad is hardcoded? I'm happy to take this upstream.

hyangah · 2019-09-19T15:38:45Z

@rob-deutsch I see in ld and some hints in macho_combine_dwarf.go

andydotxyz · 2019-09-30T16:38:40Z

Sorry I don't recall @rob-deutsch - I just used grep for "headerpad" I think...

hyangah · 2019-10-02T18:01:05Z

For people who didn't follow the cl -
@eliasnaur commented in the cl.

This CL effectively reverts https://golang.org/cl/168062, potentially re-opening https://golang.org/issue/22395. What changed?

Does anyone have an answer?

bcmills · 2019-10-02T18:11:27Z

@hyangah, which CL? (I don't see one linked here — it's possible that GopherBot missed a cross-reference.)

eliasnaur · 2019-10-07T21:55:12Z

@bcmills I believe @hyangah refers to https://golang.org/cl/189857.

eliasnaur · 2019-10-07T23:02:52Z

I did some digging and found that the answer to my own question is: nothing changed, but gomobile build mode broke with my bitcode-enabling CL168062. Presumably I tested gomobile bind but failed to test gomobile build when preparing that CL.

gomobile build builds an iOS executable, while gomobile bind builds a C archive (using buildmode=c-archive). Building an iOS executable with -fembed-bitcode is not compatible with the flags cmd/link pass to the host linker; I've found -headerpad, -pagezero_size and -fno_pie to be incompatible.

It seems to me the fix is to figure out why cmd/link needs to pass the incompatible flags, and if it doesn't, remove or replace them with bitcode compatible flags. Removing the incomptable flags succeeded in producing a binary but I haven't run it nor tried to upload it to the App Store for verification.

The next best fix is to work around the issue and switch gomobile build to use the buildmode=c-archive like gomobile bind. As a side effect, gomobile build will be much more similar to gomobile bind, decreasing the likelyhood of future breaks.

The easiest workaround is for gomobile build to omit -fembed-bitcode. The downside is that executables built this way will not have bitcode enabled. Since bitcode is not (yet) a requirement for iOS, I don't think missing bitcode matters.

CL 189857 implements the third option, except that it also omits the flag for gomobile bind which is wrong.

eliasnaur · 2019-10-08T20:33:56Z

I asked on golang-dev for the purpose of the incompatible flags:

https://groups.google.com/forum/#!topic/golang-dev/U1jK3xmmGAk

It seems possible that my first suggestion could work: remove -headerpad, -fno_pie, -pagezero_size and see whether gomobile build programs still work on iOS, and whether the App Store will accept them.

I won't have much time before GoLab Florence, and we're approaching the freeze. If someone else could do the experiment, I'd be very grateful.

eliasnaur · 2019-10-16T07:36:52Z

With #34501 (comment) we can now target iOS builders with the TRY= directive.

gopherbot · 2019-10-16T08:24:05Z

Change https://golang.org/cl/201358 mentions this issue: cmd/link/internal/ld: remove flags incompatible with -fembed-bitcode

eliasnaur · 2019-10-16T09:00:04Z

https://golang.org/cl/201358 seems to complete all.bash (on at least darwin/arm64). Can someone using gomobile build check whether App Store still accepts binaries with bitcode and CL 201358 applied? Thanks.
@hajimehoshi @andydotxyz @rob-deutsch @dlo

andydotxyz · 2019-10-26T17:46:45Z

Testing on the latest macOS with gomobile build I get a new error:

ld: -no_pie and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation)

eliasnaur · 2019-10-26T19:38:19Z

On Sat Oct 26, 2019 at 10:49 AM Andy Williams wrote: Testing on the latest macOS with gomobile build I get a new error: ``` ld: -no_pie and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together clang: error: linker command failed with exit code 1 (use -v to see invocation) ```

I didn't make it clear, but the CL only works for darwin/arm64, not (yet) for darwin/arm. Please use $ gomobile build -target ios/arm64 ... to build a binary for arm64 only and test with that. You may have to disable arm in your Xcode as well.

andydotxyz · 2019-10-26T21:18:15Z

Ah yes, apologies.
It compiles and runs on device - I will attempt an upload to app store shortly.
First I need to figure how to manage different certification profiles using gomobile ;)

andydotxyz · 2019-10-28T13:38:22Z

OK, I have attempted the upload and it seems to have no bitcode related errors :)

eliasnaur · 2019-10-28T13:51:00Z

Nice, thanks for checking.

…

The flags -headerpad, -Wl,-no_pie and -pagezero_size are incompatible with the -fembed-bitcode flag used by `gomobile build`. Than McIntosh suggested we might not need the offending flags; this change removes the flags on darwin/arm64 and -headerpad, -pagezero_size on darwin/arm. The -Wl,-no_pie flag is left for darwin/arm because linking fails without it: ld: warning: PIE disabled. Absolute addressing (perhaps -mdynamic-no-pic) not allowed in code signed PIE, but used in _runtime.rodata from /var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/tmp/go-link-225285265/go.o. To fix this warning, don't compile with -mdynamic-no-pic or link with -Wl,-no_pie Discussion: https://groups.google.com/d/msg/golang-dev/U1jK3xmmGAk/j0_ty46EDAAJ I've verified the CL on the builders, built the "flappy" example from gomobile with `gomobile build`, and verified that flappy runs on an iPhone 5S. Updates #32963 Change-Id: I783abc93ccf3c1d2b7ca00144b7164ba223d3529 Reviewed-on: https://go-review.googlesource.com/c/go/+/201358 Reviewed-by: Cherry Zhang <[email protected]>

eliasnaur · 2019-11-04T09:14:11Z

@andydotxyz could you be persuaded to check out Cherry's https://golang.org/cl/205060? It should fix the darwin/arm build as well.

andydotxyz · 2019-11-04T11:02:18Z

I would be happy to but am only at a Mac on Wednesday this week - I’ll do my best to get it checked then :)

We used to pass -no_pie to external linker on darwin/arm, which is incompatible with -fembed-bitcode. CL 201358 attempted to remove the -no_pie flag, but it resulted the darwin linker to complain about absolute addressing in TEXT segment. On darwin/arm, we already get away from absolute addressing in the TEXT section. The complained absolute addressing is in RODATA, which was embedded in the TEXT segment. This CL moves RODATA to the DATA segment, like what we already did on ARM64 and on AMD64 in c-archive/c-shared buildmodes for the same reason. So there is no absolute addressing in the TEXT segment, which allows us to remove -no_pie flag. Fixes #35252. Updates #32963. Change-Id: Id6e3a594cb066d257d4f58fadb4a3ee4672529f7 Reviewed-on: https://go-review.googlesource.com/c/go/+/205060 Reviewed-by: Elias Naur <[email protected]>

gopherbot · 2019-11-04T15:31:31Z

Change https://golang.org/cl/205060 mentions this issue: cmd/link: enable PIE on darwin/arm

andydotxyz · 2019-11-06T09:12:47Z

With both patches applied I get:

/Users/andy/Code/Go/goroot/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: warning: OS version (7.0.0) too small, changing to 8.0.0
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation)

eliasnaur · 2019-11-06T09:22:19Z

Both patches are in go tip, where headerpad is guarded by

      if !ctxt.Arch.InFamily(sys.ARM, sys.ARM64) {
          // -headerpad is incompatible with -fembed-bitcode.
          argv = append(argv, "-Wl,-headerpad,1144")
      }

so -headerpad shouldn't be emitted on darwin/arm nor darwin/arm64. Can you try again with Go tip?

andydotxyz · 2019-11-06T10:20:21Z

arm and arm64 are both working, however the "ios" target is not - so it's still not building for simulator.

works:

gomobile build -target ios/arm
gomobile build -target ios/arm64

does not:

gomobile build -target ios

eliasnaur · 2019-11-06T12:49:41Z

Thank you, I had forgotten about the simulators. Let me know whether golang.org/cl/205340 works, which fixes the problem for:

$ gomobile build -target ios -bundleid <...> golang.org/x/mobile/example/flappy

gopherbot · 2019-11-06T13:02:38Z

Change https://golang.org/cl/205340 mentions this issue: cmd/link/internal/ld: omit bitcode-incompatible flags on iOS simulator

andydotxyz · 2019-11-06T14:06:09Z

Excellent, thanks - that fixes "-target ios"

$ file myapp.app/main
Mach-O universal binary with 4 architectures: [i386:Mach-O executable i386] [x86_64] [arm_v7] [arm64]
(for architecture i386):	Mach-O executable i386
(for architecture x86_64):	Mach-O 64-bit executable x86_64
(for architecture armv7):	Mach-O executable arm_v7
(for architecture arm64):	Mach-O 64-bit executable arm64

The -Wl,-headerpad, -Wl,-no_pie, -Wl,-pagezero_size flags are incompatible with the bitcode-related flags used for iOS. We already omitted the flags on darwin/arm and darwin/arm64; this change omits the flags on all platforms != macOS so that building for the iOS simulator works. Updates #32963 Change-Id: Ic9af0daf01608f5ae0f70858e3045e399de7e95b Reviewed-on: https://go-review.googlesource.com/c/go/+/205340 Run-TryBot: Elias Naur <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>

gopherbot · 2019-11-10T07:54:23Z

Change https://golang.org/cl/206337 mentions this issue: cmd/link/internal/ld: set MachO platform to PLATFORM_IOS on iOS

CL 205340 changed the linker to skip certain flags when linking for iOS. However, for host linking on iOS (such as on the Corellium darwin/arm64 builders) the MachO platform defaults to PLATFORM_MACOS, and the flags are not skipped. Avoids warnings such as ld: warning: -no_pie ignored for arm64 Updates #32963 Change-Id: Ib6b4c2375fd14cf89410bf5ff1537b692b7a1c15 Reviewed-on: https://go-review.googlesource.com/c/go/+/206337 Run-TryBot: Elias Naur <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>

andydotxyz · 2019-11-28T08:36:00Z

I guess this can be closed when 1.13.5 comes out?

eliasnaur · 2019-11-28T08:51:11Z

The fixes will be in Go 1.14 and are not going to be backported. I left this issue open in case someone wanted to work on a gomobile workaround for Go 1.13. If not, go ahead and close it.

hajimehoshi · 2019-11-28T11:22:56Z

What would the workaround in gomobile be?

andydotxyz · 2019-11-28T23:20:20Z

Probably something like: fyne-io/mobile@9aa5623

eliasnaur · 2019-11-28T23:56:10Z

Yes, but activated only with Go < 1.14.

jvburnes · 2019-12-04T22:52:40Z

Hey peeps! I'm trying to use go mobile for an ios target and I ran into the same problem. It seems like the patch submitted above still hasn't been accepted in the master branch. I clone'd from github, applied the patch manually but now I'm trying to figure out how to rebuild go mobile from source. I did see the docs on fetching subrepositories like mobile, but I couldn't figure out how to rebuild from a local copy. Bear in mind I'm relatively new to go, but an experienced dev. Any help greatly appreciated.

hajimehoshi · 2020-01-15T11:36:25Z

I'm trying to create a CL.

gopherbot · 2020-01-15T11:42:14Z

Change https://golang.org/cl/214899 mentions this issue: cmd/gomobile: disable bitcode on Go 1.13 or older

gopherbot · 2020-01-15T18:16:40Z

Change https://golang.org/cl/214957 mentions this issue: cmd/gomobile: enable the test for gomobile-build on iOS

iOS's bitcode conflicts with headerpad on Go 1.13 or older. This problem is fixed on Go 1.14. This CL disables bitcode only on Go 1.13 or older. Fixes golang/go#32963 Cherry picked from github.com/golang/mobile

iOS's bitcode conflicts with headerpad on Go 1.13 or older. This problem is fixed on Go 1.14. This CL disables bitcode only on Go 1.13 or older. Fixes golang/go#32963 Change-Id: Iac9edd56aaf7819288e8f46b9e29d310305acd9d Reviewed-on: https://go-review.googlesource.com/c/mobile/+/214899 Run-TryBot: Hajime Hoshi <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Hyang-Ah Hana Kim <[email protected]>

bcmills changed the title ~~gomobile build fails for iOS targets~~ x/mobile: gomobile build fails for iOS targets Jul 8, 2019

gopherbot added this to the Unreleased milestone Jul 8, 2019

gopherbot added the mobile Android, iOS, and x/mobile label Jul 8, 2019

bcmills added help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jul 8, 2019

hajimehoshi mentioned this issue Oct 6, 2019

gomobile build failed on iOS hajimehoshi/ebiten#951

Closed

gopherbot closed this as completed in golang/mobile@875e90d Jan 15, 2020

golang locked and limited conversation to collaborators Jan 14, 2021

gopherbot added the FrozenDueToAge label Jan 14, 2021

x/mobile: gomobile build fails for iOS targets #32963

x/mobile: gomobile build fails for iOS targets #32963

Comments

dlo commented Jul 6, 2019 • edited Loading

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

bcmills commented Jul 8, 2019

dlo commented Jul 23, 2019

andydotxyz commented Jul 28, 2019

andydotxyz commented Jul 28, 2019 • edited Loading

andydotxyz commented Jul 28, 2019

andydotxyz commented Jul 28, 2019

rob-deutsch commented Sep 4, 2019

andydotxyz commented Sep 4, 2019

rob-deutsch commented Sep 5, 2019

hyangah commented Sep 19, 2019

andydotxyz commented Sep 30, 2019

hyangah commented Oct 2, 2019

bcmills commented Oct 2, 2019

eliasnaur commented Oct 7, 2019

eliasnaur commented Oct 7, 2019

eliasnaur commented Oct 8, 2019

eliasnaur commented Oct 16, 2019

gopherbot commented Oct 16, 2019

eliasnaur commented Oct 16, 2019

andydotxyz commented Oct 26, 2019

eliasnaur commented Oct 26, 2019 via email

andydotxyz commented Oct 26, 2019 • edited Loading

andydotxyz commented Oct 28, 2019

eliasnaur commented Oct 28, 2019 via email

eliasnaur commented Nov 4, 2019

andydotxyz commented Nov 4, 2019

gopherbot commented Nov 4, 2019

andydotxyz commented Nov 6, 2019

eliasnaur commented Nov 6, 2019

andydotxyz commented Nov 6, 2019

eliasnaur commented Nov 6, 2019

gopherbot commented Nov 6, 2019

andydotxyz commented Nov 6, 2019

gopherbot commented Nov 10, 2019

andydotxyz commented Nov 28, 2019

eliasnaur commented Nov 28, 2019

hajimehoshi commented Nov 28, 2019

andydotxyz commented Nov 28, 2019

eliasnaur commented Nov 28, 2019

jvburnes commented Dec 4, 2019

hajimehoshi commented Jan 15, 2020

gopherbot commented Jan 15, 2020

gopherbot commented Jan 15, 2020

dlo commented Jul 6, 2019 •

edited

Loading

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

andydotxyz commented Jul 28, 2019 •

edited

Loading

andydotxyz commented Oct 26, 2019 •

edited

Loading