runtime: tracking bug for ARM-based macOS and GOOS/GOARCH values

[bradfitz]

The ARM-based Mac rumors of the past couple years continue to heat up (e.g. https://www.theverge.com/2020/3/27/21196611/arm-macbook-desktop-apple-2021-release-date)

Has anybody given any thought to what we're going to do with runtime.GOOS and runtime.GOARCH if/when that happens, so we don't add ambiguity between darwin/arm64 (which currently means iOS only) and darwin/arm64 (which might also mean macOS in the future)?

Changing GOARCH seems out.

Adding GOOS=macos would break any code that switches on those values.

Perhaps some magic behavior like we did for GOOS=android (implies linux build tags) and GOOS=illumos (implies solaris build tags)? But what would still break code that checks runtime.GOOS.

Perhaps keep darwin/arm64 on both iOS and macOS but add some some runtime exported const DarwinOS = {"macos", "ios", ""} that people could switch on when it matters?

If there's new API, might be nice to get that in earlier. But OTOH, don't want to jump the gun if rumors don't pan out.

[rsc]

If darwin/arm64 doesn't work for macos,
then I think we should reclaim it for macos
and move ios over to ios/arm64, much as we have
android as an almost-synonym for Linux on mobile.
(Many fewer GOOS=darwin users on mobile than desktop.)

I've thought about whether we should have GOOS=macos
independently of this issue, but it seems like too much churn
for too little benefit.

Interesting to note that way back when, we could have done
GOOS=osx and we'd be in the same boat of having the wrong
name. Moving to GOOS=macos will only match Apple terminology
until the next OS rename.

[ianlancetaylor]

I think the suggestion here is:

1. add ios as a valid GOOS value
2. let ios imply darwin much as android implies linux

This will potentially break any existing mobile programs that test runtime.GOOS and expect to see darwin. After this change they will see ios.

This suggests that for 1.15 we should

1. add ios as a valid build tag that implies darwin
2. announce in the release notes that we plan to change to GOOS=ios in a future release

[ianlancetaylor]

CC @hyangah

[cherrymui]

I think (but not sure) gomobile sets ios tag, to distinguish iOS emulator on x86 vs. macOS.

[hyangah]

gomobile uses ios tag and also we use the ios tag in crypto/x509 package. gomobile sets GOOS/GOARCH for crosscompile so it needs to use the right value depending on the go version. I don't think that's too difficult. In addition to gomobile, cc/ @eliasnaur <[email protected]> for gio, @hajimehoshi for ebiten users

…

On Fri, May 1, 2020 at 2:38 PM cherrymui ***@***.***> wrote: I think (but not sure) gomobile sets ios tag, to distinguish iOS emulator on x86 vs. macOS. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#38485 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGESL63JC4FILQFUNGUQH3RPMJJPANCNFSM4MJZNOPQ> .

-- __

[networkimprov]

cc @eliasnaur @hajimehoshi

[dmitshur]

Interesting to note that way back when, we could have done
GOOS=osx and we'd be in the same boat of having the wrong
name. Moving to GOOS=macos will only match Apple terminology
until the next OS rename.

Also interesting is that Apple currently has macOS, iOS, iPadOS, tvOS, and watchOS.

[hajimehoshi]

As far as I know about mobile application developing, there should be no problem to add ios to GOOS :-)

[gopherbot]

Change https://golang.org/cl/239278 mentions this issue: all: experiment with adding ios as a recognized GOOS value and build constraint that implies darwin

[dmitshur]

I expect we will learn relevant official information from Apple's WWDC 20 Keynote later today, and be able to use that information to make any definitive 1.15-specific decision.

However, given we are well into the code freeze for 1.15 with not much time left, and a change to build tags now would require getting a freeze exception (per bullet point 3 from the planning thread), I started investigating the plan for 1.15 @ianlancetaylor described in #38485 (comment) in advance to learn more and be able to move quickly today if needed.

Based on what I've learned from the investigation (also see a draft DNS CL 239278), I suspect we will not need or want to make any code changes for 1.15, only add a note to the release notes. The rationale follows.

The motivation behind doing "add ios as a valid build tag that implies darwin" for 1.15, as I understand, is to enable people to start moving their iOS-specific code away from darwin,arm64 to something else as early as 1.15, rather than later. However, as @hyangah mentioned in #38485 (comment), the gomobile tool already sets an "ios" build constraint whenever targeting iOS (see code here, here and here). That means we don't need to make changes in Go 1.15 for people to be able to start moving iOS-specific code behind a potentially more future-proof ios build constraint. They can already do that now, even with Go 1.14, 1.13, etc. (Runtime code that attempts to check runtime.GOOS to tell whether iOS is the target is a separate matter, and I'll address that in a later comment.)

Work to add a new port must be started before the code freeze, so by now we know for sure that Go 1.15 will not be able to support macOS on any new architectures beyond amd64. We can also defer any changes to the main tree until we know for certain we need to reclaim darwin,arm64 (as @rsc emphasized in his original comment).

This suggests for 1.15 it's likely sufficient to update release notes without any code changes.

I'll revisit and update this after the WWDC keynote, based on new official information that may be made available.

[bradfitz]

It's official.

(One source: https://www.wired.com/story/wwdc-2020-everything-apple-announced/)

[dmitshur]

It is! Apple has announced it during the WWDC20 keynote. Here are more sources:

• https://developer.apple.com/macos/
• https://developer.apple.com/programs/universal/

The timeline they've announced is that it'll be a 2 year transition, with the first Apple Silicon Macs shipping by the end of the year and development kits this week. There will be more Intel-powered macOS devices coming out too.

I believe with this information, the plan to update release notes for Go 1.15 mentioned in #38485 (comment) is still valid. All other changes can wait until the tree re-opens for 1.16 development. Feedback is welcome. I plan to send a CL for the release note entry for review this week.

[networkimprov]

It's possible the new "Apple Silicon" will be a superset of 64-bit ARM, or even a departure. Perhaps a new GOARCH symbol should be considered if that's the case, e.g. apl64?

[ianlancetaylor]

We'll cross that bridge when we come to it.

(It seems fairly unlikely that Apple would stop running existing programs. If they introduce new instructions that we care about, we can address that through a GOARM64 environment variable if necessary.)

[jpap]

Apple just announced in their State of the Union talk at WWDC that they've developed patches for Go, that they hope to submit in the coming days. So exciting! Hope we can onboard support for macOS/arm64 (even on a branch) very soon. :)

[dmitshur]

@jpap Thanks for the heads up. A recording of that talk is available at https://developer.apple.com/videos/play/wwdc2020/102/. The phrase I heard around the 20:34 mark is a less definitive "we've already done the initial work for some of the more widely used open-source projects", so we may need to wait to learn details.

[jpap]

@jpap Thanks for the heads up. A recording of that talk is available at https://developer.apple.com/videos/play/wwdc2020/102/. The phrase I heard around the 20:34 mark is a less definitive "we've already done the initial work for some of the more widely used open-source projects", so we may need to wait to learn details.

That's right, with the statement that followed: "...and we will be publishing patches to them in the next days". You can see a reference to the Go project on the slide behind the speaker. Now I've only just noticed the bold type on some projects on the second slide below -- perhaps Apple will focus on those first, for which Go may have to wait.

[rsc]

I don't believe we have to do anything in the Go 1.15 release notes.
Nothing is changing for Go 1.15.

[cherrymui]

the gomobile tool already sets an "ios" build constraint whenever targeting iOS (see code here, here and here). That means we don't need to make changes in Go 1.15 for people to be able to start moving iOS-specific code behind a potentially more future-proof ios build constraint.

I just want to mention that gomobile is not the only way to build Go programs on iOS. There are probably other tools (not sure if they have ios tag set). Also it could be done by hand (only using Go tools).

(I'm not suggesting we should or should not do anything for it in Go 1.15.)

[felipecsl]

all.bash failed though:

arch --arm64e env GOROOT_BOOTSTRAP=$GO_TMP/go-darwin-arm64-bootstrap ./all.bash
Building Go cmd/dist using /var/folders/73/8zyvcpds2612pj0nmtwt68_c0000gn/T/go-arm-bootstrap.PB58nt4i/go-darwin-arm64-bootstrap. (devel +cf7aa585ac Tue Dec 1 23:39:15 2020 +0000 darwin/arm64)
Building Go toolchain1 using /var/folders/73/8zyvcpds2612pj0nmtwt68_c0000gn/T/go-arm-bootstrap.PB58nt4i/go-darwin-arm64-bootstrap.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for darwin/arm64.

##### Testing packages.
ok  	archive/tar	0.464s
ok  	archive/zip	0.572s
ok  	bufio	0.425s
ok  	bytes	0.533s
ok  	compress/bzip2	0.208s
ok  	compress/flate	0.970s
ok  	compress/gzip	0.543s
ok  	compress/lzw	0.658s
ok  	compress/zlib	0.863s
ok  	container/heap	0.285s
ok  	container/list	0.807s
ok  	container/ring	0.855s
ok  	context	0.678s
ok  	crypto	0.805s
ok  	crypto/aes	0.747s
ok  	crypto/cipher	0.656s
ok  	crypto/des	0.547s
ok  	crypto/dsa	0.723s
ok  	crypto/ecdsa	0.765s
ok  	crypto/ed25519	0.471s
ok  	crypto/elliptic	0.704s
ok  	crypto/hmac	0.492s
ok  	crypto/internal/subtle	0.579s
ok  	crypto/md5	0.678s
ok  	crypto/rand	0.686s
ok  	crypto/rc4	0.722s
ok  	crypto/rsa	0.905s
ok  	crypto/sha1	0.522s
ok  	crypto/sha256	0.604s
ok  	crypto/sha512	0.551s
ok  	crypto/subtle	0.591s
ok  	crypto/tls	1.305s
ok  	crypto/x509	1.077s
ok  	database/sql	0.874s
ok  	database/sql/driver	0.156s
ok  	debug/dwarf	0.203s
ok  	debug/elf	0.167s
ok  	debug/gosym	0.294s
ok  	debug/macho	0.240s
ok  	debug/pe	0.426s
ok  	debug/plan9obj	0.276s
ok  	embed/internal/embedtest	0.405s
ok  	encoding/ascii85	0.438s
ok  	encoding/asn1	0.159s
ok  	encoding/base32	0.186s
ok  	encoding/base64	0.242s
ok  	encoding/binary	0.200s
ok  	encoding/csv	0.171s
ok  	encoding/gob	0.263s
ok  	encoding/hex	0.160s
ok  	encoding/json	0.397s
ok  	encoding/pem	0.200s
ok  	encoding/xml	0.161s
ok  	errors	0.162s
ok  	expvar	0.194s
ok  	flag	0.235s
ok  	fmt	0.400s
ok  	go/ast	0.298s
ok  	go/build	1.393s
ok  	go/constant	0.284s
ok  	go/doc	0.389s
ok  	go/format	0.382s
ok  	go/importer	0.296s
ok  	go/internal/gccgoimporter	0.382s
ok  	go/internal/gcimporter	0.291s
ok  	go/internal/srcimporter	4.194s
ok  	go/parser	0.361s
ok  	go/printer	0.537s
ok  	go/scanner	0.275s
ok  	go/token	0.275s
ok  	go/types	1.547s
ok  	hash	0.183s
ok  	hash/adler32	0.179s
ok  	hash/crc32	0.339s
ok  	hash/crc64	0.209s
ok  	hash/fnv	0.212s
ok  	hash/maphash	0.717s
ok  	html	0.641s
ok  	html/template	0.283s
ok  	image	0.322s
ok  	image/color	0.218s
ok  	image/draw	0.216s
ok  	image/gif	0.555s
ok  	image/jpeg	0.471s
ok  	image/png	0.494s
ok  	index/suffixarray	0.594s
ok  	internal/cpu	0.240s
ok  	internal/fmtsort	0.156s
ok  	internal/poll	0.182s
ok  	internal/profile	0.159s
ok  	internal/reflectlite	0.173s
ok  	internal/singleflight	0.185s
ok  	internal/trace	0.192s
ok  	internal/unsafeheader	0.253s
ok  	internal/xcoff	0.168s
ok  	io	0.294s
ok  	io/fs	0.154s
ok  	io/ioutil	0.401s
ok  	log	0.333s
ok  	log/syslog	1.558s
ok  	math	0.272s
ok  	math/big	0.666s
ok  	math/bits	0.379s
ok  	math/cmplx	0.165s
ok  	math/rand	0.237s
ok  	mime	0.239s
ok  	mime/multipart	0.434s
ok  	mime/quotedprintable	0.445s
ok  	net	1.907s
ok  	net/http	5.836s
ok  	net/http/cgi	1.129s
ok  	net/http/cookiejar	0.182s
ok  	net/http/fcgi	0.189s
ok  	net/http/httptest	0.325s
ok  	net/http/httptrace	0.162s
ok  	net/http/httputil	0.232s
ok  	net/http/internal	0.148s
ok  	net/http/pprof	5.399s
ok  	net/internal/socktest	0.363s
ok  	net/mail	0.194s
ok  	net/rpc	0.220s
ok  	net/rpc/jsonrpc	0.182s
ok  	net/smtp	0.179s
ok  	net/textproto	0.176s
ok  	net/url	0.153s
ok  	os	0.936s
ok  	os/exec	1.325s
ok  	os/signal	2.320s
ok  	os/user	0.301s
ok  	path	0.140s
ok  	path/filepath	0.153s
ok  	plugin	0.167s
ok  	reflect	0.524s
ok  	regexp	0.231s
ok  	regexp/syntax	0.768s
--- FAIL: ExampleFrames (0.00s)
got:
- more:true | runtime.Callers
- more:true | runtime_test.ExampleFrames.func1
- more:true | runtime_test.ExampleFrames.func2
- more:true | runtime_test.ExampleFrames.func3
- more:true | runtime_test.ExampleFrames
- more:true | testing.runExample
- more:true | testing.runExamples
- more:true | testing.(*M).Run
- more:true | runtime_test.TestMain
want:
- more:true | runtime.Callers
- more:true | runtime_test.ExampleFrames.func1
- more:true | runtime_test.ExampleFrames.func2
- more:true | runtime_test.ExampleFrames.func3
- more:true | runtime_test.ExampleFrames
FAIL
FAIL	runtime	68.642s
ok  	runtime/debug	0.186s
ok  	runtime/internal/atomic	0.314s
ok  	runtime/internal/math	0.150s
ok  	runtime/internal/sys	0.174s
ok  	runtime/metrics	0.289s
ok  	runtime/pprof	6.390s
ok  	runtime/race	0.478s
ok  	runtime/trace	0.655s
ok  	sort	0.189s
ok  	strconv	0.403s
ok  	strings	0.250s
ok  	sync	0.568s
ok  	sync/atomic	0.216s
ok  	syscall	0.335s
ok  	testing	0.633s
ok  	testing/fstest	0.218s
ok  	testing/iotest	0.168s
ok  	testing/quick	0.200s
ok  	text/scanner	0.392s
ok  	text/tabwriter	0.316s
ok  	text/template	0.189s
ok  	text/template/parse	0.156s
ok  	time	2.770s
ok  	unicode	0.156s
ok  	unicode/utf16	0.160s
ok  	unicode/utf8	0.182s
ok  	cmd/addr2line	4.773s
ok  	cmd/api	5.862s
ok  	cmd/asm/internal/asm	1.224s
ok  	cmd/asm/internal/lex	0.170s
ok  	cmd/compile	0.202s
ok  	cmd/compile/internal/gc	10.480s
ok  	cmd/compile/internal/logopt	0.210s
ok  	cmd/compile/internal/ssa	0.668s
ok  	cmd/compile/internal/syntax	0.166s
ok  	cmd/compile/internal/test	0.207s
ok  	cmd/compile/internal/types	0.170s
ok  	cmd/cover	3.462s
ok  	cmd/doc	0.358s
ok  	cmd/fix	1.869s
ok  	cmd/go	63.549s
ok  	cmd/go/internal/auth	0.162s
ok  	cmd/go/internal/cache	0.383s
ok  	cmd/go/internal/fsys	0.194s
ok  	cmd/go/internal/generate	0.293s
ok  	cmd/go/internal/get	0.312s
ok  	cmd/go/internal/imports	0.218s
ok  	cmd/go/internal/load	0.183s
ok  	cmd/go/internal/lockedfile	0.228s
ok  	cmd/go/internal/lockedfile/internal/filelock	0.176s
ok  	cmd/go/internal/modconv	0.195s
ok  	cmd/go/internal/modfetch	0.256s
ok  	cmd/go/internal/modfetch/codehost	0.217s
ok  	cmd/go/internal/modfetch/zip_sum_test	0.192s
ok  	cmd/go/internal/modload	0.512s
ok  	cmd/go/internal/mvs	0.189s
ok  	cmd/go/internal/par	0.244s
ok  	cmd/go/internal/renameio	2.166s
ok  	cmd/go/internal/search	0.662s
ok  	cmd/go/internal/str	0.595s
ok  	cmd/go/internal/test	0.364s
ok  	cmd/go/internal/txtar	0.168s
ok  	cmd/go/internal/vcs	0.219s
ok  	cmd/go/internal/web	0.184s
ok  	cmd/go/internal/work	0.481s
ok  	cmd/gofmt	0.202s
ok  	cmd/internal/archive	3.054s
ok  	cmd/internal/buildid	0.374s
ok  	cmd/internal/dwarf	0.336s
ok  	cmd/internal/edit	0.418s
ok  	cmd/internal/goobj	1.053s
ok  	cmd/internal/moddeps	7.314s
ok  	cmd/internal/obj	0.603s
ok  	cmd/internal/obj/arm64	0.433s
ok  	cmd/internal/obj/ppc64	0.453s
ok  	cmd/internal/obj/riscv	0.318s
ok  	cmd/internal/obj/s390x	0.149s
ok  	cmd/internal/obj/x86	7.075s
ok  	cmd/internal/objabi	0.196s
ok  	cmd/internal/pkgpath	0.273s
ok  	cmd/internal/src	0.142s
ok  	cmd/internal/sys	0.206s
ok  	cmd/internal/test2json	0.339s
ok  	cmd/link	8.938s
ok  	cmd/link/internal/benchmark	0.498s
ok  	cmd/link/internal/ld	13.037s
ok  	cmd/link/internal/loader	0.310s
ok  	cmd/nm	5.512s
ok  	cmd/objdump	8.401s
ok  	cmd/pack	4.955s
ok  	cmd/trace	0.613s
ok  	cmd/vet	21.658s
FAIL
go tool dist: Failed: exit status 1

[cherrymui]

Hmmm, I cannot reproduce the ExampleFrames failure. Maybe try again?

[makew0rld]

#38485 (comment)

Can confirm I'm also getting these errors when building for darwin/arm64 on 64-bit Linux, with Go version 1.15.5

/usr/lib/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/tmp/go-link-303716583/go.o: file not recognized: file format not recognized
collect2: error: ld returned 1 exit status

[cherrymui]

@makeworld-the-better-one Go 1.15.x does not support macOS/ARM64 port (see #38485 (comment)). Use tip of the master branch.

[cherrymui]

As Go 1.16 beta is released, now macOS ARM64 binary release can be downloaded at https://golang.org/dl/#go1.16beta1 .

[enty8080]

@cherrymui

Problem: #43371 with ios/arm64 executables, the will not execute on iOS devices without specified min version.

I think this should be fixed.

[skyzyx]

Everyone probably knows this, but for anyone who comes across this in the future, Go 1.16 was released today with non-prerelease, first-class support for darwin/arm64. https://golang.org/doc/go1.16#darwin

[gopherbot]

Change https://golang.org/cl/344069 mentions this issue: internal/dl: change GOARCH=arm64 pretty string from ARMv8 to ARM64