cmd/link: LC_UUID not generated by go linker, resulting in failure to access local network on macOS 15 #68678

torarnv · 2024-07-31T11:38:55Z

Go version

go version go1.22.5 darwin/arm64

Output of `go env` in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/torarne/Library/Caches/go-build'
GOENV='/Users/torarne/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/var/folders/7w/09x_jw654938vd439254v6r80000gn/T//gomodcache'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/torarne/.go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.22.5/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.22.5/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.5'
GCCGO='gccgo'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/7w/09x_jw654938vd439254v6r80000gn/T/go-build1376446515=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

Go binaries built with the default linker for macOS do not have a LC_UUID load command. This UUID seems important to the network privacy machinery on macOS, that in macOS 15 now includes local network access.

Running a small go binary that does a HTTP request to a local network HTTP server results in a permission request from the OS, but accepting this does not result in the go process being able to access the server.

The logs show:
nehelper +[NEProcessInfo copyUUIDsForExecutable:]_block_invoke: failed to get UUIDs for /Users/foo/FetchGo

So it seems the nehelper relies on the binary having a UUID to allow network requests through.

I can work around this by passing -ldflags="-linkmode=external" when building the go binary, which does produce a UUID:

❯ dwarfdump -u FetchGo
UUID: C7B47B23-54A7-3F52-837D-E455B57855A1 (arm64) FetchGo

And in this case the HTTP request works after accepting the privacy dialog. But I don't think this will work when cross-compiling darwin binaries on Linux e.g.

Note that for testing this, the binary must be run as an app bundle, or launch agent -- not via SSH/Terminal, as in the latter case the permission to access the network is inherited by those processes, and they have an UUID.

What did you see happen?

Network request fails to route

What did you expect to see?

Network request succeeds

The text was updated successfully, but these errors were encountered:

torarnv · 2024-07-31T12:55:53Z

Thanks @seankhliao :)

torarnv · 2024-07-31T13:37:05Z

Sample reproducer. Change plist files to point to web server running in local network. Loading the launch agents should bring up permission dialogs.

missing-lc-uuid.zip

ianlancetaylor · 2024-07-31T13:46:28Z

CC @cherrymui

torarnv · 2024-07-31T19:07:14Z

See also https://forums.developer.apple.com/forums/thread/737416 for a description of the UUID mechanism.

vsarunas · 2024-09-18T19:28:51Z

Also ran into this; this now affects all existing Go applications that are launched via launchctl; even when the macOS Sequoia firewall is off.

Go application on the new OS simply has an error: connect: no route to host

Minimal example:

package main
import (
	"fmt"
	"net"
	"os"
	"time"
)
func main() {
	if len(os.Args) != 2 {
		fmt.Println("Usage: <address:port>")
		return
	}
	for {
		if conn, err := net.Dial("tcp", os.Args[1]); err == nil {
			buf := make([]byte, 1024)
			n, _ := conn.Read(buf)
			fmt.Println("Connected, response:", string(buf[:n]))
			conn.Close()
		} else {
			fmt.Println("Error:", err, "Retrying...")
		}
		time.Sleep(1 * time.Second)
	}
}

Build:

go build -o /opt/homebrew/bin/go-connect error-test.go

Config for auto start and restart:

cat <<EOF > ~/Library/LaunchAgents/go-connect.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>go-connect</string>
    <key>ProgramArguments</key>
    <array>
        <string>/opt/homebrew/bin/go-connect</string>
        <string>192.168.1.217:22</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardOutPath</key>
    <string>/opt/homebrew/var/log/go-connect</string>
</dict>
</plist>
EOF

Start:

launchctl load -w ~/Library/LaunchAgents/go-connect.plist

Monitor failures:

tail -f /opt/homebrew/var/log/go-connect 
Error: dial tcp 192.168.1.217:22: connect: no route to host Retrying...
Error: dial tcp 192.168.1.217:22: connect: no route to host Retrying...
Error: dial tcp 192.168.1.217:22: connect: no route to host Retrying...

Network is fine:

ping  192.168.1.217    
PING 192.168.1.217 (192.168.1.217): 56 data bytes
64 bytes from 192.168.1.217: icmp_seq=0 ttl=64 time=6.015 ms
64 bytes from 192.168.1.217: icmp_seq=1 ttl=64 time=4.680 ms

arp -an | grep 192.168.1.217
? (192.168.1.217) at bc:d0:74[...] on en0 ifscope [ethernet]

Reported this to Apple as:

Go binaries do not have LC_UUID and result in network failures [NEProcessInfo copyUUIDsForExecutable:]_block_invoke: failed to get UUID #FB15168752```

gopherbot · 2024-10-08T19:11:05Z

Change https://go.dev/cl/618595 mentions this issue: cmd/link: generate Mach-O UUID when -B flag is specified

gopherbot · 2024-10-08T19:11:06Z

Change https://go.dev/cl/618597 mentions this issue: cmd/internal/buildid: skip over Mach-O UUID from buildid computation

gopherbot · 2024-10-08T19:11:07Z

Change https://go.dev/cl/618598 mentions this issue: cmd/link: on Mach-O, generate LC_UUID by default

gopherbot · 2024-10-08T19:11:07Z

Change https://go.dev/cl/618596 mentions this issue: cmd/internal/macho: new package for handling mach-o files in toolchain

gopherbot · 2024-10-08T19:11:08Z

Change https://go.dev/cl/618599 mentions this issue: cmd/link: apply -B UUID to external linking on Mach-O

Currently, on Mach-O, the Go linker doesn't generate LC_UUID in internal linking mode. This causes some macOS system tools unable to track the binary, as well as in some cases the binary unable to access local network on macOS 15. This CL makes the linker start generate LC_UUID. Currently, the UUID is generated if the -B flag is specified. And we'll make it generate UUID by default in a later CL. The -B flag is currently for generating GNU build ID on ELF, which is a similar concept to Mach-O's UUID. Instead of introducing another flag, we just use the same flag and the same setting. Specifically, "-B gobuildid" will generate a UUID based on the Go build ID. For #68678. Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64_14,gotip-darwin-arm64_13 Change-Id: I90089a78ba144110bf06c1c6836daf2d737ff10a Reviewed-on: https://go-review.googlesource.com/c/go/+/618595 Reviewed-by: Michael Knyszek <[email protected]> Reviewed-by: Ingo Oeser <[email protected]> Reviewed-by: Than McIntosh <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

Currently the linker has some code handling and manipulating Mach-O files. Specifically, it augments the debug/macho package with file offset and length, so the content can be handled or updated easily with the file. Move this code to an internal package, so it can be used by other part of the toolchain, e.g. buildid computation. For #68678. Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64_14,gotip-darwin-arm64_13 Change-Id: I2311af0a06441b7fd887ca5c6ed9e6fc44670a16 Reviewed-on: https://go-review.googlesource.com/c/go/+/618596 Reviewed-by: Than McIntosh <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

With the "-B gobuildid" linker option (which will be the default on some platforms), the host build ID (GNU build ID, Mach-O UUID) depends on the Go buildid. If the host build ID is included in the Go buildid computation, it will lead to convergence problem for the toolchain binaries. So ignore the host build ID in the buildid computation. This CL only handles Mach-O UUID. ELF GNU build ID will be handled later. For #68678. For #63934. Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64_14,gotip-darwin-arm64_13 Change-Id: Ie8ff20402a1c6083246d25dea391140c75be40d0 Reviewed-on: https://go-review.googlesource.com/c/go/+/618597 Reviewed-by: Michael Knyszek <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Than McIntosh <[email protected]>

Currently, on Mach-O, the -B UUID setting is only applied in internal linking mode, whereas in external linking mode the UUID is always rewritten to a hash of Go build ID. This CL makes it apply to external linking as well. This makes the behavior consistent on both linkmodes, and also consistent with the -B flag's behavior for GNU build ID on ELF. Add tests. Updates #68678. Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64_14,gotip-darwin-arm64_13 Change-Id: I276a5930e231141440cdba16e8812df28ac4237b Reviewed-on: https://go-review.googlesource.com/c/go/+/618599 Reviewed-by: Than McIntosh <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>

cherrymui · 2024-10-22T20:50:44Z

@gopherbot please backport this. This causes programs fail to access local network on macOS 15.

The full fix would be backporting CL 618595, CL 618596, CL 618597, and CL 618598, which may be too complex for backport. A simpler alternative is backporting just CL 618595, which enables generating LC_UUID with a flag. It still needs a flag, but not the C toolchain, therefore helpful for cross compilation. We might be able to do a hacky way to enable it by default except for the toolchain binaries. I'll look into that.

gopherbot · 2024-10-22T20:51:31Z

Backport issue(s) opened: #69991 (for 1.22), #69992 (for 1.23).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

gopherbot · 2024-10-25T15:17:55Z

Change https://go.dev/cl/622595 mentions this issue: [release-branch.go1.23] cmd/link: generate Mach-O UUID when -B flag is specified

gopherbot · 2024-10-25T15:23:21Z

Change https://go.dev/cl/622596 mentions this issue: [release-branch.go1.22] cmd/link: generate Mach-O UUID when -B flag is specified

…s specified Currently, on Mach-O, the Go linker doesn't generate LC_UUID in internal linking mode. This causes some macOS system tools unable to track the binary, as well as in some cases the binary unable to access local network on macOS 15. This CL makes the linker start generate LC_UUID. Currently, the UUID is generated if the -B flag is specified. And we'll make it generate UUID by default in a later CL. The -B flag is currently for generating GNU build ID on ELF, which is a similar concept to Mach-O's UUID. Instead of introducing another flag, we just use the same flag and the same setting. Specifically, "-B gobuildid" will generate a UUID based on the Go build ID. Updates #68678. Fixes #69991. Cq-Include-Trybots: luci.golang.try:go1.22-darwin-amd64_14,go1.22-darwin-arm64_13 Change-Id: I90089a78ba144110bf06c1c6836daf2d737ff10a Reviewed-on: https://go-review.googlesource.com/c/go/+/618595 Reviewed-by: Michael Knyszek <[email protected]> Reviewed-by: Ingo Oeser <[email protected]> Reviewed-by: Than McIntosh <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> (cherry picked from commit 20ed603) Reviewed-on: https://go-review.googlesource.com/c/go/+/622596 Auto-Submit: Michael Pratt <[email protected]> Reviewed-by: Michael Pratt <[email protected]> TryBot-Bypass: Michael Pratt <[email protected]>

…s specified Currently, on Mach-O, the Go linker doesn't generate LC_UUID in internal linking mode. This causes some macOS system tools unable to track the binary, as well as in some cases the binary unable to access local network on macOS 15. This CL makes the linker start generate LC_UUID. Currently, the UUID is generated if the -B flag is specified. And we'll make it generate UUID by default in a later CL. The -B flag is currently for generating GNU build ID on ELF, which is a similar concept to Mach-O's UUID. Instead of introducing another flag, we just use the same flag and the same setting. Specifically, "-B gobuildid" will generate a UUID based on the Go build ID. Updates #68678. Fixes #69992. Cq-Include-Trybots: luci.golang.try:go1.23-darwin-amd64_14,go1.23-darwin-arm64_13 Change-Id: I90089a78ba144110bf06c1c6836daf2d737ff10a Reviewed-on: https://go-review.googlesource.com/c/go/+/618595 Reviewed-by: Michael Knyszek <[email protected]> Reviewed-by: Ingo Oeser <[email protected]> Reviewed-by: Than McIntosh <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> (cherry picked from commit 20ed603) Reviewed-on: https://go-review.googlesource.com/c/go/+/622595 Reviewed-by: Michael Pratt <[email protected]>

cherrymui · 2024-10-30T18:06:34Z

Just give an update: This is fixed on tip (to be released in Go 1.24), by generating LC_UUID by default.

For previous releases (Go 1.22 and 1.23), we did a simpler backport, which requires a Go linker flag (-B gobuildid) to generate the UUID. Compared to -linkmode=external, this does not require a C toolchain installation (and therefore easier to work with in a cross compilation setting). This will be included in the next Go 1.22 and 1.23 minor releases (early November).

stanhu · 2024-11-15T18:17:38Z

This change is now breaking our macOS builds: #70380

dmitshur · 2024-11-26T23:40:08Z

A comment on the CL suggested that this change should be mentioned in Go 1.24 release notes. Reopening this as a release blocker to track that.

gopherbot · 2024-12-04T22:34:02Z

Change https://go.dev/cl/633875 mentions this issue: doc/next: add linker change release notes

seankhliao changed the title ~~LC_UUID not generated by go linker, resulting in failure to access local network on macOS 15~~ cmd/link: LC_UUID not generated by go linker, resulting in failure to access local network on macOS 15 Jul 31, 2024

gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 31, 2024

seankhliao added the OS-Darwin label Jul 31, 2024

mknyszek added this to the Backlog milestone Jul 31, 2024

mknyszek added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 31, 2024

mknyszek added this to Go Compiler / Runtime Jul 31, 2024

cherrymui self-assigned this Jul 31, 2024

mknyszek moved this to Todo in Go Compiler / Runtime Aug 7, 2024

vsarunas mentioned this issue Sep 18, 2024

Consul on macOS Sequoia is unable to connect to server hashicorp/consul#21765

Open

edigaryev mentioned this issue Sep 20, 2024

Cannot connect to VM when ran from launch agent cirruslabs/gitlab-tart-executor#85

Closed

This was referenced Sep 24, 2024

Consul on macOS Sequoia is unable to connect to server dhiaayachi/consul#19

Open

Consul on macOS Sequoia is unable to connect to server dhiaayachi/consul#39

Open

gopherbot closed this as completed in 6a4feb5 Oct 21, 2024

github-project-automation bot moved this from Todo to Done in Go Compiler / Runtime Oct 21, 2024

dmitshur removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 21, 2024

gopherbot mentioned this issue Oct 22, 2024

cmd/link: LC_UUID not generated by go linker, resulting in failure to access local network on macOS 15 [1.22 backport] #69991

Closed

gopherbot mentioned this issue Oct 22, 2024

cmd/link: LC_UUID not generated by go linker, resulting in failure to access local network on macOS 15 [1.23 backport] #69992

Closed

ispysoftware mentioned this issue Oct 31, 2024

Connections dropping on OSX Sequoia sipsorcery-org/sipsorcery#1197

Closed

timpeeters mentioned this issue Nov 12, 2024

running as root no longer works in Sequoia cirruslabs/tart#921

Closed

uberjay mentioned this issue Nov 13, 2024

the hostname could not be number moul/assh#674

Open

stanhu mentioned this issue Nov 15, 2024

cmd/link: running go build -ldflags "-B <SHA1>" fails in Go 1.23.3 on macOS #70380

Closed

StefanSchoof mentioned this issue Nov 17, 2024

MacOS: New server not functioning, get "connect: no route to host" error messages evcc-io/evcc#17251

Closed

1 task

timpeeters mentioned this issue Nov 20, 2024

Launchd worker fails with "no route to host" on Sequoia cirruslabs/orchard#221

Closed

This was referenced Nov 20, 2024

Fix #221 by bumping Go. cirruslabs/orchard#223

Merged

build: ensure that LC_UUID is generated for Darwin binaries hashicorp/packer#13214

Merged

dmitshur reopened this Nov 26, 2024

github-project-automation bot moved this from Done to In Progress in Go Compiler / Runtime Nov 26, 2024

dmitshur added the release-blocker label Nov 26, 2024

dmitshur mentioned this issue Nov 26, 2024

doc: write release notes for Go 1.24 #68545

Open

GoVeronicaGo added the Documentation Issues describing a change to documentation. label Nov 27, 2024

andig mentioned this issue Nov 29, 2024

Cannot override ldflags goreleaser/goreleaser#5322

Closed

4 tasks

vsarunas mentioned this issue Dec 5, 2024

Unable to connect to instance on MacOS 15.1 (no route to host) canonical/multipass#3766

Open

gopherbot closed this as completed in be0c454 Dec 5, 2024

github-project-automation bot moved this from In Progress to Done in Go Compiler / Runtime Dec 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/link: LC_UUID not generated by go linker, resulting in failure to access local network on macOS 15 #68678

cmd/link: LC_UUID not generated by go linker, resulting in failure to access local network on macOS 15 #68678

torarnv commented Jul 31, 2024 •

edited

Loading

torarnv commented Jul 31, 2024

torarnv commented Jul 31, 2024

ianlancetaylor commented Jul 31, 2024

torarnv commented Jul 31, 2024

vsarunas commented Sep 18, 2024

gopherbot commented Oct 8, 2024

gopherbot commented Oct 8, 2024

gopherbot commented Oct 8, 2024

gopherbot commented Oct 8, 2024

gopherbot commented Oct 8, 2024

cherrymui commented Oct 22, 2024 •

edited by gabyhelp

Loading

gopherbot commented Oct 22, 2024

gopherbot commented Oct 25, 2024

gopherbot commented Oct 25, 2024

cherrymui commented Oct 30, 2024 •

edited

Loading

stanhu commented Nov 15, 2024

dmitshur commented Nov 26, 2024

gopherbot commented Dec 4, 2024

cmd/link: LC_UUID not generated by go linker, resulting in failure to access local network on macOS 15 #68678

cmd/link: LC_UUID not generated by go linker, resulting in failure to access local network on macOS 15 #68678

Comments

torarnv commented Jul 31, 2024 • edited Loading

Go version

Output of go env in your module/workspace:

What did you do?

What did you see happen?

What did you expect to see?

torarnv commented Jul 31, 2024

torarnv commented Jul 31, 2024

ianlancetaylor commented Jul 31, 2024

torarnv commented Jul 31, 2024

vsarunas commented Sep 18, 2024

gopherbot commented Oct 8, 2024

gopherbot commented Oct 8, 2024

gopherbot commented Oct 8, 2024

gopherbot commented Oct 8, 2024

gopherbot commented Oct 8, 2024

cherrymui commented Oct 22, 2024 • edited by gabyhelp Loading

gopherbot commented Oct 22, 2024

gopherbot commented Oct 25, 2024

gopherbot commented Oct 25, 2024

cherrymui commented Oct 30, 2024 • edited Loading

stanhu commented Nov 15, 2024

dmitshur commented Nov 26, 2024

gopherbot commented Dec 4, 2024

torarnv commented Jul 31, 2024 •

edited

Loading

Output of `go env` in your module/workspace:

cherrymui commented Oct 22, 2024 •

edited by gabyhelp

Loading

cherrymui commented Oct 30, 2024 •

edited

Loading