-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: update obfs4, goptlib, snowflake #1347
Conversation
This diff upgrades obfs4, goptlib, and snowflake. I tried to import lyrebird instead of obfs4, but that did not work as intended. It seems tags aren't compatible with the format expected by Go (vX.Y.Z) and I did not want to pin to the latest commit. I will report this issue to lyrebird developers later today. While there, also upgrade x/net and x/exp. This work is part of ooni/probe#2524. Here's what we could not upgrade (based on the content of `go.mod` after we committed this diff and ran `go get -u -v -d ./...`): ```diff diff --git a/go.mod b/go.mod index 67e0af02..18175d73 100644 --- a/go.mod +++ b/go.mod @@ -80,7 +80,7 @@ require ( golang.org/x/exp/typeparams v0.0.0-20230522175609-2e198f4a06a1 // indirect golang.org/x/time v0.3.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect - gvisor.dev/gvisor v0.0.0-20230603040744-5c9219dedd33 // indirect + gvisor.dev/gvisor v0.0.0-20231006032704-15cc3fcbbd77 // indirect ) require ( @@ -89,7 +89,7 @@ require ( github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect github.com/armon/go-proxyproto v0.0.0-20210323213023-7e956b284f0a // indirect github.com/beorn7/perks v1.0.1 // indirect - github.com/bifurcation/mint v0.0.0-20180306135233-198357931e61 // indirect + github.com/bifurcation/mint v0.0.0-20210616192047-fd18df995463 // indirect github.com/cespare/xxhash/v2 v2.2.0 // indirect github.com/cheekybits/genny v1.0.0 // indirect github.com/cognusion/go-cache-lru v0.0.0-20170419142635-f73e2280ecea // indirect @@ -132,7 +132,7 @@ require ( github.com/prometheus/common v0.44.0 // indirect github.com/prometheus/procfs v0.12.0 // indirect github.com/refraction-networking/gotapdance v1.7.4 // indirect - github.com/refraction-networking/utls v1.3.3 // indirect + github.com/refraction-networking/utls v1.5.3 // indirect github.com/sergeyfrolov/bsbuffer v0.0.0-20180903213811-94e85abb8507 // indirect github.com/sirupsen/logrus v1.9.3 // indirect github.com/spf13/cobra v1.7.0 @@ -142,10 +142,10 @@ require ( github.com/templexxx/xorsimd v0.4.2 // indirect github.com/tjfoc/gmsm v1.4.1 // indirect github.com/wader/filtertransport v0.0.0-20200316221534-bdd9e61eee78 // indirect - github.com/xtaci/kcp-go/v5 v5.6.2 // indirect + github.com/xtaci/kcp-go/v5 v5.6.3 // indirect github.com/xtaci/smux v1.5.24 // indirect gitlab.com/yawning/bsaes.git v0.0.0-20190805113838-0a714cd429ec // indirect - golang.org/x/mod v0.12.0 // indirect + golang.org/x/mod v0.13.0 // indirect golang.org/x/term v0.13.0 // indirect golang.org/x/text v0.13.0 // indirect golang.org/x/tools v0.13.0 // indirect ``` We cannot upgrade: - gvisor because (a) we're fine with our pinned version used by netem and (b) the main branch of gvistor is not suitable for go-getting anyway - mint because this is the version Psiphon requires - utls because this is the version Psiphon requires - kcp-go because v.5.6.3 requires go1.21 - x/mod which seems to be needed by quic-go to generate mocks, so not upgrading it isn't a big deal.
Uhm, before I wrote:
However, this sentence is inaccurate. Here's what happens. First, we need to create a new project: $ mkdir example
$ cd example
$ go mod init example.com/lyrebird Then, we create the following package main
import "gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird"
func main() {
txp := &lyrebird.Transport{}
_ = txp
} Then we run: $ go mod tidy -x
go: finding module for package gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird
# get https://proxy.golang.org/gitlab.torproject.org/@v/list
# get https://proxy.golang.org/gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/@v/list
# get https://proxy.golang.org/gitlab.torproject.org/tpo/anti-censorship/@v/list
# get https://proxy.golang.org/gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird/@v/list
# get https://proxy.golang.org/gitlab.torproject.org/tpo/@v/list
# get https://proxy.golang.org/gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/@v/list: 404 Not Found (0.151s)
# get https://proxy.golang.org/gitlab.torproject.org/tpo/@v/list: 404 Not Found (0.188s)
# get https://proxy.golang.org/gitlab.torproject.org/@v/list: 404 Not Found (0.188s)
# get https://proxy.golang.org/gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird/@v/list: 200 OK (0.190s)
# get https://proxy.golang.org/gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird/@latest
# get https://proxy.golang.org/gitlab.torproject.org/tpo/anti-censorship/@v/list: 404 Not Found (0.191s)
# get https://proxy.golang.org/gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird/@latest: 200 OK (0.046s)
example.com/lyrebird imports
gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird: module gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird@latest found (v0.0.0-20231005141435-20cf093f50ae), but does not contain package gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird The commit in there is the latest commit as of today. Hence the issue is not related with tags. Running What am I missing? Ah, so the issue seems to be the package name that I am importing. This package main
import "gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird/transports/obfs4"
func main() {
txp := &obfs4.Transport{}
_ = txp
} Leads to: $ go mod tidy
go: finding module for package gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird/transports/obfs4
go: found gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird/transports/obfs4 in gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird v0.0.0-20231005141435-20cf093f50ae Hence, it turns out we had a case of PEBCAK 🤦 I'll make a new PR to switch to lyrebird! |
This diff replaces obfs4 and uses lyrebird instead. See #1347 (comment) to understand why my previous assessment that we could not import lyrebird was actually caused by PEBCAK. The build currently fails because the set of dependencies required by Psiphon conflict with lyrebird ones, so upgrading isn't possible. Here is how the build fails: ```console $ go build -v ./... github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon/common/protocol /home/sbs/go/pkg/mod/github.com/!psiphon-!labs/[email protected]/psiphon/common/protocol/customTLSProfiles.go:180:16: undefined: utls.UtlsExtendedMasterSecretExtension ``` So, I think the underlying issue here may be that utls has removed some symbols without bumping their major version number. If it was possible to merge this work, this work would be part of the ooni/probe#2524 issue.
This diff upgrades obfs4, goptlib, and snowflake. I tried to import lyrebird instead of obfs4, but that did not work as intended. It seems tags aren't compatible with the format expected by Go (vX.Y.Z) and I did not want to pin to the latest commit. I will report this issue to lyrebird developers later today. While there, also upgrade x/net and x/exp. This work is part of ooni/probe#2524. Here's what we could not upgrade (based on the content of `go.mod` after we committed this diff and ran `go get -u -v -d ./...`): ```diff diff --git a/go.mod b/go.mod index 67e0af02..18175d73 100644 --- a/go.mod +++ b/go.mod @@ -80,7 +80,7 @@ require ( golang.org/x/exp/typeparams v0.0.0-20230522175609-2e198f4a06a1 // indirect golang.org/x/time v0.3.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect - gvisor.dev/gvisor v0.0.0-20230603040744-5c9219dedd33 // indirect + gvisor.dev/gvisor v0.0.0-20231006032704-15cc3fcbbd77 // indirect ) require ( @@ -89,7 +89,7 @@ require ( github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect github.com/armon/go-proxyproto v0.0.0-20210323213023-7e956b284f0a // indirect github.com/beorn7/perks v1.0.1 // indirect - github.com/bifurcation/mint v0.0.0-20180306135233-198357931e61 // indirect + github.com/bifurcation/mint v0.0.0-20210616192047-fd18df995463 // indirect github.com/cespare/xxhash/v2 v2.2.0 // indirect github.com/cheekybits/genny v1.0.0 // indirect github.com/cognusion/go-cache-lru v0.0.0-20170419142635-f73e2280ecea // indirect @@ -132,7 +132,7 @@ require ( github.com/prometheus/common v0.44.0 // indirect github.com/prometheus/procfs v0.12.0 // indirect github.com/refraction-networking/gotapdance v1.7.4 // indirect - github.com/refraction-networking/utls v1.3.3 // indirect + github.com/refraction-networking/utls v1.5.3 // indirect github.com/sergeyfrolov/bsbuffer v0.0.0-20180903213811-94e85abb8507 // indirect github.com/sirupsen/logrus v1.9.3 // indirect github.com/spf13/cobra v1.7.0 @@ -142,10 +142,10 @@ require ( github.com/templexxx/xorsimd v0.4.2 // indirect github.com/tjfoc/gmsm v1.4.1 // indirect github.com/wader/filtertransport v0.0.0-20200316221534-bdd9e61eee78 // indirect - github.com/xtaci/kcp-go/v5 v5.6.2 // indirect + github.com/xtaci/kcp-go/v5 v5.6.3 // indirect github.com/xtaci/smux v1.5.24 // indirect gitlab.com/yawning/bsaes.git v0.0.0-20190805113838-0a714cd429ec // indirect - golang.org/x/mod v0.12.0 // indirect + golang.org/x/mod v0.13.0 // indirect golang.org/x/term v0.13.0 // indirect golang.org/x/text v0.13.0 // indirect golang.org/x/tools v0.13.0 // indirect ``` We cannot upgrade: - gvisor because (a) we're fine with our pinned version used by netem and (b) the main branch of gvistor is not suitable for go-getting anyway - mint because this is the version Psiphon requires - utls because this is the version Psiphon requires - kcp-go because v.5.6.3 requires go1.21 - x/mod which seems to be needed by quic-go to generate mocks, so not upgrading it isn't a big deal.
This diff upgrades obfs4, goptlib, and snowflake. I tried to import lyrebird instead of obfs4, but that did not work as intended. It seems tags aren't compatible with the format expected by Go (vX.Y.Z) and I did not want to pin to the latest commit. I will report this issue to lyrebird developers later today.
While there, also upgrade x/net and x/exp.
This work is part of ooni/probe#2524.
Here's what we could not upgrade (based on the content of
go.mod
after we committed this diff and rango get -u -v -d ./...
):We cannot upgrade:
gvisor because (a) we're fine with our pinned version used by netem and (b) the main branch of gvistor is not suitable for go-getting anyway
mint because this is the version Psiphon requires
utls because this is the version Psiphon requires
kcp-go because v.5.6.3 requires go1.21
x/mod which seems to be needed by quic-go to generate mocks, so not upgrading it isn't a big deal.