go-ipfs: update to 0.17.0. #40919

sgn · 2022-12-05T16:39:18Z

Testing the changes

I tested the changes in this PR: NO

sgn · 2022-12-05T16:40:13Z

CMB · 2022-12-05T17:17:33Z

Upstream renamed to kubo. Why not rename the package to follow
upstream's name?
link to relevant section of the void-packages manual

We also have a hot mess in the form of the fs-repo-migrations package.
This is a thing that gets used once after upgrade, to convert the IPFS
repository layout and metadata to a possibly new format.
Most distros don't even bother packaging it, because the ipfs binary
is capable of downloading and running repo migration binaries from the Internet.
When I made the go-ipfs package back in the day, I packaged fs-repo-migrations,
because downloading and running random binaries from the net is not cool.
Nix and Void are possibly the only two distros that package it.
Upstream has made it
difficult to package fs-repo-migrations issue 148,
but that problem is soluble. There's even a draft pull request for it!
Unfortunately, the package fails at runtime when built with Go 1.19.
fs-repo-migrations issue 156

So if this gets merged, our only recourse right now is to have people remove fs-repo-migrations and use binaries from upstream. Unfortunately that won't work for people on musl.

sgn · 2022-12-06T00:43:50Z

Upstream renamed to kubo. Why not rename the package to follow upstream's name? link to relevant section of the void-packages manual

I'm not care that must, the binary's name is still ipfs.

We also have a hot mess in the form of the fs-repo-migrations package. This is a thing that gets used once after upgrade, to convert the IPFS repository layout and metadata to a possibly new format. Most distros don't even bother packaging it, because the ipfs binary is capable of downloading and running repo migration binaries from the Internet. When I made the go-ipfs package back in the day, I packaged fs-repo-migrations, because downloading and running random binaries from the net is not cool. Nix and Void are possibly the only two distros that package it. Upstream has made it difficult to package fs-repo-migrations issue 148, but that problem is soluble. There's even a draft pull request for it! Unfortunately, the package fails at runtime when built with Go 1.19. fs-repo-migrations issue 156

Let's see.

So if this gets merged, our only recourse right now is to have people remove fs-repo-migrations and use binaries from upstream. Unfortunately that won't work for people on musl.

This is about trying to see if #39083 can be resolved.
I can look into them later.

CMB · 2022-12-06T00:58:45Z

I'm not care that must, the binary's name is still ipfs.

Fair enough.

This is about trying to see if #39083 can be resolved. I can look into them later.

Ok. So if it builds, then ship it I guess?

sug0 · 2022-12-17T16:51:15Z

it's broken under armv7l-musl, which is the host architecture of my ipfs node

Error: linux-musl not supported, you must build the binary from source for your platform

I think this is because it's trying to fetch fs-repo-migrations, and there isn't a build for my arch.

sgn · 2022-12-17T17:07:55Z

You need to install fs-repo-migrations package.

sug0 · 2022-12-17T19:28:35Z

fs-repo-migrations is segfaulting on armv7l-musl actually. there's another annoying issue: the migrations logic in ipfs is hard-coded to reject musl targets: https://github.com/ipfs/kubo/blob/430a65ad5f78dc2aed0f54946b2c5922fa4b1ee1/repo/fsrepo/migrations/migrations.go#L258-L260

technically it's possible to package each individual fs-repo-migrations binary, and remove fs-repo-migrations's package

sug0 · 2022-12-17T22:06:30Z

@sgn I've written a couple of patches that remove the need to package fs-repo-migrations:

I've updated the go-ipfs template. the diff against this PR is:

diff against this PR

diff --git a/srcpkgs/go-ipfs/files/README.voidlinux b/srcpkgs/go-ipfs/files/README.voidlinux
new file mode 100644
index 00000000..3840b85d
--- /dev/null
+++ b/srcpkgs/go-ipfs/files/README.voidlinux
@@ -0,0 +1,3 @@
+To migrate between repository versions, install `ipfs-migrate-repo`. This
+package provides the same functionality as `fs-repo-migrations`, but will
+work with `musl` targets, while `fs-repo-migrations` might not.
diff --git a/srcpkgs/go-ipfs/template b/srcpkgs/go-ipfs/template
index 316f1d36..1cd14657 100644
--- a/srcpkgs/go-ipfs/template
+++ b/srcpkgs/go-ipfs/template
@@ -6,7 +6,6 @@ build_style=go
 go_import_path="github.com/ipfs/kubo"
 go_package="${go_import_path}/cmd/ipfs ${go_import_path}/cmd/ipfswatch"
 hostmakedepends="git"
-depends="fs-repo-migrations>=1.7.1"
 short_desc="Global versioned P2P merkle DAG file system - kubo"
 maintainer="Christopher Brannon <[email protected]>"
 license="MIT, Apache-2.0"
@@ -29,5 +28,6 @@ post_install() {
 	vlicense LICENSE-MIT
 	vdoc README.md
 	vdoc CHANGELOG.md
+	vdoc "${FILESDIR}/README.voidlinux"
 	vsv ipfs
 }

now for the new packages, which replace fs-repo-migrations:

fs-repo-11-to-12

From afe3ade012b0172a4b33244064628dcab50dd62a Mon Sep 17 00:00:00 2001
From: Tiago Carvalho <[email protected]>
Date: Sat, 17 Dec 2022 21:28:06 +0000
Subject: [PATCH] New package: fs-repo-11-to-12-1.0.2_1

---
 .../0001-fs-repo-11-to-12-no-qtls-init.patch  | 31 +++++++++++++++++++
 srcpkgs/fs-repo-11-to-12/template             | 22 +++++++++++++
 2 files changed, 53 insertions(+)
 create mode 100644 srcpkgs/fs-repo-11-to-12/patches/0001-fs-repo-11-to-12-no-qtls-init.patch
 create mode 100644 srcpkgs/fs-repo-11-to-12/template

diff --git a/srcpkgs/fs-repo-11-to-12/patches/0001-fs-repo-11-to-12-no-qtls-init.patch b/srcpkgs/fs-repo-11-to-12/patches/0001-fs-repo-11-to-12-no-qtls-init.patch
new file mode 100644
index 00000000..f21ca95d
--- /dev/null
+++ b/srcpkgs/fs-repo-11-to-12/patches/0001-fs-repo-11-to-12-no-qtls-init.patch
@@ -0,0 +1,31 @@
+diff --git a/fs-repo-11-to-12/vendor/github.com/marten-seemann/qtls-go1-15/unsafe.go b/fs-repo-11-to-12/vendor/github.com/marten-seemann/qtls-go1-15/unsafe.go
+index 1bf13d8..94aa684 100644
+--- a/fs-repo-11-to-12/vendor/github.com/marten-seemann/qtls-go1-15/unsafe.go
++++ b/fs-repo-11-to-12/vendor/github.com/marten-seemann/qtls-go1-15/unsafe.go
+@@ -1,26 +1,10 @@
+ package qtls
+ 
+ import (
+-	"crypto/tls"
+ 	"reflect"
+ 	"unsafe"
+ )
+ 
+-func init() {
+-	if !structsEqual(&tls.ConnectionState{}, &connectionState{}) {
+-		panic("qtls.ConnectionState doesn't match")
+-	}
+-	if !structsEqual(&tls.ClientSessionState{}, &clientSessionState{}) {
+-		panic("qtls.ClientSessionState doesn't match")
+-	}
+-	if !structsEqual(&tls.Config{}, &config{}) {
+-		panic("qtls.Config doesn't match")
+-	}
+-	if !structsEqual(&tls.ClientHelloInfo{}, &clientHelloInfo{}) {
+-		panic("qtls.ClientHelloInfo doesn't match")
+-	}
+-}
+-
+ func toConnectionState(c connectionState) ConnectionState {
+ 	return *(*ConnectionState)(unsafe.Pointer(&c))
+ }
diff --git a/srcpkgs/fs-repo-11-to-12/template b/srcpkgs/fs-repo-11-to-12/template
new file mode 100644
index 00000000..93ff060b
--- /dev/null
+++ b/srcpkgs/fs-repo-11-to-12/template
@@ -0,0 +1,22 @@
+# Template file for 'fs-repo-11-to-12'
+pkgname=fs-repo-11-to-12
+version=1.0.2
+revision=1
+_tag=${pkgname}/v${version}
+build_style=go
+wrksrc="fs-repo-migrations-${pkgname}-v${version}"
+build_wrksrc="${pkgname}"
+go_import_path="github.com/ipfs/fs-repo-migrations/fs-repo-11-to-12"
+short_desc="Tool to upgrade an IPFS repository from version 11 to 12"
+maintainer="Orphaned <[email protected]>"
+license="MIT"
+homepage="https://github.com/ipfs/fs-repo-migrations"
+distfiles="https://github.com/ipfs/fs-repo-migrations/archive/${_tag}.tar.gz"
+checksum=e6d6320e280f3118873dfb194b68681fa2747b7893d99831fa6fa2bbb67ebd71
+
+# fix: collect2: fatal error: cannot find 'ld'
+export LDFLAGS="-fuse-ld=bfd"
+
+post_install() {
+	vlicense ../LICENSE
+}
-- 
2.39.0

fs-repo-12-to-13

not needed yet, but it's a future proof against go-ipfs 0.18, that receives yet another repository version upgrade

From e37a435c6467187dac7c931ccba51aaace9e29d5 Mon Sep 17 00:00:00 2001
From: Tiago Carvalho <[email protected]>
Date: Sat, 17 Dec 2022 21:34:38 +0000
Subject: [PATCH] New package: fs-repo-12-to-13-1.0.0_1

---
 srcpkgs/fs-repo-12-to-13/template | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 srcpkgs/fs-repo-12-to-13/template

diff --git a/srcpkgs/fs-repo-12-to-13/template b/srcpkgs/fs-repo-12-to-13/template
new file mode 100644
index 00000000..331b8dd8
--- /dev/null
+++ b/srcpkgs/fs-repo-12-to-13/template
@@ -0,0 +1,22 @@
+# Template file for 'fs-repo-12-to-13'
+pkgname=fs-repo-12-to-13
+version=1.0.0
+revision=1
+_tag=${pkgname}/v${version}
+build_style=go
+wrksrc="fs-repo-migrations-${pkgname}-v${version}"
+build_wrksrc="${pkgname}"
+go_import_path="github.com/ipfs/fs-repo-migrations/fs-repo-12-to-13"
+short_desc="Tool to upgrade an IPFS repository from version 12 to 13"
+maintainer="Orphaned <[email protected]>"
+license="MIT"
+homepage="https://github.com/ipfs/fs-repo-migrations"
+distfiles="https://github.com/ipfs/fs-repo-migrations/archive/${_tag}.tar.gz"
+checksum=454b32a86efc410d6edfef20726b36472c1d8f11abc60f2f0722d6fa8d65f94b
+
+# fix: collect2: fatal error: cannot find 'ld'
+export LDFLAGS="-fuse-ld=bfd"
+
+post_install() {
+	vlicense ../LICENSE
+}
-- 
2.39.0

ipfs-migrate-repo

From e65c4a3813de5edbfddb8a823fe50a529092322b Mon Sep 17 00:00:00 2001
From: Tiago Carvalho <[email protected]>
Date: Sat, 17 Dec 2022 22:26:35 +0000
Subject: [PATCH] New package: ipfs-migrate-repo-1.0.2_1

---
 srcpkgs/ipfs-migrate-repo/template | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100644 srcpkgs/ipfs-migrate-repo/template

diff --git a/srcpkgs/ipfs-migrate-repo/template b/srcpkgs/ipfs-migrate-repo/template
new file mode 100644
index 00000000..b032a988
--- /dev/null
+++ b/srcpkgs/ipfs-migrate-repo/template
@@ -0,0 +1,16 @@
+# Template file for 'ipfs-migrate-repo'
+pkgname=ipfs-migrate-repo
+version=1.0.2
+revision=1
+short_desc="Tool to upgrade an IPFS repository to newer versions"
+depends="fs-repo-11-to-12 fs-repo-12-to-13"
+maintainer="Orphaned <[email protected]>"
+license="GPL-2.0-or-later"
+homepage="https://github.com/sug0/ipfs-migrate-repo"
+distfiles="https://github.com/sug0/ipfs-migrate-repo/archive/v${version}.tar.gz"
+checksum=3e3064ddf0d10bc053f2f1fcea9989071d5ed9471cb924cc845fdf65870e3e4d
+
+do_install() {
+	vdoc README.md
+	vbin ipfs-migrate-repo
+}
-- 
2.39.0

unfortunately, armv7l-musl is segfaulting with fs-repo-11-to-12, but for some reason fs-repo-12-to-13 works fine...

sug0 · 2022-12-18T16:33:05Z

disabling cgo when building these new go packages fixes segmentation faults in armv7l-musl

fs-repo-11-to-12

From 39c9205a26fd6cc563e6261aedc4c6b7a8d178c8 Mon Sep 17 00:00:00 2001
From: Tiago Carvalho <[email protected]>
Date: Sat, 17 Dec 2022 21:28:06 +0000
Subject: [PATCH 1/2] New package: fs-repo-11-to-12-1.0.2_1

---
 .../0001-fs-repo-11-to-12-no-qtls-init.patch  | 31 +++++++++++++++++++
 srcpkgs/fs-repo-11-to-12/template             | 21 +++++++++++++
 2 files changed, 52 insertions(+)
 create mode 100644 srcpkgs/fs-repo-11-to-12/patches/0001-fs-repo-11-to-12-no-qtls-init.patch
 create mode 100644 srcpkgs/fs-repo-11-to-12/template

diff --git a/srcpkgs/fs-repo-11-to-12/patches/0001-fs-repo-11-to-12-no-qtls-init.patch b/srcpkgs/fs-repo-11-to-12/patches/0001-fs-repo-11-to-12-no-qtls-init.patch
new file mode 100644
index 00000000..f21ca95d
--- /dev/null
+++ b/srcpkgs/fs-repo-11-to-12/patches/0001-fs-repo-11-to-12-no-qtls-init.patch
@@ -0,0 +1,31 @@
+diff --git a/fs-repo-11-to-12/vendor/github.com/marten-seemann/qtls-go1-15/unsafe.go b/fs-repo-11-to-12/vendor/github.com/marten-seemann/qtls-go1-15/unsafe.go
+index 1bf13d8..94aa684 100644
+--- a/fs-repo-11-to-12/vendor/github.com/marten-seemann/qtls-go1-15/unsafe.go
++++ b/fs-repo-11-to-12/vendor/github.com/marten-seemann/qtls-go1-15/unsafe.go
+@@ -1,26 +1,10 @@
+ package qtls
+ 
+ import (
+-	"crypto/tls"
+ 	"reflect"
+ 	"unsafe"
+ )
+ 
+-func init() {
+-	if !structsEqual(&tls.ConnectionState{}, &connectionState{}) {
+-		panic("qtls.ConnectionState doesn't match")
+-	}
+-	if !structsEqual(&tls.ClientSessionState{}, &clientSessionState{}) {
+-		panic("qtls.ClientSessionState doesn't match")
+-	}
+-	if !structsEqual(&tls.Config{}, &config{}) {
+-		panic("qtls.Config doesn't match")
+-	}
+-	if !structsEqual(&tls.ClientHelloInfo{}, &clientHelloInfo{}) {
+-		panic("qtls.ClientHelloInfo doesn't match")
+-	}
+-}
+-
+ func toConnectionState(c connectionState) ConnectionState {
+ 	return *(*ConnectionState)(unsafe.Pointer(&c))
+ }
diff --git a/srcpkgs/fs-repo-11-to-12/template b/srcpkgs/fs-repo-11-to-12/template
new file mode 100644
index 00000000..07cd9f4f
--- /dev/null
+++ b/srcpkgs/fs-repo-11-to-12/template
@@ -0,0 +1,21 @@
+# Template file for 'fs-repo-11-to-12'
+pkgname=fs-repo-11-to-12
+version=1.0.2
+revision=1
+_tag=${pkgname}/v${version}
+build_style=go
+wrksrc="fs-repo-migrations-${pkgname}-v${version}"
+build_wrksrc="${pkgname}"
+go_import_path="github.com/ipfs/fs-repo-migrations/fs-repo-11-to-12"
+short_desc="Tool to upgrade an IPFS repository from version 11 to 12"
+maintainer="Orphaned <[email protected]>"
+license="MIT"
+homepage="https://github.com/ipfs/fs-repo-migrations"
+distfiles="https://github.com/ipfs/fs-repo-migrations/archive/${_tag}.tar.gz"
+checksum=e6d6320e280f3118873dfb194b68681fa2747b7893d99831fa6fa2bbb67ebd71
+
+export CGO_ENABLED=0
+
+post_install() {
+	vlicense ../LICENSE
+}
-- 
2.39.0

fs-repo-12-to-13

From ab3e71db9b054319b46aa7f33d9caa42a143b23d Mon Sep 17 00:00:00 2001
From: Tiago Carvalho <[email protected]>
Date: Sat, 17 Dec 2022 21:34:38 +0000
Subject: [PATCH 2/2] New package: fs-repo-12-to-13-1.0.0_1

---
 srcpkgs/fs-repo-12-to-13/template | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100644 srcpkgs/fs-repo-12-to-13/template

diff --git a/srcpkgs/fs-repo-12-to-13/template b/srcpkgs/fs-repo-12-to-13/template
new file mode 100644
index 00000000..a4dc1862
--- /dev/null
+++ b/srcpkgs/fs-repo-12-to-13/template
@@ -0,0 +1,21 @@
+# Template file for 'fs-repo-12-to-13'
+pkgname=fs-repo-12-to-13
+version=1.0.0
+revision=1
+_tag=${pkgname}/v${version}
+build_style=go
+wrksrc="fs-repo-migrations-${pkgname}-v${version}"
+build_wrksrc="${pkgname}"
+go_import_path="github.com/ipfs/fs-repo-migrations/fs-repo-12-to-13"
+short_desc="Tool to upgrade an IPFS repository from version 12 to 13"
+maintainer="Orphaned <[email protected]>"
+license="MIT"
+homepage="https://github.com/ipfs/fs-repo-migrations"
+distfiles="https://github.com/ipfs/fs-repo-migrations/archive/${_tag}.tar.gz"
+checksum=454b32a86efc410d6edfef20726b36472c1d8f11abc60f2f0722d6fa8d65f94b
+
+export CGO_ENABLED=0
+
+post_install() {
+	vlicense ../LICENSE
+}
-- 
2.39.0

These updated templates depend on #41167

sug0 · 2023-03-08T23:12:37Z

@sgn ping. what do you think of the patches I posted here?

github-actions · 2023-06-07T02:06:14Z

Pull Requests become stale 90 days after last activity and are closed 14 days after that. If this pull request is still relevant bump it or assign it.

eoli3n · 2023-08-04T07:34:13Z

is that dead ?
IMO, the package should be renamed "kubo" and 0.21.0 is out.

go-ipfs: update to 0.17.0.

6339349

sgn mentioned this pull request Dec 5, 2022

[tracking] List of currently failing builds #39083

Open

sug0 mentioned this pull request Dec 18, 2022

common/environment/build-style/go.sh: allow setting CGO_ENABLED #41167

Merged

github-actions bot added the Stale label Jun 7, 2023

github-actions bot closed this Jun 22, 2023

sgn deleted the go-ipfs branch August 23, 2023 08:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

go-ipfs: update to 0.17.0. #40919

go-ipfs: update to 0.17.0. #40919

sgn commented Dec 5, 2022

sgn commented Dec 5, 2022

CMB commented Dec 5, 2022

sgn commented Dec 6, 2022

CMB commented Dec 6, 2022 via email

sug0 commented Dec 17, 2022 •

edited

Loading

sgn commented Dec 17, 2022

sug0 commented Dec 17, 2022 •

edited

Loading

sug0 commented Dec 17, 2022 •

edited

Loading

sug0 commented Dec 18, 2022 •

edited

Loading

sug0 commented Mar 8, 2023

github-actions bot commented Jun 7, 2023

eoli3n commented Aug 4, 2023 •

edited

Loading

go-ipfs: update to 0.17.0. #40919

go-ipfs: update to 0.17.0. #40919

Conversation

sgn commented Dec 5, 2022

Testing the changes

sgn commented Dec 5, 2022

CMB commented Dec 5, 2022

sgn commented Dec 6, 2022

CMB commented Dec 6, 2022 via email

sug0 commented Dec 17, 2022 • edited Loading

sgn commented Dec 17, 2022

sug0 commented Dec 17, 2022 • edited Loading

sug0 commented Dec 17, 2022 • edited Loading

sug0 commented Dec 18, 2022 • edited Loading

sug0 commented Mar 8, 2023

github-actions bot commented Jun 7, 2023

eoli3n commented Aug 4, 2023 • edited Loading

sug0 commented Dec 17, 2022 •

edited

Loading

sug0 commented Dec 17, 2022 •

edited

Loading

sug0 commented Dec 17, 2022 •

edited

Loading

sug0 commented Dec 18, 2022 •

edited

Loading

eoli3n commented Aug 4, 2023 •

edited

Loading