v0.20.0-rc2: cannot build plugins #9839

hsanjuan · 2023-04-26T12:15:23Z

Checklist

This is a bug report, not a question. Ask questions on discuss.ipfs.io.
I have searched on the issue tracker for my bug.
I am running the latest kubo version or have an issue updating.

Installation method

ipfs-update or dist.ipfs.tech

Version

Kubo version: 0.20.0-rc2
Repo version: 13
System version: amd64/linux
Golang version: go1.19.8

Config

No response

Description

Building plugins that link against Kubo v0.20.0-rc2 is not possible:

asm: xxhash_amd64.s:120: when dynamic linking, R15 is clobbered by a global variable access and is used here: 00092 (/home/hector/go/pkg/mod/github.com/cespare/[email protected]/xxhash_amd64.s:120)	ADDQ	R15, AX
asm: assembly failed

The issue has been fixed in cespare/xxhash#54. However, boxo/gateway pulls xxhash v1, which seems unmaintained and carries this issue.

I don't know why this hasn't shown up before... boxo?

The text was updated successfully, but these errors were encountered:

hsanjuan · 2023-04-26T12:16:54Z

This can be simply reproduced with go build -buildmode=plugin in the cmd/ipfs folder.

hsanjuan · 2023-04-26T12:21:32Z

Unfortunately, anything that imports badger suffers this: dgraph-io/badger#1777

hsanjuan · 2023-04-26T12:34:56Z

So all plugins somehow importing badger at the version we use (i.e. via Kubo or boxo) are going to be broken.

We would need to port the badger fix to the 1.6.2 branch. 🤦

@aschmahmann would it be ok to fork badger and have this fixed ourselves? Badger at 1.6.2 is fully unmaintained...

hsanjuan · 2023-04-26T12:56:19Z

Issue would be fixed by including something like:

replace github.com/dgraph-io/ristretto => github.com/hsanjuan/ristretto v0.0.0-20230426124528-4a4a94a6bb9c

in Kubo, plus switching boxo/gateway to v2.

This is related to ipfs/kubo#9839. boxo still depends on badger, which still requires xxhash/v1, but at least doesn't require it xxhash/v1 itself directly anymore.

aschmahmann · 2023-04-27T02:22:46Z

Out this week so can't take a look at this yet, but this sounds like you're reporting a regression of #8550.

There are sharness tests that test building plugins and make sure they work. IIRC of the possible solutions here, the easiest was to just not build Badger as a plugin but just include the code (https://github.com/ipfs/kubo/pull/8815/files#r832461708)... awkwardly I probably neglected to file an issue about bringing back some external test building with plugins for after the dependency issues linked were resolvable.

Is it possible that some of the tooling you're using to build the plugins (e.g. the bash scripts in the kubo repo) is buggy and has re-added building Badger as an external plugin rather than just included in the codebase?

Side note: @hsanjuan I wonder if #9782 might work for you as an alternative to Go plugins. Easier/harder/similar. If you've got some thoughts and time to comment on that PR it might help motivate changing our documentation to really dissuade users from using Go plugins (as opposed to the other plugin options or using boxo) unless necessary.

hsanjuan · 2023-04-27T07:38:46Z

The issue is that I cannot build a plugin with buildmode=plugin when linking Kubo/Boxo. But I could perfectly do that a while ago while we merged #9750.

I'm not using any Kubo tooling to build the plugin nor doing anything weird. Perhaps the issue depend on how dependencies are resolved and the order is not always the same and refactors have affected this. Or somehow before I didn't transitively require badger and now with Boxo things have moved around and everything comes in.

#9782 is meh... I can always have kubo as submodule and patch things in and get custom binaries. What I would like to have is regular plugins working so they can be dropped in with any official builds.

Anyways fix is simple and safe enough, even if not overly pretty. Without fix then I'm forced to fork.

This is related to ipfs/kubo#9839. boxo still depends on badger, which still requires xxhash/v1, but at least doesn't require it xxhash/v1 itself directly anymore.

lidel · 2023-04-27T14:20:12Z

fwiw I've merged ipfs/boxo#285 and boxo/gateway no longer pulls-in the old xxhash.

As for badger in kubo, badger v1 is unmaintained, and the entire company that maintained newer versions imploded 🙈

Short term, the fix proposed in v0.20.0-rc2: cannot build plugins #9839 (comment) sounds like the only play we have if we want to avoid issues caused by unmaintained dependency?
Long term.. ideas welcome?
- my intuition is to make a plan to deprecate and remove badgerv1 from kubo, and move it to a plugin similar to go-ds-s3

aschmahmann · 2023-04-27T15:54:03Z

@lidel before adding in replace directives or forking I think we should figure out what changed and make sure the sharness tests are working again. We've been operating with this since we added Go1.17 support and IIUC nothing has changed except maybe some dependencies moved around/swapped during the boxo migration. Seems like fixing that will do the job.

Edit: We should confirm when things broke. It's possible the "fix" linked above didn't actually do much of anything for people building plugins which if true would mean nobody has been able to build any external plugins for quite a while which could inform future directions here.

Longer term yeah, we should remove badgerv1 being built-in to kubo by default and if we want badger still included use the latest version.

BigLep · 2023-05-02T16:54:35Z

2023-05-02 maintainer conversation:

Determine when this broke
If this broke last release, we likely screwed up a dependency with Boxo things.
If it's been broken for 1.5 years then it makes clear than no one is using the feature and the feature is effectively dead.

hsanjuan · 2023-05-04T15:13:40Z

It may be that this is broken for a long time. At least go build -buildmode=plugin did not work before in cmd/kubo.

But I still could build the nopfs plugin with github.com/ipfs/kubo v0.18.2-0.20230324125910-9300769122d9 and I don't seem to be able to with v0.20-0-rc2. I will look some more but with all the refactor it's not super easy to tell when things stopped from the plugin (I have to additionally manually align every other dependency in the go.mod).

hsanjuan · 2023-05-04T15:23:41Z

Closed by mistake. So before

github.com/cespare/xxhash v1.1.0 // indirect

Did not need to be in the go.mod. Now requiring v0.20.0-rc2 makes it appear.

There is a good chance that bubbling ipfs/boxo#285 to Kubo fixes my issue since it will make xxhash only reachable via test code (again).

hsanjuan · 2023-05-04T15:27:02Z

go mod why:

Before:

go mod why -m github.com/cespare/xxhash
# github.com/cespare/xxhash
github.com/ipfs-shipyard/nopfs/ipfs/plugin
github.com/ipfs/kubo/core/node
github.com/ipfs/go-ipfs-pinner/dspinner
github.com/ipfs/go-ipfs-pinner/dspinner.test
github.com/ipfs/go-ds-badger
github.com/dgraph-io/badger
github.com/dgraph-io/ristretto/z
github.com/cespare/xxhash

After:

go mod why -m github.com/cespare/xxhash
# github.com/cespare/xxhash
github.com/ipfs-shipyard/nopfs/ipfs/plugin
github.com/ipfs/kubo/core/node
github.com/ipfs/boxo/gateway
github.com/cespare/xxhash

I think it's clearly a refactor that made core/node import gateway and that imports xxhash. So releasing and bubbling boxo should fix it.

aschmahmann · 2023-05-05T17:52:43Z

@hsanjuan thanks for the clarification that the build used to succeed and then started failing in nopfs rather than if kubo was built with buildmode=plugin.

I think with things bubbled up now it should work. I've updated the dependencies in nopfs to work with kubo master in ipfs-shipyard/nopfs#1.

Going to close this issue for now and we're planning to cut v0.20 on Monday so if you get a chance to let us know if something isn't working by then reopen and we'll try and figure it out.

hsanjuan · 2023-05-10T19:24:17Z

It seems v0.20 was cut with the wrong version of boxo (still 0.8.1). So I can't attempt to integrate with the official binaries or release. Needless to say, it's disappointing. Do I have to wait another month now?

hsanjuan · 2023-05-10T19:27:46Z

I understood that v0.20.0 was going to include the fix: #9694 (comment)

aschmahmann · 2023-05-12T15:27:19Z

Needless to say, it's disappointing ... I understood that v0.20.0 was going to include the fix

@hsanjuan sorry I messed this up. Should've been obvious that some commits needed to be cherry picked and a boxo release cut, but I missed it.

Will check with the other maintainers on if getting a patch release out soon is feasible.

BigLep · 2023-05-17T17:20:37Z

Reopening as a signal that we need to talk about the release of this.

BigLep · 2023-05-18T22:35:52Z

@hsanjuan : so as @aschmahmann shared, we messed up here. By default this will go out with the 0.21 release (currently scheduled for 2023-06-08). A Kubo release isn't free on our end, but we will do a 0.20.1 release on end if it impacts you to wait until 0.21. I'd obviously like to avoid doing extra work here, but we will certainly do it if this is impacting you. Please don't hesitate to say if it is. Thanks - our apologies on the delay here.

In terms of how to prevent this from happening again in the future, I have created ipfs/kuboreleaser#6, which I'm hoping we can land for 0.21.

hsanjuan · 2023-05-27T08:10:06Z

Waiting for 0.21 is ok, thanks @BigLep !

BigLep · 2023-06-08T15:40:33Z

Closing since there is a Boxo 0.9 release that has the required items and is in Kubo master. Thanks for your patience!

hsanjuan · 2023-06-13T13:24:46Z

Everything works now, thanks!

hsanjuan added kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization labels Apr 26, 2023

hsanjuan added a commit to ipfs/boxo that referenced this issue Apr 26, 2023

Switch to xxhash/v2

ca834b6

This is related to ipfs/kubo#9839. boxo still depends on badger, which still requires xxhash/v1, but at least doesn't require it xxhash/v1 itself directly anymore.

hsanjuan mentioned this issue Apr 26, 2023

refactor(gateway): switch to xxhash/v2 ipfs/boxo#285

Merged

lidel pushed a commit to ipfs/boxo that referenced this issue Apr 27, 2023

refactor(gateway): switch to xxhash/v2 (#285)

268cad8

This is related to ipfs/kubo#9839. boxo still depends on badger, which still requires xxhash/v1, but at least doesn't require it xxhash/v1 itself directly anymore.

BigLep added this to IPFS Shipyard Team Apr 28, 2023

hsanjuan mentioned this issue May 1, 2023

refactor: remove badger, leveldb dependencies ipfs/boxo#286

Merged

BigLep assigned aschmahmann May 2, 2023

hsanjuan closed this as completed May 4, 2023

github-project-automation bot moved this to 🎉 Done in IPFS Shipyard Team May 4, 2023

hsanjuan reopened this May 4, 2023

BigLep mentioned this issue May 4, 2023

Release 0.20 #9694

Closed

aschmahmann mentioned this issue May 5, 2023

chore: update dependencies and migrate to boxo ipfs-shipyard/nopfs#1

Closed

aschmahmann closed this as completed May 5, 2023

BigLep reopened this May 17, 2023

BigLep moved this from 🎉 Done to 🥞 Todo in IPFS Shipyard Team May 17, 2023

BigLep mentioned this issue May 18, 2023

Add step to log the list of Boxo/Kubo commits in master/main that aren't making it into a release ipfs/kuboreleaser#6

Closed

BigLep mentioned this issue May 23, 2023

Release 0.20.1 #9892

Closed

hsanjuan mentioned this issue May 27, 2023

how to use nopfs? ipfs-shipyard/nopfs#2

Closed

BigLep mentioned this issue May 30, 2023

Release 0.21 #9814

Closed

BigLep closed this as completed Jun 8, 2023

github-project-automation bot moved this from 🥞 Todo to 🎉 Done in IPFS Shipyard Team Jun 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v0.20.0-rc2: cannot build plugins #9839

v0.20.0-rc2: cannot build plugins #9839

hsanjuan commented Apr 26, 2023 •

edited

Loading

hsanjuan commented Apr 26, 2023

hsanjuan commented Apr 26, 2023

hsanjuan commented Apr 26, 2023

hsanjuan commented Apr 26, 2023 •

edited

Loading

aschmahmann commented Apr 27, 2023 •

edited

Loading

hsanjuan commented Apr 27, 2023

lidel commented Apr 27, 2023

aschmahmann commented Apr 27, 2023 •

edited

Loading

BigLep commented May 2, 2023

hsanjuan commented May 4, 2023 •

edited

Loading

hsanjuan commented May 4, 2023 •

edited

Loading

hsanjuan commented May 4, 2023

aschmahmann commented May 5, 2023

hsanjuan commented May 10, 2023

hsanjuan commented May 10, 2023

aschmahmann commented May 12, 2023 •

edited

Loading

BigLep commented May 17, 2023

BigLep commented May 18, 2023

hsanjuan commented May 27, 2023

BigLep commented Jun 8, 2023

hsanjuan commented Jun 13, 2023

v0.20.0-rc2: cannot build plugins #9839

v0.20.0-rc2: cannot build plugins #9839

Comments

hsanjuan commented Apr 26, 2023 • edited Loading

Checklist

Installation method

Version

Config

Description

hsanjuan commented Apr 26, 2023

hsanjuan commented Apr 26, 2023

hsanjuan commented Apr 26, 2023

hsanjuan commented Apr 26, 2023 • edited Loading

aschmahmann commented Apr 27, 2023 • edited Loading

hsanjuan commented Apr 27, 2023

lidel commented Apr 27, 2023

aschmahmann commented Apr 27, 2023 • edited Loading

BigLep commented May 2, 2023

hsanjuan commented May 4, 2023 • edited Loading

hsanjuan commented May 4, 2023 • edited Loading

hsanjuan commented May 4, 2023

aschmahmann commented May 5, 2023

hsanjuan commented May 10, 2023

hsanjuan commented May 10, 2023

aschmahmann commented May 12, 2023 • edited Loading

BigLep commented May 17, 2023

BigLep commented May 18, 2023

hsanjuan commented May 27, 2023

BigLep commented Jun 8, 2023

hsanjuan commented Jun 13, 2023

hsanjuan commented Apr 26, 2023 •

edited

Loading

hsanjuan commented Apr 26, 2023 •

edited

Loading

aschmahmann commented Apr 27, 2023 •

edited

Loading

aschmahmann commented Apr 27, 2023 •

edited

Loading

hsanjuan commented May 4, 2023 •

edited

Loading

hsanjuan commented May 4, 2023 •

edited

Loading

aschmahmann commented May 12, 2023 •

edited

Loading