Support prebuildify

[justinkambic]

What are you trying to achieve?

Hi - I'm a Kibana contributor; we are looking to achieve some light image processing in one of the native plugins for the project. We are already using Sharp in one of our agents, and we'd like to use it on our server side as well.

One of the obstacles for us doing this is ensuring that a native module like Sharp could integrate into the Kibana ecosystem. Because we use Bazel caching, we need to pre-bundle all bins. To achieve this, we use modules that are build using prebuildify. Would Sharp be open to a change/addition like this?

I realize this impacts the way that Sharp distributes binaries, so this is obviously a serious undertaking. We would be willing to contribute changes to help make this possible.

Have you searched for similar feature requests?

I searched the Issue backlog for prebuildify and found nothing.

What would you expect the API to look like?

Sharp would produce the same binaries it does today but would bundle them all in the npm package using prebuildify.

What alternatives have you considered?

In the event that Sharp's maintainers are not interested in pursuing prebuildify, we have discussed forking the repo and maintaining a pre-packaged version ourselves.

Our non-Sharp alternative would be shipping image blobs to the client and performing our processing using the Canvas API.

Is there a sample image that helps explain?

N/A

[lovell]

Hi Justin, thanks for all your work on Kibana, it's a great tool.

The approach taken by prebuildify is perfect for relatively small modules and I've been meaning to start using it with some other native modules I maintain.

However in the case of sharp, we've got to consider the prebuilt binaries for libvips and its dependencies, the compressed tarballs for which currently weigh in at ~7.5MB per platform-arch combo.

https://github.com/lovell/sharp-libvips/releases/tag/v8.10.6

We also provide (cross-compiled) prebuilt libvips binaries for more ARM platforms than those covered by prebuilt sharp binaries, namely linux-armv6, linux-armv7 and darwin-arm64v8 (M1). These result in sharp being compiled from source, which is relatively fast, without the whole of libvips and friends also having to be compiled.

I don't have much experience with Bazel - are there any existing examples of a build process that includes prebuilt binaries for multiple platforms other than prebuildify (perhaps something from the golang ecosystem)?

At npm install time, sharp can be told to look on the filesystem for prebuilt binaries, which might be more suited to a deterministic/caching build system.

On a related note, I'm also aware that yarn and npm are currently considering how to improve package manager support for native modules as an alternative to the prebuild/node-pre-gyp approach, which is very welcome - see yarnpkg/berry#2751 (comment)

[justinkambic]

Hi @lovell thanks so much for your rapid response to the issue. For the time being we are going to perform our processing on the client because it's the strategy that will allow us to ship our feature the soonest, but we'll continue exploring how we can integrate a native lib like Sharp into the server process as it greatly improves performance and would serve to eliminate a lot of complex code.

We will be using Sharp in our Synthetics agent in an upcoming release though; thank you for the great library!