Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish the docker image used to compile libzim with emscripten #8

Open
kelson42 opened this issue Nov 13, 2022 · 13 comments
Open

Publish the docker image used to compile libzim with emscripten #8

kelson42 opened this issue Nov 13, 2022 · 13 comments
Labels
build Code relating to building or publishing assets enhancement New feature or request

Comments

@kelson42
Copy link

From kiwix-js created by mossroy: kiwix/kiwix-js#866

In some docker registry.
So that it's ready-to-use for developers on kiwix-js, without forcing them to build the image

@kelson42 kelson42 added enhancement New feature or request build Code relating to building or publishing assets labels Nov 13, 2022
@kelson42
Copy link
Author

I guess this ticket should ne now mode to openzim/javascript-libzim?

@kelson42
Copy link
Author

@mossroy Not sure to understand the rationals behind this ticket? Why such a Docker image shoukd be necessary if kiwix/kiwix-build#503 is inplemented?

@kelson42
Copy link
Author

I guess this ticket should ne now mode to openzim/javascript-libzim?

Yes (but I don't seem to be able to do that with the "Transfer issue" link of github)

@kelson42
Copy link
Author

@mossroy I will discuss this with @mgautierfr, but to me the goal is clearly that libzim is directly usable in Javascript.

@kelson42
Copy link
Author

That depends on how far we go on integrating the emscripten build(s) in kiwix-build.
The first step is libzim itself, with kiwix/kiwix-build#503
If we only do this first step, there will still be a second emscripten compilation step to be done on the C "glue" code that allows to call libzim from javascript. In this case, it might be relevant to share one docker image for both compilation steps.

But if we go further and implement kiwix/kiwix-js#768 to do all the emscripten compilation in kiwix-build, I agree with you it might not be necessary to share the docker image

To sum up, I think that depends on how much we integrate build steps on kiwix-build
In all cases, it should not be considered as a blocker: it's always possible to re-build the docker image when it's needed

@Jaifroid
Copy link
Collaborator

As I wrote in #9 (comment), I'd be in favour of full integration in the build stage, so that we publish a nightly WASM together with the JS Web Worker required to run it. This step could be completed in this repo, if #12 is completed. The nightly build workflow should also have a workflow dispatch, so that I can run a "release" version when needed.

@mgautierfr
Copy link
Collaborator

All wrapper (python-libzim, node-libzim, java-libkiwix, ...) are depending on libzim/libkiwix correctly compiled for the target but the wrapper itself is out of kiwix-build.
I think we should do the same thing fo javascript-libzim.
Of course it doesn't means that CI is not publishing a nightly WASM but it would not be to kiwix-build to do it (probably here)

@kelson42
Copy link
Author

@mgautierfr Agree, but that also mean that if we do so, we assume the errors met by @mossroy and @Jaifroid are temprorary and there is a fix to be found for them. Right?

@mgautierfr
Copy link
Collaborator

we assume the errors met by @mossroy and @Jaifroid are temprorary and there is a fix to be found for them. Right?

The first version of libzim for wasm is a first, untested version. I expect potential issues with it and of course, they have to be fixed.
Like any other bugs in libzim or its compilation, we have to fix them. And as for any users of libzim, if there is a bug in their usage of libzim, "they" have to fix it ("they" may include us). There is nothing new here :)

@Jaifroid
Copy link
Collaborator

Jaifroid commented Nov 22, 2022

I'm sure the issues are fixable, it's just that I don't know enough about Linux compilers / makefiles... To summarize, we seem to be running into two fatal issues on this Repo, which are blockers for testing a new WASM:

It's important to solve these so that we don't get stuck on a legacy version of Emscripten, even if an older version of Emscripten
can compile the WASM on a GitHub instance.

@kelson42
Copy link
Author

@mgautierfr We are commited to provide a usable version of libzim.wasm. As long as the prototype of the wrapper fails with CI compiled version, I will have a doubt about that. If I consider the work on the wrapper to be driven by the Kiwix JS people, we should help them to bootstrap the openzim/javascript-libzim repo. Could you please help @Jaifroid on this and secure the libzim.wasm is OK? That way, we would have a healthy base and you could then focus on other topics.

@Jaifroid
Copy link
Collaborator

Jaifroid commented Dec 12, 2022

The Docker container to run Emscripten compiles of the libzim (whether from source or from kiwix-build binaries) is currently built on each push to master or PR. While it's not a very long process to customize the Emscripten Docker, it might make sense to publish it as suggested in this issue. Is this Repo the right place to do that, and on what action would the container be built and published to GHCR? There would be no point building and publishing it on every push/PR here, because in that case we don't save ourselves anything. I'm just a bit unsure on what basis it should be published and how often.

A reference on how to do it is here: https://docs.github.com/en/actions/publishing-packages/publishing-docker-images.

@stale
Copy link

stale bot commented May 26, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

@stale stale bot added the stale label May 26, 2023
@Jaifroid Jaifroid removed the stale label Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Code relating to building or publishing assets enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants