[feature request] Installation via pip?

[rachtsingh]

This is a bit of an odd request, but I work in some environments where installing external dependences is difficult (e.g. it would complicate the dev setup process) except that installation of Python packages is considered acceptable.

Would you consider a PR that (a) creates a Python package that just bundles just (via maturin) as a CLI script, and (b) a Github actions workflow that cuts a new PyPI release whenever just is updated?

That way, you can install just via pip install just

(though in practice we might have to choose a different name, just is unfortunately taken on PyPI)

No worries if out of scope - I would just maintain my own such wrapper if that's the case but I thought it would appropriate to ask.

[casey]

Interesting idea! I think it depends on how weird and complicated the wrapper and CI task are. If they're really simple, then I guess why not. My main worry wouldn't be the package itself, but instead keeping the package up to date. I've personally been horrific at keeping just packages up to date, so I've given up on maintaining them myself, and wouldn't like to publish a Python package just for it to become out of date quickly.

[rachtsingh]

Ok, I think it's as simple as adding the following to a pyproject.toml in the root directory:

[build-system]
requires = ["maturin>=1.0,<2.0"]
build-backend = "maturin"

[project]
name = "just"
version = "1.34.0"
description = "🤖 Just a command runner"
authors = [{ name = "Casey Rodarmor", email = "[email protected]" }]
readme = "README.md"
requires-python = ">=3.7"
license = { file = "LICENSE" }
keywords = ["command-line", "task", "runner", "development", "utility"]
classifiers = [
    # TODO
]

[project.urls]
Repository = "https://github.com/casey/just"
Documentation = "https://github.com/casey/just"
Changelog = "https://github.com/casey/just/blob/master/CHANGELOG.md"

[tool.maturin]
bindings = "bin"
manifest-path = "Cargo.toml"
module-name = "just"
strip = true

Here's what I did (I'm on WSL so I have to specify --target):

~ ❯ cd ~/proj/just/
~ ❯ maturin build --target x86_64-unknown-linux-gnu --release
...
    Finished release [optimized] target(s) in 40.05s
📦 Built wheel to /home/singhrac/proj/just/target/wheels/just-1.34.0-py3-none-manylinux_2_34_x86_64.whl
...
~ ❯ which just
just not found
~ ❯ pip install /home/singhrac/proj/just/target/wheels/just-1.34.0-py3-none-manylinux_2_34_x86_64.whl
Processing ./proj/just/target/wheels/just-1.34.0-py3-none-manylinux_2_34_x86_64.whl
Installing collected packages: just
Successfully installed just-1.34.0
~ ❯ which just
/home/singhrac/.pyenv/shims/just
~ ❯ just --help
🤖 Just a command runner - https://github.com/casey/just

Usage: just [OPTIONS] [ARGUMENTS] ... 
...

At the end of the day a .whl file is just a zip file with the scripts/just binary and some random metadata (license, abi tag, how it was built, etc), and maturin handles that for us.

maturin also generates a Github Actions yaml if you ask for it (maturin generate-ci github > .github/workflows/python.yaml) but to finish that process you need to create a PyPI token and set the appropriate secret env in Github, I think. I think there's some choices (e.g. platforms, whether to cut a release on a new tag or just push to main, etc.) that are probably up to you there, but happy to make the PR if you'd like.

[casey]

The version number in pyproject.toml will need to be manually updated each release, which I'd like to avoid. Is it possible to auto-generate the .toml file for publishing?

Won't the python package be architecture specific? I.e., it'll contain an x86 binary, and will not run on a non-x86 machine?

Would a better approach instead be to create a python package which never changes, which downloads an appropriate binary similar to https://github.com/casey/just/blob/master/www/install.sh?

[rachtsingh]

Re: automatically generating the .toml: definitely possible, in that you can make a script to generate a pyproject.toml, but it turns out that maturin will take metadata fields by default from Cargo.toml, so you can just omit the lines you don't want to override (i.e. version, description, etc.).

Re: architecture-specific: it will, but in practice wheels are tagged by their specific variant (i.e. none-manylinux_2_34_x86_64 above), and if there isn't a wheel uploaded for the platform of the user (e.g. you built x86 wheels but not aarch64 ones), I think pip install just will set up a build environment and compile for the user. The default compiles for a bunch of different platforms, but you can customize that when generating the YAML (or after).

I think it comes down to whether you think using the Github actions build time is worth it to precompile for (any) platforms. Personally I think for maintenance burden it would be better to have "no wheels, compile it yourself" over "customized script that downloads the right binary". For example, you'd need to make sure "pip uninstall just" also works, etc., which is much simpler with just the default maturin setup.

[casey]

Got it, thanks for the explanation! That all sounds good, want to submit a PR?

[rachtsingh]

Done, thanks for the consideration.

[casey]

See comments in #2361, but I'm leaning towards not adding this. I fear it will get out of date or the build will break, and I'm honestly pretty unlikely to test it, so I would suggest maintaining this out-of-tree, and I can link to it in the readme. For the package name, I would suggest rust-just, but justcli is fine too.