Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saving custom scripts in pyproject.toml (alias npm run) #2496

Open
2 tasks done
bersace opened this issue Jun 5, 2020 · 10 comments · May be fixed by #7896
Open
2 tasks done

Saving custom scripts in pyproject.toml (alias npm run) #2496

bersace opened this issue Jun 5, 2020 · 10 comments · May be fixed by #7896
Labels
area/plugin-api Related to plugins/plugin API kind/feature Feature requests/implementations

Comments

@bersace
Copy link

bersace commented Jun 5, 2020

  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have searched the documentation and believe that my question is not covered.

Feature Request

What do you think of having a scripts described in pyproject.toml ? Something like Procfile or script section in packages.json.

Here is a suggestion of configuration & usage:

[tools.poetry.proc]
build = "python manage.py collectstatic"
test = "pytest tests/unit/"
serve = "flask run"
$ poetry proc build
...
$ poetry proc serve
...

I think it would help project to not require Makefile nor tox to share commands and in the same time, ensure commands are properly executed in selected environment.

What do you think of this ?

@bersace bersace added kind/feature Feature requests/implementations status/triage This issue needs to be triaged labels Jun 5, 2020
@SanketDG
Copy link
Contributor

SanketDG commented Jun 5, 2020

See #2310 for a broader spec.

@bersace
Copy link
Author

bersace commented Jun 5, 2020

@SanketDG I think it's something else.

  • scripts is about shipping CLI entrypoints to final users. Think /usr/bin/command
  • proc is about sharing recipes or command alias to developers. Think tox -e unit

@finswimmer finswimmer removed the status/triage This issue needs to be triaged label Jun 6, 2020
@abn
Copy link
Member

abn commented Jun 10, 2020

This has been discussed in #591 which was closed with #591 (comment).

As was stated there, this might be a candidate feature for a plugin once we have that baked into poetry >=v1.1.0.

@bersace
Copy link
Author

bersace commented Jun 10, 2020

@abn yes, that's exactly the subject of this PR :-) A plugin is a good fit for this feature.

@abn abn added area/plugin-api Related to plugins/plugin API and removed kind/feature Feature requests/implementations labels Jun 11, 2020
@nat-n
Copy link
Contributor

nat-n commented Sep 2, 2020

@bersace in the mean time I'd suggest checking out a project I'm working on that solves this problem in a slightly different way

Poe the Poet a task runner that works well with poetry.

It provides a flexible system for defining tasks in the pyroject.toml and executing them with a dedicated executable. Basic usage looks like:

Example configuration with different task types:

[tool.poe.tasks]
test       = "pytest --cov=poethepoet"                                # simple command based task
mksandwich = { script = "my_package.sandwich:build" }                 # python script based task
tunnel     = { shell = "ssh -N -L 0.0.0.0:8080:$PROD:8080 $PROD &" }  # shell script based task

Example invocation:

# extra argument is passed to the underlying pytest command
poe test -v   

# If poethepoet is added as a dev-dependency
poetry run poe tunnel

And it supports zsh command completion!

EDIT: as of v0.12.0 it can also be used as a poetry plugin

@stranger-danger-zamu
Copy link

Leaving my 2 cents here on the whole "scripts in pyproject.toml" conversation. I understand that this will probably not going to be implemented or accepted, but at least I added my 2 cents.


This is a clearly desired feature and has had well supported PRs demonstrating community support for it going back a few years.

While I do get the maintainers wanting this sort of functionality outside of poetry's core codebase and scope, people are asking for it because they see poetry as Python's primary DX/DI. The birds eye view of Python's developer experience at the moment is that pyproject.toml will be the future of Python and poetry is in the position as the developer interface for pyproject.toml projects because it's the most full featured tool that uses it that also builds.

Taking a step back, I personally just want a tool that I allows me to have a known procedure for all (modern) python projects.

Rust has this with cargo where the requisite to get up and running to contribute to a project is cloning the repo, having rust and cargo, making some changes, and running cargo build.

Go has something similar (if not a little messy due to older codebases possibly not supporting modules). That said it's simple in those languages since at most, they statically compile their artifacts and are done in terms of bundling their dependencies with their artifact.

Meanwhile in Python-land, we might build a wheel with explicit python dependencies tied to hashes. But that might not bundle all the true dependencies (eg. system libraries).

So, we have to reach out for other artifact formats which do allow us to encapsulate all our dependencies (eg. Docker, deb, rpm, conda, OS golden images, etc). This is a bigger problem with Python packaging that is definitely outside of poetry's scope.

There are plenty of tools which can easily encode the procedures to bundle python wheels into a new artifact, but nine times out of ten they are excessive or they make poetry redundant and/or useless.

  • Make is amazing. It allows developers to define complex graphs of dependencies and regenerate files based on modified time. However, Python projects aren't hugely complicated task graphs, they tend to just be linear dependencies making most of make's feature set more of a liability than a boon. For most python projects, you can just do the same thing with shell scripts with less cognitive overhead.
  • Python task runners are great and many of them have wonderful features. But they are another thing you have to install and learn. They also can be replaced with a shell script with less cognitive overhead.
  • Poetry Plugins share the same problem as python task runners. You have to install them separately and learn them separately. Even knowing that it is straight forward (not really since there is no official 1.2 release), many people can't reliably use this. These plugins can be made by anyone and, being honest, I doubt anyone is going to look at that codebase to check that it's not a security problem. I'm much more confident that poetry would be looked at instead of a random plugin by some person. You have to be cynical about security.
  • So what about those shell scripts? Well for one you can't assume that they are cross-platform (thanks Windows!), so you either have to have more than one script for the appropriate platforms (making maintainability a bit of a chore) or add system dependencies for developing but that makes it intractable for some users (yay, enterprise security policies...).
  • Technically, we have Python, so python scripts? This does work, however it can be a bit hairy to do some stuff (namely piping commands so for beginners it might be a bit much) and there is no spec or standard so get ready for arbitrary script names that will be stored arbitrarily (of course, people will not be insane and scripts will probably house everything, but people will always exceed your expectations). However, pyproject.toml seems to be swallowing all extra files that aren't your actual code which is great but then that means any extra script (python or otherwise) is swimming upstream.

To be honest, what I don't want is having to write documentation on how to set up a dev environment including on installing N plugins before they clone down the repository. Writing python scripts to run "tasks" might work but it can quickly become a soup of arbitrary code (and probably bad security practices). And if I am going through all that effort, I might as well just use setuptools or flit if poetry doesn't really have differentiating utility built-in from them.

It just feels a bit odd. If poetry is a package manager, then it shouldn't also build packages. You don't use APT or DNF to build deb or rpm files.

@Dilski
Copy link

Dilski commented Oct 3, 2022

This feels like a good candidate for a poetry plugin?

@neersighted
Copy link
Member

https://github.com/nat-n/poethepoet and https://github.com/illBeRoy/taskipy both exist to do this, and Poe has a Poetry plugin (taskipy will likely gain one)

@Dilski
Copy link

Dilski commented Oct 4, 2022

Oh perfect, thank you!

@brunolnetto
Copy link

brunolnetto commented Dec 27, 2022

I just wrote an issue on the subject, but you are ahead of me. Linguistics call these words aliases. You must be familiar with it yourselves on Git environment and setup file .git/config.

If it is not already shipped, I suggest the following syntax:

$ poetry alias "hocus-pocus" "echo Abracadabra" 
$ poetry hocus-pocus
Abracadabra

Main drawback: poetry alias introduces on the very most some code management "fear" of "malicious command". It evokes the question: who would write such a malicious command against itself? Nevertheless, a conscious peer-review vanishes such intents away.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/plugin-api Related to plugins/plugin API kind/feature Feature requests/implementations
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants