-
Notifications
You must be signed in to change notification settings - Fork 498
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modules stabilization tracking issue #929
Comments
Hi @casey, curious if this project is inspired to any extent by mmake? For me, the most painful thing that traditional make lacks is support for remote includes. For example, we have built out a set of common makefiles at my job for common projects (e.g. ExpressJS/Node apps, Spring Boot apps, AWS Lambda with SAM CLI, etc.). We also have a core makefile that exports handy commands to print messages in certain colors, require certain environment variables, etc. The most annoying part of this setup is that we have to ask people to copy/paste files into their repos. We even have an I just wanted to throw this out there. I know there is a lot of thought being put into a module system in general, but allowing for remote modules would be awesome. Thanks for all the hard work on a really cool tool :) |
I didn't know about
That's a super interesting feature. We don't have modules, but if we ever do, I wouldn't be opposed to adding something like that.
Interesting, that does seem very useful.
Feel free to open an issue about remote modules, why they're cool, how you use them. It's something that won't get implemented for a long time, but having an issue for it would probably be good.
You are most welcome! It is good to be of service 🙌 |
Found this issue also hoping it was talking about remote includes. We currently use make, and rely heavily on a centralized makefile. It's distributed using an include line that uses a shell command to invoke curl:
Works pretty well, but some of the frustrations around |
I would also jump in with support for remote modules. One of the hardest things about working with cloud infrastructure and deploying applications is the need for sharing and reusing. Wanting to reuse all these great just commands I've written in one repository are not easily transferred to another repository. I feel like deno has solved this problem (URL imports, versioned, immutable repos), and aim to write all my future cloud related coordination and scripting. But it doesn't get around not being able to re-use just commands where I really want them. I think this tool really appeals to those who pipeline software. |
I'd love to have something analogous to url imports in dhall for importing modules in Fetching from private repos is probably a more important feature and requires some though in the general case, for github nothing special is required as one can just put the personal access token in the url. Importing from public repo:
Importing from private repo
where |
Thanks a lot for the new module system in v1.19.0 🔥 It'd be amazing to have the following feature (you already envisioned) in subsequent releases:
This is since I'd like to have one Here is the way I achieved this behavior so far. |
@casey Wow, that was quick! Very greateful ❤ |
Thank you @casey for hanging in there on the import and module features! |
Nice, this looks like it could be an elegant replacement for justfile recipes such as
the purpose of which is to run But in testing out replacing this recipe with
This behavior means that justfiles with recipes that do |
Hey @casey, thank you so much for just and for this update, quick question, do you plan, or maybe it's already possible and i've missed it to include modules from git repos, like @stereobutter initially mentioned, i would love that. |
@WladyX Do you mean including modules from a local git repo, or a remote git repo? |
Yes, exactly. |
@WladyX Sorry, to be clear, do you want to load modules from a file in a local git repo, e.g., one that's on the same computer that just is running on, or a remote repo, like one on GitHub? |
no worries Casey!
ideally i want to be able to make templates and by using variables to adapt them to each repo. does that make sense? |
I created #1799 to discuss remote imports. |
I'm finding it to be off putting that the functions Edit: I missed #929 (comment) already talking about this but just wanted to echo that is it causing me some grief as well. |
From #1202 (comment):
Would you consider #1866 to be a bug related to modules? |
@crdx It's definitely a bug related to modules, although it can be fixed in a backwards-compatible way, so it isn't a stabilization blocker. |
On my end, while impressed with the subcommand features implemented with modules and submodules, I am not really satisfied with the listing of the subcommands in modules and submodules. I find it too verbose and too nested. What I would expect, as it fits my taste, would be short lines for commands, and not showing the subcommands. So, I'd suggest, as an additional feature, a directive [no-unfold] or [fold-listing] (perhaps with a max nested level) that would only give the command giving access to subcommands, but not the nested subcommands themselves. This way, I have the freedom to implement a help command in toplevel that would allow to invoke the listing of subcommands of a command and only that. Like Working in small tmuxed fractions of screen over ssh, having a long, nested, list of commands, subcommands of commands, subcommands of subcommands of commands... not that good an idea. |
I concur @gl-yziquel — here's a side-by-side comparison of the old behavior and one of the old-style way we have done since 2019 #208 (comment) and have since made way bigger: And now I can't list the The way we've made our just scripts discoverable and how we've taught that here is that we've told developers to first run
Again, I don't know if this behavior is reproducible with modules or not. But this is a crucial aspect for us to keep the scripts both maintainable and discoverable at the same time.'' The way we've made e.g. I'd go as far as suggest the |
@BCNelson There are a bunch of new functions for getting paths. |
@stuartellis and @foresterre Referring to parent and submodule recipes in aliases and dependencies is planned, but not for initial stabilization, since it can be added in a backwards-compatible way. |
@gl-yziquel and @valscion Thanks so much for the feedback! I'm not a heavy user of modules (in fact, I don't think I've ever actually used one in my own To sum up:
|
I opened #2107 to track improving |
Something else I noticed, is that using the interactive chooser, e.g. I figured, if supplied with the module name like |
@foresterre I wasn't actually able to reproduce this, but I opened #2110 to track it. If you could provide a repro there that would be great. |
I just merged #2107, so As far as I know, there are no current blockers to stabilization. There are a lot of features that I want to add later, but they can be added in a backwards compatible way, so they don't block stabilization. My current plan is to stabilize modules on July 1st, so testing and feedback are much appreciated! |
TL:DR;
I've tested out the different ways we use submodules and couldn't come up with an example where the submodules wouldn't have worked. I really like how the end result in #2113 felt like! We do have to do a few changes to how we have our modules created but the changes are quite straightforward — the biggest one being that we have to change how our default For some cases where we've called to recipes defined in other submodules, I think we'll use Here's a few of the changes we'd need to do, with one invocation of git diff details
diff --git a/Justfile b/Justfile
--- a/Justfile
+++ b/Justfile
@@ -7,34 +7,21 @@ help:
-admin +args='help':
- #!/usr/bin/env bash
- just --working-directory `pwd` --justfile tasks/admin/Justfile $(cat <<EOF;
- {{args}}
- EOF)
+mod admin 'tasks/admin/Justfile'
-assets +args='help':
- @just --working-directory `pwd` --justfile tasks/assets/Justfile {{ args }}
+mod assets 'tasks/assets/Justfile'
-aws +args='help':
- @just --working-directory `pwd` --justfile tasks/aws/Justfile {{ args }}
+mod aws 'tasks/aws/Justfile'
-clean +args='help':
- @just --working-directory `pwd` --justfile tasks/clean/Justfile {{ args }}
+mod clean 'tasks/clean/Justfile'
-csv +args='help':
- @just --working-directory `pwd` --justfile tasks/csv/Justfile {{ args }}
+mod csv 'tasks/csv/Justfile'
-deps +args='help':
- @just --working-directory `pwd` --justfile tasks/deps/Justfile {{ args }}
+mod deps 'tasks/deps/Justfile'
-devices +args='help':
- @just --working-directory `pwd` --justfile tasks/devices/Justfile {{ args }}
+mod devices 'tasks/devices/Justfile'
-docker +args='help':
- @just --working-directory `pwd` --justfile tasks/docker/Justfile {{ args }}
+mod docker 'tasks/docker/Justfile'
-docs +args='help':
- @just --working-directory `pwd` --justfile tasks/docs/Justfile {{ args }}
+mod docs 'tasks/docs/Justfile'
-elasticsearch +args='help':
- @just --working-directory `pwd` --justfile tasks/elasticsearch/Justfile {{ args }}
+mod elasticsearch 'tasks/elasticsearch/Justfile'
@@ -44,16 +31,11 @@ elasticsearch +args='help':
-mixpanel +args='help':
- @just --working-directory `pwd` --justfile tasks/mixpanel/Justfile {{ args }}
+mod mixpanel 'tasks/mixpanel/Justfile'
-start +args='help':
- @just --working-directory `pwd` --justfile tasks/start/Justfile {{ args }}
+mod start 'tasks/start/Justfile'
-stripo +args='help':
- @just --working-directory `pwd` --justfile tasks/stripo/Justfile {{ args }}
+mod stripo 'tasks/stripo/Justfile'
-tests +args='help':
- @just --working-directory `pwd` --justfile tasks/tests/Justfile {{ args }}
+mod tests 'tasks/tests/Justfile'
-translations +args='help':
- @just --working-directory `pwd` --justfile tasks/translations/Justfile {{ args }}
+mod translations 'tasks/translations/Justfile'
diff --git a/tasks/admin/Justfile b/tasks/admin/Justfile
--- a/tasks/admin/Justfile
+++ b/tasks/admin/Justfile
@@ -8,3 +8,3 @@ set dotenv-load := true
help:
- @just admin -l
+ @just -l
diff --git a/tasks/assets/Justfile b/tasks/assets/Justfile
--- a/tasks/assets/Justfile
+++ b/tasks/assets/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just assets -l
+ @just -l
diff --git a/tasks/aws/Justfile b/tasks/aws/Justfile
--- a/tasks/aws/Justfile
+++ b/tasks/aws/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just aws -l
+ @just -l
diff --git a/tasks/clean/Justfile b/tasks/clean/Justfile
--- a/tasks/clean/Justfile
+++ b/tasks/clean/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just clean -l
+ @just -l
diff --git a/tasks/csv/Justfile b/tasks/csv/Justfile
--- a/tasks/csv/Justfile
+++ b/tasks/csv/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just csv -l
+ @just -l
diff --git a/tasks/deps/Justfile b/tasks/deps/Justfile
--- a/tasks/deps/Justfile
+++ b/tasks/deps/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just deps -l
+ @just -l
diff --git a/tasks/devices/Justfile b/tasks/devices/Justfile
--- a/tasks/devices/Justfile
+++ b/tasks/devices/Justfile
@@ -5,14 +5,6 @@ set dotenv-load := true
help:
- @just devices -l
+ @just -l
-# Example: just devices android start
-android +args='help':
- @just --working-directory `pwd` --justfile tasks/devices/android/Justfile {{ args }}
-
-# Example: just devices ie11 start
-ie11 +args='help':
- @just --working-directory `pwd` --justfile tasks/devices/ie11/Justfile {{ args }}
-
-# Example: just devices ios start
-ios +args='help':
- @just --working-directory `pwd` --justfile tasks/devices/ios/Justfile {{ args }}
+mod android './android/Justfile'
+mod ie11 './ie11/Justfile'
+mod ios './ios/Justfile'
diff --git a/tasks/devices/android/Justfile b/tasks/devices/android/Justfile
--- a/tasks/devices/android/Justfile
+++ b/tasks/devices/android/Justfile
@@ -4,3 +4,3 @@ set dotenv-load := false
help:
- @just devices android -l
+ @just -l
@@ -66,3 +66,3 @@ list:
export ANDROID_AVD_HOME={{justfile_directory()}}/avd
- just deps require-brew-program avdmanager android-commandlinetools
+ just --justfile ../../../Justfile deps require-brew-program avdmanager android-commandlinetools
avdmanager list avd
diff --git a/tasks/devices/ie11/Justfile b/tasks/devices/ie11/Justfile
--- a/tasks/devices/ie11/Justfile
+++ b/tasks/devices/ie11/Justfile
@@ -11,3 +11,3 @@ ie11_win7_virtualbox_dir := 'tasks/devices/virtualbox/ie11_win7'
help:
- @just devices ie11 -l
+ @just -l
diff --git a/tasks/devices/ios/Justfile b/tasks/devices/ios/Justfile
--- a/tasks/devices/ios/Justfile
+++ b/tasks/devices/ios/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just devices ios -l
+ @just -l
diff --git a/tasks/docker/Justfile b/tasks/docker/Justfile
--- a/tasks/docker/Justfile
+++ b/tasks/docker/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just docker -l
+ @just -l
diff --git a/tasks/docs/Justfile b/tasks/docs/Justfile
--- a/tasks/docs/Justfile
+++ b/tasks/docs/Justfile
@@ -4,3 +4,3 @@ set dotenv-load := false
help:
- @just docs -l
+ @just -l
diff --git a/tasks/elasticsearch/Justfile b/tasks/elasticsearch/Justfile
--- a/tasks/elasticsearch/Justfile
+++ b/tasks/elasticsearch/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just elasticsearch -l
+ @just -l
diff --git a/tasks/mixpanel/Justfile b/tasks/mixpanel/Justfile
--- a/tasks/mixpanel/Justfile
+++ b/tasks/mixpanel/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just mixpanel -l
+ @just -l
diff --git a/tasks/start/Justfile b/tasks/start/Justfile
--- a/tasks/start/Justfile
+++ b/tasks/start/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just start -l
+ @just -l
diff --git a/tasks/stripo/Justfile b/tasks/stripo/Justfile
--- a/tasks/stripo/Justfile
+++ b/tasks/stripo/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just stripo -l
+ @just -l
diff --git a/tasks/tests/Justfile b/tasks/tests/Justfile
--- a/tasks/tests/Justfile
+++ b/tasks/tests/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just tests -l
+ @just -l
diff --git a/tasks/translations/Justfile b/tasks/translations/Justfile
--- a/tasks/translations/Justfile
+++ b/tasks/translations/Justfile
@@ -5,3 +5,3 @@ set dotenv-load := true
help:
- @just translations -l
+ @just -l There will most likely be some cases where these "calls to submodules" will need to be fixed later and ones we'll forget to update when going all-in to Just modules. Most of the calls should still be for One thing that I found was confusing is the error when a submodule is calling out to a recipe that is missing. For example: $ just admin fix-venue-location
error: Justfile does not contain recipe `admin`.
error: Recipe `fix-venue-location` failed with exit code 1 And I had these: # <root>/Justfile
mod admin 'tasks/admin/Justfile'
# <root>/tasks/admin/Justfile
fix-venue-location env='development' +ARGS='--usage':
#!/usr/bin/env bash
just admin _run-ruby-script-using-confirmation {{ env }} fix_venue_location $(cat <<EOF;
{{ ARGS }}
EOF) It was also nice that I could still do Current file structure we have without modules looked like this for that case:
If I had an error with the old way of calling, for example if I had $ just admin fix-venue-location
error: Justfile does not contain recipe `not-correct-admin`.
error: Recipe `fix-venue-location` failed with exit code 1
error: Recipe `admin` failed with exit code 1 So the error message did refer to "Recipe `admin` failed with exit code 1" that at least somehow pointed to the problem being inside the admin "submodule". |
Maybe related #2215 |
@valscion Thanks for the detailed write up! I'm really glad to hear that modules are mostly working for your use-case, especially since it was your comment way back in 2019 which got me thinking about modules in the first place.
I added both of these to the list of future changes to make in the top comment, since both of them can be implemented in a backwards compatible fashion. When modules are stabilized and this issue is closed, I'll make dedicated issues for all of them. |
Is there any reason why mod 'dir1/dir2/dir3' './dir1/dir2/dir3/justfile'
build:
dir1/dir2/dir3::build |
Module names are identifiers, which are of the form |
@gl-yziquel. Can you do |
I'll keep that in mind... |
One thing I came across is importing a module with a circular dependency, where examples uses main, and main uses examples. My solution is to split this module into two files, for example:
build: deps
...
deps:
...
import './build.just'
mod examples './examples/justfile'
test: build
just examples::run
mod root '../build.just'
deps:
just root::build
run: deps
... |
Trying to figure out how to reexport var from other module I wonder if is X: export INTERNAL_VAR := "..."
INTERNAL_VAR:
echo "{{INTERNAL_VAR}}" Y: mod X '../X/justfile'
export SOME_ENV := `just --justfile ../X/justfile INTERNAL_VAR` Attribute could be a solution here to not repeat recipe hmm [echo]
export INTERNAL_VAR := "..." EDIT: This wont work well with circular dependencies as |
As of There aren't any major changes to modules 1.31, the only minor one is that a mod statement with a path may now point to a directory:
In which case There are still a lot of things missing, but they can be added in a backwards compatible fashion. I'm going to create issues for all the items in the top post here, and make a module improvements tracking issue. Thanks to everyone for bearing with me, thanks to the contributors and commenters, and thanks to @neunenak who did the initial work with |
The module stabilization tracking issue is dead, long live the module improvement tracking issue! |
This issue tracks stabilization of
mod
statements.There are a bunch of features which aren't yet implemented, but I think they can all be implemented in a backwards compatible fashion:
.env
filesjust --show
work with modulesI want the feature to be well used and well tested before stabilizing it.
module_file()
andmodule_directory()
functions #2105--list
output with modules #2107The text was updated successfully, but these errors were encountered: