Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modules and subcommands #383

Closed
casey opened this issue Jan 16, 2019 · 60 comments · Fixed by #1095 or #1782
Closed

Modules and subcommands #383

casey opened this issue Jan 16, 2019 · 60 comments · Fixed by #1095 or #1782

Comments

@casey
Copy link
Owner

casey commented Jan 16, 2019

This is apropos of @valscion's comment in #208, where he talks about some of the annoyances of using just to emulate subcommands. This is also somewhat related to #237 and #367, where sub-justfiles are discussed.

I'd like to come up with something that enables all of these things. Some ideas:

A statement mod foo parses foo.just or foo/mod.just as a submodule. Recipes in foo.just / foo/mod.just can now be called on the command line with just foo RECIPE.

If the submodule source file is foo.just, the current working directory for recipes in that submodule will be the current directory. If the it is in foo/mod.just, the current working directory will be foo/.

Questions:

  • What do we do about directories with weird characters? I think punting is reasonable. But eventually we might want to support something like mod "path with spaces/bar". Any solution should be based on user feedback.

  • How do we handle dependencies between recipes and variables in different submodules?

  • Can you have a submodule in a submodule, and call those commands?


Notes for a first path for out-of-line modules only:

  • make a module loading step after parsing
  • look at parsed module and load all other modules, recursively
  • module map has all modules in it
  • pass module map to the analyzer
  • analyzer produces justfile

  • will need to modify error messages to include file path
  • will need to modify argument parser to successfully call submodules
  • will need to disallow recipes and modules to have same name
@valscion
Copy link

Thanks for opening this more focused issue ☺️

Currently for us, it would be enough to support one level deep submodules. But of course, it would be nice if the solution we could come up with would work for deeper submodules, too.

I'm not sure if just needs to automatically support directory traversal. To me it sounds reasonable to first keep on supporting a single Justfile, and that you could add special statements there to tell just about the existence of submodules. Like so:

# <root>/Justfile
mod foo

# <root>/foo/mod.just
test:
  echo "All OK"

And then you could call <root>/foo.just#test by running just foo test.

This way just wouldn't have to glob the filesystem to find .just files to figure out what submodules exist. That would also free just to use directory syntax for other purposes, e.g. sub-submodules. I could imagine one could run just foo/bar RECIPE which would try to something like this:

# <root>/Justfile
mod foo

# <root>/foo/mod.just
mod bar

# <root>/foo/bar/mod.just
RECIPE:
  echo "All OK"

I'm not excited about naming sub-modules with .just extension and having to either put them to <submodule>/mod.just or <submodule>.just file. If possible, I'd like to be able to control where those submodules are located. With something like this perhaps?

# <root>/Justfile
mod foo "tasks/foo/Justfile"

# <root>/tasks/foo/Justfile
RECIPE:
  echo "All OK"

which could be called like so:

just foo RECIPE

One could continue with the same way to enable sub-submodules:

# <root>/Justfile
mod foo "tasks/foo/Justfile"

# <root>/tasks/foo/Justfile
mod bar "bar/Justfile"

# <root>/tasks/foo/bar/Justfile
RECIPE:
  echo "All OK"

and then that could be called with something like this:

just foo/bar RECIPE

It could be that overloading the directory syntax is not wise, as just already has functionality for that:

# <root>/foo/Justfile
RECIPE:
  echo "All OK"
$ just foo/RECIPE

The syntax isn't the main point for sub-submodules. One possibility would be to denote sub-submodules with e.g. ::, making one of my previous examples callable like so:

just foo::bar RECIPE

I'd be fine with that kind of solution, too.


What do you think? What kind of discussion would you want to have here? I'd be happy to continue the conversation in any way you'd want to ☺️

@casey
Copy link
Owner Author

casey commented Feb 18, 2019

Thanks for opening this more focused issue ☺️

I'm just going respond like I didn't let your reply in my inbox for a month >_< I beg your forgiveness!

Currently for us, it would be enough to support one level deep submodules. But of course, it would be nice if the solution we could come up with would work for deeper submodules, too.

Agreed, it would be nice to do something that works with deeper submodules. I've heard tell that some companies use just, and it would be nice if I could infect their giant monorepos with justfiles.

I'm not sure if just needs to automatically support directory traversal. To me it sounds reasonable to first keep on supporting a single Justfile, and that you could add special statements there to tell just about the existence of submodules. Like so:

# <root>/Justfile
mod foo

# <root>/foo/mod.just
test:
  echo "All OK"

And then you could call <root>/foo.just#test by running just foo test.

This sounds good.

This way just wouldn't have to glob the filesystem to find .just files to figure out what submodules exist.

Yeah, I definitely want to avoid globbing.

That would also free just to use directory syntax for other purposes, e.g. sub-submodules. I could imagine one could run just foo/bar RECIPE which would try to something like this:

# <root>/Justfile
mod foo

# <root>/foo/mod.just
mod bar

# <root>/foo/bar/mod.just
RECIPE:
  echo "All OK"

We should come up with something that lets just foo bar RECIPE work in that case, since as you note / is already interpreted on the command line.

I'm not excited about naming sub-modules with .just extension and having to either put them to <submodule>/mod.just or <submodule>.just file. If possible, I'd like to be able to control where those submodules are located. With something like this perhaps?

# <root>/Justfile
mod foo "tasks/foo/Justfile"

# <root>/tasks/foo/Justfile
RECIPE:
  echo "All OK"

which could be called like so:

just foo RECIPE

I think using a string as in mod name "path" sound good, if you want just to look in a non-default location.

I was thinking that given mod NAME just should look in NAME.just or NAME/mod.just or NAME/Justfile, but I'm not sure if that's weird. What do you think?

What do you think? What kind of discussion would you want to have here? I'd be happy to continue the conversation in any way you'd want to ☺️

Here is great! I promise I'll respond faster next time 😅

@casey
Copy link
Owner Author

casey commented Feb 18, 2019

One thing that needs to get resolved:

Can a recipe in one module have dependencies on a recipe in another module? And if so, how do they refer to recipes in other modules?

Is this something that you need or would use? Also if there are others out there who would make use of modules, I'd definitely be interested in your thoughts.

@valscion
Copy link

I'm just going respond like I didn't let your reply in my inbox for a month >_< I beg your forgiveness!

I promise I'll respond faster next time 😅

Hey, it's completely OK to take your time to reply, or to not even reply at all ☺️. Nobody's getting paid on working on this, so any comment at all is only a value add, no matter how long it takes to respond. 💞

I was thinking that given mod NAME just should look in NAME.just or NAME/mod.just or NAME/Justfile, but I'm not sure if that's weird. What do you think?

Hmm yeah, we can make the location optional for sure — but I'm not sure if we know yet what would be a good choice from all of these, so I'm a bit weary on making the decision already in the first implementation.

I kinda like the name Justfile to describe files using this project. I'm a bit skeptical on using .just extension for modules — at least in the first iteration.

I'm worried about folder bloat if this default were any of the three you suggested. If, on the other hand, modules were to live inside one directory deeper first, then I don't have a strong feeling on any of the suggestions. What I mean is something like just_modules/NAME/Justfile, just_modules/NAME.just or just_modules/NAME/mod.just — i.e. that all submodules would be inside just_modules (the name is not important) directory first, so that on the root level, there would only be the root Justfile and another directory for the modules.

We should come up with something that lets just foo bar RECIPE work in that case, since as you note / is already interpreted on the command line.

This could work — it would also be possible to give a helpful error should someone have this kind of Justfile:

mod foo

foo:
  echo "hi"

and saying something like this:

Error: Names of modules and tasks must be unique. You have defined both a module named foo and a task named foo. Rename one of them to avoid this error.

> 1 | mod foo
    |     ^^^
  2 | 
> 3 | foo:
    | ^^^
  4 |   echo "Hello, world"

That way we wouldn't even have to figure out any special syntax 🙌

One thing that needs to get resolved:

Can a recipe in one module have dependencies on a recipe in another module? And if so, how do they refer to recipes in other modules?

Is this something that you need or would use? Also if there are others out there who would make use of modules, I'd definitely be interested in your thoughts.

Hoo boy, we're going to the dependency management domain 😅. If at all possible, I think we should avoid making any decisions on automatic behavior yet, in order to see how submodules would be used on their own.

If one would need to share recipes, one could refer to a module by path, e.g. by climbing up the tree:

# <root>/Justfile
mod foo

hello:
  foo bar greet World


# <root>/just_modules/foo/Justfile
mod bar "../bar/Justfile"


# <root>/just_modules/bar/Justfile
greet subject:
  echo "Hello {{subject}}"

Then just would need to handle cyclical dependencies somehow. I assume dependency trees will be quite small, so raising an error and refusing to work with cyclical dependencies could be an OK tradeoff.

So if one would have something like this:

# <root>/Justfile
mod foo

hello:
  foo bar greet World


# <root>/just_modules/foo/Justfile
mod bar "../bar/Justfile"


# <root>/just_modules/bar/Justfile
mod foo "../foo/Justfile"

greet subject:
  foo bar greet "Hello {{subject}}"

just could raise an error when loading the module trees before evaluating the module contents.

Error: Cyclical dependencies are not allowed. Module foo is being loaded from one of its submodules again, resulting in a cycle.

foo module was first loaded from <root>/Justfile:1:

> 1 | mod foo
    | ^^^^^^^
  2 | 
  3 | hello:

Cyclic dependency loading was discovered in <root>/just_modules/bar/Justfile:1:

> 1 | mod foo "../foo/Justfile"
    | ^^^     ^^^^^^^^^^^^^^^^^
  2 | 
  3 | greet subject:

@casey
Copy link
Owner Author

casey commented Feb 20, 2019

Hey, it's completely OK to take your time to reply, or to not even reply at all ☺️. Nobody's getting paid on working on this, so any comment at all is only a value add, no matter how long it takes to respond. 💞

Aww, thanks 😊

I was thinking that given mod NAME just should look in NAME.just or NAME/mod.just or NAME/Justfile, but I'm not sure if that's weird. What do you think?

Hmm yeah, we can make the location optional for sure — but I'm not sure if we know yet what would be a good choice from all of these, so I'm a bit weary on making the decision already in the first implementation.

Yeah, definitely. Hmm, so maybe the best thing to is to wait until we have more experience with the feature, before picking a default location or locations that just should look for submodules.

If we do the explicit-path form, as you suggested:

mod foo "foo/bar/whatever"

Then we can wait and see which, if any, default location make sense, based on user feedback

I'm worried about folder bloat if this default were any of the three you suggested. If, on the other hand, modules were to live inside one directory deeper first, then I don't have a strong feeling on any of the suggestions. What I mean is something like just_modules/NAME/Justfile, just_modules/NAME.just or just_modules/NAME/mod.just — i.e. that all submodules would be inside just_modules (the name is not important) directory first, so that on the root level, there would only be the root Justfile and another directory for the modules.

I think you have a good point about folder bloat, and we can use the explicit-path form at first to avoid committing to anything.

We should come up with something that lets just foo bar RECIPE work in that case, since as you note / is already interpreted on the command line.

This could work — it would also be possible to give a helpful error should someone have this kind of Justfile:

mod foo

foo:
  echo "hi"

and saying something like this:

Error: Names of modules and tasks must be unique. You have defined both a module named foo and a task named foo. Rename one of them to avoid this error.

> 1 | mod foo
    |     ^^^
  2 | 
> 3 | foo:
    | ^^^
  4 |   echo "Hello, world"

That way we wouldn't even have to figure out any special syntax 🙌

I think that's a good idea. One thing that I've learned from writing just is that it pays off to be as restrictive as possible initially.

If you allow a construct, it's annoying to deprecate it and can be dangerous to suddenly change its behavior in a way that users don't expect. However, if you didn't allow it in the first place, adding it later is very easy.

Hoo boy, we're going to the dependency management domain 😅. If at all possible, I think we should avoid making any decisions on automatic behavior yet, in order to see how submodules would be used on their own.

Yeah, good idea. This stuff is why the idea of adding submodules has kind of freaked me out, so it's probably best to implement a true MVP first, and then expand as necessary.

If one would need to share recipes, one could refer to a module by path, e.g. by climbing up the tree:

# <root>/Justfile
mod foo

hello:
  foo bar greet World


# <root>/just_modules/foo/Justfile
mod bar "../bar/Justfile"


# <root>/just_modules/bar/Justfile
greet subject:
  echo "Hello {{subject}}"

Then just would need to handle cyclical dependencies somehow. I assume dependency trees will be quite small, so raising an error and refusing to work with cyclical dependencies could be an OK tradeoff.

just actually already raises errors on cyclical dependencies. It can detect cyclical dependencies in recipes:

foo: bar
bar: foo
: just
error: Recipe `foo` has circular dependency `bar -> foo -> bar`
  |
1 | foo: bar
  |      ^^^

And cyclical dependencies in variable assignments:

foo = bar
bar = foo
: just
error: Variable `bar` depends on its own value: `bar -> foo -> bar`
  |
2 | bar = foo
  | ^^^

So if one would have something like this:

# <root>/Justfile
mod foo

hello:
  foo bar greet World


# <root>/just_modules/foo/Justfile
mod bar "../bar/Justfile"


# <root>/just_modules/bar/Justfile
mod foo "../foo/Justfile"

greet subject:
  foo bar greet "Hello {{subject}}"

just could raise an error when loading the module trees before evaluating the module contents.

That sounds good. Static analysis for the win 🧐

@valscion
Copy link

Great! I guess we're now in the "plan is ready, time to start coding" phase 😅. My rust skills are, err..., rusty — I have been meaning to get to know rust a bit but I haven't had the time. Is there any way I could help even though I would not be able to implement this?

Again, even if this would not be implemented, that would also be OK ☺️. Don't feel like you have any obligations to do this if you don't have the time or the motivation to do free work 😄

@runeimp
Copy link

runeimp commented Feb 21, 2019

I like where this seems to be going for MVP. One addition I'd really appreciate is the ability to point to a module via ~ in the path. I share much of my setup between computers but often have different home directories on different systems. I also have some boilerplate I put in every single Justfile I've ever created. So if the following would resolve:

# <root>/Justfile
mod foo "~/.config/just_awesome/Justfile"

I would be extremely happy. 😃

@valscion
Copy link

@runeimp do you mean that ~ would point to $HOME or to the <root> or what? It isn't quite clear to me what you mean

@casey
Copy link
Owner Author

casey commented Feb 22, 2019

Fortunately, before needing to worry about implementing it, we can continue to noodle on the design!

I've been thinking that using strings to represent paths might be problematic. Paths differ on different platforms, e.g. unix and windows:

  • ~ does not exist on Windows
  • Drive prefixes (C:) do not exist on linux
  • / vs \ delimiter

If module paths are strings, without some extra compatibility mechanism, justfiles will not be portable across platforms.

A couple ideas to resolve this:

  1. Parse strings as unix-style on all platforms, and translate them into platform-native paths on non-unix platforms

  2. Use a platform-neutral non-string representation, like foo::bar::baz, which translates to ./foo/bar/baz on linux and .\foo\bar\baz on windows

I like 2 a bit better, but it comes with problems of its own. If modules are required to be identifiers, are spaces and special characters allowed?

@valscion
Copy link

Fortunately, before needing to worry about implementing it, we can continue to noodle on the design!

Yay! 😄

I've been thinking that using strings to represent paths might be problematic.

Might be, yes

  1. Parse strings as unix-style on all platforms, and translate them into platform-native paths on non-unix platforms

I like this approach better than a custom syntax. This would map to how e.g. node does imports from other files — they just support / syntax for folder traversal. That's also how ruby works if I'm not mistaken.

If we were to take any other choice than using / syntax, I suppose it would be confusing to people. We could always show a helpful error when using / instead of :: or some similar name, though.

I am only speaking from my experiences, though. But from my point of my view, I would expect / syntax to "just work". Anything else would be confusing.

@runeimp
Copy link

runeimp commented Feb 25, 2019

@valscion I meant to point to $HOME so I can target common module(s) I will use on all systems I work on. So to be clear either ~ or some methodology that allows me to target my home directory whatever that name might be on any given system. I would settle for any decided system. But I'm partial to POSIX methodologies.

@runeimp
Copy link

runeimp commented Feb 25, 2019

@casey Actually Windows has supported POSIX/UNIX style paths since Windows XP. And in DOS prior to that if I remember correctly. It just prefers \ for historical reasons. So I highly recommend sticking with the POSIX/UNIX standard pathing.

@valscion
Copy link

valscion commented Feb 26, 2019

Thanks for clarifying, @runeimp ☺️. And thanks for the additional tidbit about Windows supporting forward slash in paths. I agree that sticking with / pathing would be a good idea.

@casey
Copy link
Owner Author

casey commented Mar 4, 2019

I tried it out, and it looks like windows doesn't support ~. If we go with concrete / syntax, I'll parse it instead of using it directly, so that I can error on non-portable syntax.

@casey
Copy link
Owner Author

casey commented Mar 4, 2019

I wish I had time to dedicate to this, but it'll have to go on the back burner for now. It's a big project, with lots of implications codebase-wide, but if someone is interested in picking this up, I can provide some pointers.

@runeimp
Copy link

runeimp commented Mar 5, 2019

@casey Actually, ~ is supported by Windows PowerShell but not CMD. You'd have to do a swap for %UserProfile% in CMD/Command Prompt.

@casey casey added this to the soon milestone Apr 16, 2019
@casey
Copy link
Owner Author

casey commented Apr 19, 2019

I'm still thinking about this and definitely want to do it.

I landed a much improved lexer, and I'm rewriting the parser to better separate the actual parsing (turning a token stream into higher-level structures) and consistency checks (duplicate variable names, circular dependencies, etc). Both will make extensions like these easier.

I think the way that I can best move forward with this is to start with inline modules. From rust, for example:

// inline module
mod foo {
  fn bar() {}
}

// out of line module, loaded from `baz.rs` or `baz/mod.rs`
mod baz;

By starting with inline modules, we can nail down the semantics of submodules and subcommands, without having to think about loading multiple files from disk, which will be a separate complication.

We could lift rust's syntax:

mod foo {
  bar:
    echo baz
}
$ just foo bar
baz

Another option is to use whitespace:

mod foo
  bar:
    echo baz

Inter-module paths would be :: delimited:

# call foo::bar as a dependency
yik: foo::bar

I think that mod NAME { ... } is probably the most readable and the least alien. What do y'all think?

cc: @cledoux, since he was interested in including other justfiles, possibly conditionally, which this could be extended to. By adding some kind of conditional activation of modules,

@windows
mod build "build-windows.just"

@unix
mod build "build-unix.just"

@casey
Copy link
Owner Author

casey commented Apr 21, 2019

I'm leaning towards using something with {...}. Curly brace delimiters are widely used and will be easier for users to pick up and recognize.

Also thinking about "block recipes", which introduce an anonymous scope just for that recipe, for local variables:

# recipe `foo` with deps `a` and `b`
foo: a b {
   #  undecided mystery syntax in here
}

@casey casey modified the milestones: soon, eventually May 27, 2019
@ethankhall
Copy link

Hi @casey, I might have some free cycles coming up. Do you have a "current" design/expected syntax on what's wanted for this feature?

@casey
Copy link
Owner Author

casey commented Aug 24, 2019

Hi @ethankhall!

I'm still pretty uncertain about the syntax. I think the way to go is pick some syntax, implement modules with that syntax, and it behind a flag. Then we can bikeshed the syntax and details of the semantics after we have more experience with the feature.

Also, we should implement in-line modules first, and then later implement out-of-line modules (i.e. modules in another source file) since that's more complex.

Some syntax possibilities for a module foo containing a recipe bar:

curly braces

foo {
  bar:
    echo hello
}

double colon

foo::
  bar:
    echo hello

keyword + colon

mod foo:
  bar:
    echo hello

I'm leaning towards the double colon syntax:

  • module paths are often delimited with ::, which will hopefully make it easier for users to learn and remember
  • for better or worse, Just syntax is already terse and whitespace-sensitive, so those possible objections to the double-colon syntax are less salient
  • the lexer already handles emitting whitespace-based INDENT and DEDENT tokens, so this should be the easiest to implement given what already exists
  • GNU make has recipes that are written with a :: so existing syntax highlighting will work

Out-of-line modules can just omit the module suite:

foo:: # this loads `foo.just` or `foo/mod.just`

Or provide a path to the file to load:

foo:: path/to/some/file.just

But that can be done later.

How does that all sound? Please let me know if you think there are particularly strong reasons to prefer another syntax.

@casey
Copy link
Owner Author

casey commented Aug 24, 2019

This will likely be a pretty complex feature, and I'm very happy to provide as much support as needed to implement it!

@ethankhall
Copy link

Sounds good to me. I will look over the code base and see what I can find. I'll come up with an impl plan and post it here, or rough PR with my thoughts.

I'll reach out with questions!

@casey
Copy link
Owner Author

casey commented Aug 27, 2019

Sounds good! In general, parsing and consistency analysis is pretty gross, so feel free to shoot me any questions about it.

@casey casey removed the help wanted label Oct 27, 2019
@MadBomber
Copy link

I agree its weird out of context. Just pushed step one. I have the flag added to the config object and the help test modified to pass with the new option. Next step is to add the business logic to the analyzer and a few other files.

@casey
Copy link
Owner Author

casey commented Jul 19, 2021

@MadBomber Sounds good! Feel free to push a WIP draft PR if you want to get early feedback or have questions about anything.

@valscion
Copy link

Hmm I wonder if #1095 was supposed to entirely close this issue?

@casey
Copy link
Owner Author

casey commented Feb 16, 2022

Ah, sorry, no I definitely didn't meean to close this issue

@casey casey reopened this Feb 16, 2022
@hhoeflin
Copy link

Cool project - this import sounds very interesting. I had 2 questions:
a) @MadBomber - did you also try https://logological.org/gpp or just the m4 preprocessor?
b) @casey For something like this, an option to read a justfile from stdin would be useful. How difficult do you think that would be?

Thanks - very great work. I love how many people create cool CLI tools in Rust these days!

@MadBomber
Copy link

@hhoeflin I've used M4 before to do similar things - docker compose junk. This time I wanted to see how close I could come with a Ruby/Crystal implementation that shared a common code base. Still not there. Like everyone else time is in short supply these days.

The justprep project is not general purpose. Its a specific stop-gap measure for use with the just utility until such time as the new modular architecture can be implemented. M4 and GPP could do the same thing; but, I would not have as much fun playing with Ruby/Crystal cross-breeding.

@casey
Copy link
Owner Author

casey commented Apr 12, 2022

@hhoeflin I don't think reading a justfile from stdin would be hard, exactly, but it might be a tricky PR, since it would require modifying a lot of error messages. Feel free to open an issue!

@hhoeflin
Copy link

Thank you, but looking at this https://serverfault.com/questions/398514/pass-a-pipe-to-a-command-that-expects-a-filename I learned it is not actually necessary and there is a good workaround, at least on linux in bash. Always something new to learn :)

@nickgarber
Copy link

As I learn about just I appreciate the balance being struck by the project lead and community.

So much so that it seems that just includes by default many of the workflow/ergonomic enhancements I've built on to GNU-make over time. This makes just feel like coming home.

The one thing I've looked for and not seen is GNU-make style includes, though it seems like there are a few related discussions.


I'd suggest that the ideas of modules and namespace could be made more independent from each other. Both are useful but could perhaps be implemented with some optional interdependence.

In fact, the ability to compose recipes from modular Makefiles based on path is one of most successful and comfortable patterns I've seen emerge.

For my use-cases I've taken to adding an -include .assets/makefile/*.mk to a minimal Makefile in the project root, with glob-included modular makefiles, each with prefixed target names according to their modules as a simple form of namespacing / collision avoidance.

It's very useful to be able to conceptually organize content while avoiding the disadvantages of recursive calls. It helps to keep the size of each individual file to a reasonable minimum, as well as making it easy to disable a module by renaming it from .mk to .mk.disabled.

I trust the course of this issue and just wanted to put forward the merits of this approach since it's something that's worked well for me that I can happily recommend.

@MadBomber
Copy link

Interesting idea to make the include path glob'able. Current justprep the pre-processor stop-gap to modules feature only allows absolute or relative paths. I will look at adding a form of glob to the path.

@MadBomber
Copy link

@nickgarber I just released v1.2.2 of justprep a stop-gap pre-processor for just utility that adds the path globing capability. It works in both the Ruby gem and Crystal binary implementations.

I took this issue as the excuse to refactor the code base to extract common methods into their own files for use by both Ruby and Crystal. I added the file extension ".crb" to those files which can run under both implementations. I originally thought of ".crb" to meaning Crystal and Ruby; but, now I've come closer to thinking of it as "compiled ruby."

https://github.com/MadBomber/justprep

@SteveChurch
Copy link

Hi @casey, firstly this is an outstanding tool! Kudos. Secondly, is there any update on the progress or idea when this feature may be worked on? I appreciate you're busy and have a life outside of GH 😄 - but submodules would be a killer feature to have 😄

@MadBomber
Copy link

@SteveChurch There is a "fake module" convention that works in most circumstances. I documented in the wiki for the pre-processer justprep

https://github.com/MadBomber/justprep/wiki/Fake-Module-Convention

@casey
Copy link
Owner Author

casey commented Feb 7, 2023

HI @SteveChurch, thanks for checking in! I actually don't have a life outside of GH 😅 but I'm currently working on another project, ord, that's getting a huge amount of attention, so it's taking up all spare bandwidth.

No updates to report on modules, but check out include directives which, depending on your use-case, might be useful.

@Splines
Copy link

Splines commented Dec 17, 2023

@SteveChurch There is a "fake module" convention that works in most circumstances. I documented in the wiki for the pre-processer justprep

https://github.com/MadBomber/justprep/wiki/Fake-Module-Convention

Thanks a lot for the link! However, I haven't found a solution yet to pass arguments, have you heard of any? E.g. I try to call

just myModule myRecipeInModule myArgument

// inside the module file:
myRecipeInModule myArgument="defaultValue":
  @echo {{myArgument}}

I've also tried to set positional-arguments in the root .justfile which didn't work.

Note I've also seen this Interactive Chooser section in the docs. Maybe it's useful for other solutions to implement "fake" modules?


Edit: To clarify my goal a bit: We have a big project and try to use just with command groups. The solution you provided here, allows us -- in principle -- to have a root .justfile and then a folder commands/ where we can put in more justfiles e.g. docker.justfile. The main justfile delegates to the module via

docker recipe="help":
    just -f ./commands/docker.justfile {{recipe}}

Right now, it's just passing the arguments that is problematic for us.

@Splines
Copy link

Splines commented Dec 18, 2023

Ok, in the meantime, I found a solution for my "problem" using variadic parameters. You may find this helpful (or not ;)

# ./justfile

docker recipe *ARGS:
    just -f ./commands/docker.justfile {{recipe}} {{ARGS}}
# ./commands/docker.justfile

# List all available Docker commands.
help:
    @just --list --justfile {{justfile()}}

alias s := shell
# Puts you into a shell of your desired development docker container.
shell name="myDefaultContainer":
    docker exec -it $(docker ps -qf "name=development-{{name}}") bash

@valscion
Copy link

valscion commented Dec 20, 2023

Ok, in the meantime, I found a solution for my "problem" using variadic parameters. You may find this helpful (or not ;)

Yeah this is basically the same solution we've been quite happy with since the beginning!

The only downside of this is that if you have special characters (such as whitespaces or quotes), then passing them down successfully to the downstream justfiles gets quite complex.

You might want to try the positional arguments as per the comment by @casey here:

I haven't tried that fully yet myself.

@Splines
Copy link

Splines commented Dec 20, 2023

@valscion Thanks, will keep that in mind and come back to it if/once we encounter that situation 👍

@casey casey mentioned this issue Dec 27, 2023
@casey
Copy link
Owner Author

casey commented Dec 28, 2023

IT HAS BEEN NIGH ON FIVE YEARS, BUT JUST FINALLY HAS MODULES.

I just released 1.19.0, which includes modules.

See the readme for more information, but in short, modules can be defined using a mod statement, for example, mod foo, which will load a module from foo.just, foo/mod.just, or foo/justfile. Recipes in modules can then be called like subcommands. If the module foo has recipe bar, it can be called with just foo bar.

There are currently a bunch of limitations and missing features, so modules are unstable for now. See #929 for more details.

Thanks so much for everyone's copious patience! And special thanks to @neunenak, who laid a bunch of the ground work which enabled this.

If this doesn't work for your use case, please feel free to open an issue detailing what doesn't work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants