importing within module

[vasdee]

I've come across an issue I'm struggling to understand. I thought I knew the limitations of using modules but this one has tripped me up.

Our usage of just is centralised, as in a single set of recipes living in ~/.config/just/ allows us to operate over dozens of project repositories. The basic premise is we have a single justfile in any given project, which imports the centralised just file from ~/.config/just/ - which in turn declares a series of modules to use, for example docker, pre-commit, security, python etc.

I'm looking at ways that a project could 'override' recipes within a module, if the need arises. My initial thinking was to provide an optional import at the end of each module, that looks for a local 'override' file matching that module ie. The docker module would try import? "docker.override.just", the security module would try import? "security.override.just" etc. In my head I can't think of a reason this wouldn't work. but the override of a recipe never happens, it is always the original that remains.

I've broken down the issue I'm seeing in a simple scenario that mimics that behavior.

docker.mod.just

set allow-duplicate-recipes
set allow-duplicate-variables

build:
        echo "Original Docker build command"


# if an override file exists for this module, import it here
import? "docker.override.just"

docker.override.just

build:
    echo "Overridden docker build recipe"

hello:
    echo "hello from overide"

Justfile

set allow-duplicate-recipes
set allow-duplicate-variables

mod docker "docker.mod.just"

In this scenario, running docker::build doesn't run the expected recipe

❯ just docker::build
echo "Original Docker build command"
Original Docker build command

However the supplemental recipe is in fact included, so there's no issue with the import

❯ just docker::hello
echo "hello from ovveride"
hello from overide

Any clues as to why this wouldn't work as expected?

[casey]

Thanks for the issue!

So the reason this doesn't work is because the allow-duplicate-recipes tie-breaking mechanism is prefer recipes defined in the current file vs an import. (More specifically, it uses the file depth, where being in an import increases the file depth by one.)

This works for the use-case of overriding recipes defined in another file:

set allow-duplicate-recipes

# contains foo recipe
import "import.just"

# override foo defined in `import.just`
foo:

But of course this doesn't have the desired effect for your use-case.

It's lame, but if you introduce another layer of indirection, such that override recipes are at the same import depth, you can do what you want:

docker.mod.just:

set allow-duplicate-recipes

import "actual-docker-recipes.just"
import? "docker.override.just"

I can't really think of a better way to solve this at the moment. We could add a setting which reverses the tie breaker, preferring recipes which are at a larger file depth, but that seems like a very niche setting.

[vasdee]

@casey thanks for the workaround and explanation, much appreciated.

Happy for you to close this ticket, but would you consider a new ticket for perhaps allowing imports to have an optional prefix? With the above issue and a few outstanding module issues, I'm starting to circle back my thinking to just use imports. The one gotcha with that, is that keeping it organised when you have a lot of recipes is difficult. If I was able to prefix an import with an optional string and a delimiter like . or / I'd be perfectly happy

For example, say i have:

docker.just

build:
    echo "docker build ..."
login: 
    echo "docker login ..."

Then in my main Justfile I do something hypothetical lke

import "docker.just" "docker/"

This would prefix all the imported recipes with the namespace "docker/" (or similar), so I could now run

just docker/build or just docker/login

Happy to raise a ticket for this, if you think it is worth it.

[casey]

I'd rather not add features to work around limitations with modules, just since I do hope to address those limitations going forward, and I'd rather not have features become redundant as modules are improved. Can you manually prefix recipes with docker- in order to keep things organized?

[vasdee]

Manually prefixing is what we have done in the past, before moving to sub-modules. The main issue, which really is only due to the way I liked to organise things when I was using Make, is that there is no nice delimiter. Allowing a \ or / or : or . within the recipe name would help - I mentioned this a bit more in the module improvement tracker ticket

#2252 (comment)

[casey]

Adding additional characters to identifiers would make it impossible to use those characters for other things in the future. / and \ couldn't be used, because they are already used as a join operator and as a escape character, but using : or . would mean they couldn't be used in the future for other things, for example in a ternary operator, or for property access.