[Bug] unable to use hydra with click module

[KosukeMizufune]

🐛 Bug

I tried to use hydra with click module for creating CLI tools, but got an error.

To reproduce

import click
import hydra


@click.command()
@click.option('--foo', default='foo')
def main(**kwargs):
    my_app()
    print('hoge')
    print(kwargs)


@hydra.main(config_path='config.yml')
def my_app(cfg):
    print(cfg.pretty())


if __name__ == '__main__':
    main()

** Stack trace/error message **

hydra.errors.MissingConfigException: Cannot find primary config file: config.yml
Search path:
	pkg://hydra.conf (from hydra)
	pkg://click (from main)

Expected Behavior

System information

• hydra-core==0.11.3
• Click==7.0
• python 3.6.10

Additional context

This error might be due to the arguement 3 of this line:

hydra/hydra/_internal/utils.py

Line 142 in 2a99161

calling_file, calling_module = detect_calling_file_or_module(3)

If I change this argument to -1, then it works well.

[omry]

Hydra is meant to completely replace argparse or click.
It's not compatible with Click (I think you are the first one trying as far as I know).

Instead of using a Click module here, just put foo: foo in your config.yaml file:

config.yaml

foo: foo

app.py

@hydra.main(config_path='config.yml')
def my_app(cfg):
    print(f"foo={cfg.foo}) # here is foo


if __name__ == '__main__':
    main()

$ python app.py foo=bar
foo=bar

$ python app.py foo=foo
foo=foo

Closing as invalid, but happy to discuss further if needed.

[tasercake]

Does Hydra offer anything analogous to Click's group?

I'm used to doing:

@click.group()
def main():
    pass

@main.command()
def sub_command_1():
    # Sub-functionality

@main.command()
def sub_command_2():
    # Sub-functionality

in Click and was hoping to do something similar with Hydra.

The workaround I'm using for now involves setting a task key in my config and calling various functions according to its value, but that seems brittle compared to what Click offers.

[omry]

Generally speaking, Hydra not calling code for you.

See this for example of driving runtime behavior from config:
https://hydra.cc/docs/patterns/objects

Hydra 1.0 will also cleanly support regular methods:
https://hydra.cc/docs/next/patterns/objects

[boweeb]

For anyone late to the party (like me), there is a supported way to decouple the CLI and thus use click and hydra together. I just discovered this project this morning so this may not be the best answer but the the "compose api" looks like the correct path.

Reference:

• Docs: Compose API
• Compose API Example

[omry]

The Compose API is certainly an option if you just need the config composition capabilities from Hydra.
Keep in mind that you lose command line integration, multi run, remote launching, tab completion and probably a few other things by not using @hydra.main.

[rr-hari-prakash]

Does Hydra offer anything analogous to Click's group?

I'm used to doing:

@click.group()
def main():
    pass

@main.command()
def sub_command_1():
    # Sub-functionality

@main.command()
def sub_command_2():
    # Sub-functionality

in Click and was hoping to do something similar with Hydra.

The workaround I'm using for now involves setting a task key in my config and calling various functions according to its value, but that seems brittle compared to what Click offers.

Do we have this functionality in hydra to execute functions individually from command line (similar to @click.command)?

[Jasha10]

Do we have this functionality in hydra to execute functions individually from command line (similar to @click.command)?

Not exactly.
The @hydra.main decorator is the closest analog to @click.command.