[dev] Config Registration Functions in __init__.py

[romesco]

implements config registration for optim via __init__.py

If this looks good, I'll broadcast the same strategy across the rest of the configs we currently have.

Partially Addressing #53

[romesco]

Before merging, I just want to point out that because of the way pytorch structures its files (and __init__.py for certain modules), the optimizer register can be done like this:

hydra_configs.torch.optim.register()

however, losses and data-related must be done like this:

hydra_configs.torch.nn.modules.register()
hydra_configs.torch.utils.data.register()

For now it's not possible to do:

hydra_configs.torch.nn.modules.loss.register()

because loss.py is an auto-generated file and we aren't putting logic in those.

Semantically, registering optimizers with optim.register() is nicer than the case for loss which is registered by modules.register().

I'm open to ideas for ameliorating this semantic ambiguity.

[omry]

feels like this would better be inside loss.py and __init__.py would just call it from its register function.

[romesco]

It makes sense there, but loss.py is a configen output. This is manual code.

[omry]

maybe we should create a convention:

loss.py # generated
loss_handcoded.py # manual additions.

At this point, the __init__.py file can import them both (and possibly also export from both, making them appear like one file).

[romesco]

not a bad idea. I think we need a convention for handling manual configs anyways.

[omry]

here is an even better idea.
We can add support to custom import to configen at the top and the bottom of the generated configs.
Allowing automatic import of user provided files if they exist.

[romesco]

Could you elaborate on this?

Do you mean chain the 'manual' file to the 'generated' one through importing that is automatically added by configen if a field is passed in the configen conf yaml?

[omry]

not exactly.
I mean importing if a file with a specific name exists.

generated/foo.py

try:
 from . import foo_header
except ImportError:
  pass

# generated code here

try:
 from . import foo_footer
except ImportError:
  pass

This way, you can add manual foo_footer.py and foo_header.py files that will get imported automatically when the user imports foo.py.

[omry]

Overall looks okay.