Proposal(CLI): after_instantiate_classes hook

[AlessandroW]

Description & Motivation

Adds a after_instantiate_classes hook to the Lightning CLI, called after self.instantiate_classes() during the initalization of LightningCLI.

Pitch

While having the Lightning CLI is great, it is not perfect for each use case out-of-the-box. Hence, you included hooks like before_instantiate_classes and describe in the docs how to extend the CLI. Problem is, you cannot extend this feature without hacks or substantial copy-pasta.

I think, to further improve the CLI, without adding any complexity, it makes sense to add a after_instantiate_classes hook, too.

Alternatives

1. Hacks

• Extend the Lightning CLI and run the after_instantiate_classes function before the self._run_subcommand function.
• Problems: it's not intuitive that the function is called there, won't be called if self.subcommand is None

2. Copy-Pasta

• Extend the Lightning CLI and replace the original __init__ with the proposed one.
• Problems: could break with any update, lots of code duplication

Additional context

No response

cc @Borda @tchaton @justusschock @awaelchli @mauvilsa

[AlessandroW]

Glad, the proposal was accepted! Thank you for merging it.

Can you add an example for how the new hook will be used?
#20401 (comment)

I used it for logging and data-specific cross-validation. One reason for this approach was the use of MLFlow's parent/child runs for logging without using an additional dependency like Hydra. For refactoring I moved some parts of the logic into before_fit.

Anyway, my situation is very specific and not a good, simple example. The more important part is Lightning's approach of providing hooks to create such specific solutions without having to write everything from scratch. The after_instantiate_classes is just another case where, e.g., run-specific logging that shouldn't be part of the model or the datamodule can be called.