Get final defaults tree in code

[OrrAvrech]

Hi,

Is it possible to obtain the final defaults tree in code (as dict/object), instead of executing python app.py --info defaults-tree, saving to a file and then reading it? I tried using HydraConfig.get().runtime.choices but it does not provide the full default tree.

In addition, when using the compose API instead of the hydra.main decorator, how can I compute the final defaults tree?

Thanks.

[omry]

There is no public API for it. The exposure of the defaults tree is for debugging purposes.
You could look at the code and call the same internal methods used by --info defaults-tree.

Can you explain the motivation for this?

[OrrAvrech]

Yes, we want to implement hierarchical configurations for our applications and started using Hydra for this purpose.
Our setup consists of multiple micro-services deployed across multiple environments (dev, staging, several prod envs, multi-region, etc.) as well as global configurations that are common to all apps for each environment. We want to efficiently share configurations between environments and services while maintaining clean config files without duplicates.

We need to compute the defaults tree for two main reasons:

1. CI/Testing: when a config changes, we want to run tests to ensure that everything runs as expected. Since some configurations inherit from the modified configuration file, we should generate a list of all child configurations and perform a sanity test for each one. An API that provides access to the defaults tree would be very helpful for this purpose.
2. Deployments: our deployments consume their configurations as YAML files in a volume. Therefore, it is necessary to map the app config and its defaults tree to this volume.

I took a look at the code and accessing the internal methods used by --info defaults-tree didn't look very convenient, which is why I posted this question. But I'll give it another shot if there's no API for it. Do you plan to support this functionality in the future? I mean, since the defaults tree is already being computed, it would make sense to expose it through a public API.

Thanks for you help.

[omry]

CI/Testing: when a config changes, we want to run tests to ensure that everything runs as expected. Since some configurations inherit from the modified configuration file, we should generate a list of all child configurations and perform a sanity test for each one. An API that provides access to the defaults tree would be very helpful for this purpose.

An alternative is to generate the configs using the Compose API, and compare the resulting config to the known tested config.
If a config has changed, you need to re-test the tests using it.

There are no plans to expose this as an API. You are the first user expressing need for it.
For now use the internal API for it.