From 7666cf6863523d0bc597236918d00a30c82574a2 Mon Sep 17 00:00:00 2001 From: Jieru Hu Date: Tue, 8 Dec 2020 18:28:01 -0800 Subject: [PATCH] Add configerator config source doc --- website/docs/fb/configerator-config-source.md | 52 +++++++++++++++++++ website/docs/fb/fbcode.md | 2 +- website/sidebars.js | 1 + .../fb/configerator-config-source.md} | 30 ++++++++--- 4 files changed, 76 insertions(+), 9 deletions(-) create mode 100644 website/docs/fb/configerator-config-source.md rename website/{docs/fb/configerator_config_source.md => versioned_docs/version-1.0/fb/configerator-config-source.md} (51%) diff --git a/website/docs/fb/configerator-config-source.md b/website/docs/fb/configerator-config-source.md new file mode 100644 index 00000000000..63a1f3d851f --- /dev/null +++ b/website/docs/fb/configerator-config-source.md @@ -0,0 +1,52 @@ +--- +id: fbcode-configerator-config-source +title: Configerator Config Source Plugin +--- + +The ConfigeratorConfigSource plugin makes it possible for Hydra applications to consume a domain of configs from configerator. + +### Dependency + +Add the following to your `TARGET` file +```commandline +//fair_infra/fbcode_hydra_plugins/configerator_config_source:configerator_config_source +``` + +### Usage + +1. Since `configerator` does not have a client for listing configs under a path, you need to group your configs under +a `domain` for Hydra to access. + 1. Matching the name of your domain and directory of configs is necessary for the plugin to extract information on the full config names, due to limitations from Configerator APIs + 1. For example, the config paths returned by Configerator API could look like `fair_infra/hydra_plugins/configerator_config_source/example/db/mysql`. The plugin needs to know where the directory of configs begins ([`example`](https://fburl.com/diffusion/7c0c5tig)), in order to determine the full config name (`db/mysql`). So in this case the domain should be named [`example.cconf`](https://fburl.com/diffusion/pyymoo1t) +1. Create a [SearchPathPlugin](https://hydra.cc/docs/next/advanced/search_path) to add the Configerator path to the list of search paths. + + 1. The path you add in your SearchPathPlugin should be the name of your domain of configs, such as in this [example SearchPathPlugin](https://fburl.com/diffusion/ljggtux5) + + Note: There will be an easier way to add the Configerator path to the list of search paths, with a planned change in Hydra 1.1 to [allow configuring the search path via the config file](https://github.com/facebookresearch/hydra/issues/274). + +### Example: + +#### Example SearchPathPlugin +[`ConfigeratorExampleSearchPathPlugin`](https://fburl.com/diffusion/vwa82fbg) adds the example configerator domain to the search path of the example applications. + +#### Reading primary config from configerator +This example reads its primary config from configerator [here](https://fburl.com/diffusion/twk3smkj) which has a default list defined. + +```commandline +$ buck run //fair_infra/fbcode_hydra_plugins/configerator_config_source/example/primary_config:my_app +... +Parsing buck files: finished in 1.1 sec +... +{'driver': 'mysql', 'user': 'alau'} +``` + +#### Compose config with configerator +This example reads its primary config from local yaml file `primary_config.yaml` but reads config groups info from configerator. + +```commandline +$ buck run //fair_infra/fbcode_hydra_plugins/configerator_config_source/example/config_group:my_app -- +db=mysql +... +Parsing buck files: finished in 1.1 sec +... +{'foo': 'bar', 'driver': 'mysql', 'user': 'alau'} +``` diff --git a/website/docs/fb/fbcode.md b/website/docs/fb/fbcode.md index ca53eb8ae52..de176c20d62 100644 --- a/website/docs/fb/fbcode.md +++ b/website/docs/fb/fbcode.md @@ -16,7 +16,7 @@ title: Hydra at fbcode ### Facebook specified plugins - fbcode_defaults : Changes configuration defaults to be appropriate for fbcode (e.g: Output directories are in `fbcode/outputs` and `fbcode/multirun`) - - configerator_config_source: See doc for more information. + - configerator_config_source: See [doc](/fb/configerator-config-source.md) for more information. #### TARGETS Hydra includes buck TARGETS you can use in fbcode. In general, if there is TARGET there are two options: diff --git a/website/sidebars.js b/website/sidebars.js index 41cdec4cc8d..a406dd88971 100755 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -158,6 +158,7 @@ module.exports = { 'fb/fbcode', 'fb/internal-fb-cluster', 'fb/fair-cluster', + 'fb/fbcode-configerator-config-source', ]), } } diff --git a/website/docs/fb/configerator_config_source.md b/website/versioned_docs/version-1.0/fb/configerator-config-source.md similarity index 51% rename from website/docs/fb/configerator_config_source.md rename to website/versioned_docs/version-1.0/fb/configerator-config-source.md index 5dac1d1f003..29ceda868e2 100644 --- a/website/docs/fb/configerator_config_source.md +++ b/website/versioned_docs/version-1.0/fb/configerator-config-source.md @@ -23,15 +23,29 @@ a `domain` for Hydra to access. Note: There will be an easier way to add the Configerator path to the list of search paths, with a planned change in Hydra 1.1 to [allow configuring the search path via the config file](https://github.com/facebookresearch/hydra/issues/274). -1. Modify the `config_name` in the `@hydra.main` decorator of your application. - 1.To fetch a config file, one way is to modify the `config_name` in the `@hydra.main` decorator of your application should be a logical path to a config within the domain passed to your SearchPathPlugin. +### Example: -1. Your config file can contain a [defaults list](https://hydra.cc/docs/next/tutorials/basic/your_first_app/defaults) to select defaults for config groups. To use a defaults list in your config, import the [`configerator/source/fair_infra/hydra_plugins/configerator_config_source/defults`](https://fburl.com/diffusion/xhmy4bwc) thrift structure and specify a default_dict (for defaults in config groups) and/or default_string (for default configs) -- When the application is run, the configs specified in the defaults list are loaded by default. -- For example, [`config.cconf`](https://fburl.com/diffusion/tm9qbpm8) imports the `hydra_configerator_config_default` thrift to create the defaults list, with [`mysql`](https://fburl.com/diffusion/cxgennae) as the default for the [`db`](https://fburl.com/diffusion/99i3uxpu) config group +#### Example SearchPathPlugin +[`ConfigeratorExampleSearchPathPlugin`](https://fburl.com/diffusion/vwa82fbg) adds the example configerator domain to the search path of the example applications. -### Example: +#### Reading primary config from configerator +This example reads its primary config from configerator [here](https://fburl.com/diffusion/twk3smkj) which has a default list defined. -An example [application](https://fburl.com/diffusion/pndzq58m) using the ConfigeratorConfigSource plugin and example [SearchPathPlugin](https://fburl.com/diffusion/j86krh3r) are provided. The example app has the decorator `@hydra.main(config_name=…)`, where config_name specifies the path to the configs that will be loaded and printed. +```commandline +$ buck run //fair_infra/fbcode_hydra_plugins/configerator_config_source/example/primary_config:my_app +... +Parsing buck files: finished in 1.1 sec +... +{'driver': 'mysql', 'user': 'alau'} +``` -To run the app, run `buck run //fair_infra/fbcode_hydra_plugins/configerator_config_source/example:my_app` from the fbsource/fbcode directory. +#### Compose config with configerator +This example reads its primary config from local yaml file `primary_config.yaml` but reads config groups info from configerator. + +```commandline +$ buck run //fair_infra/fbcode_hydra_plugins/configerator_config_source/example/config_group:my_app -- +db=mysql +... +Parsing buck files: finished in 1.1 sec +... +{'foo': 'bar', 'driver': 'mysql', 'user': 'alau'} +```