From 530e0b2541b87dea57782efe5701d1a8f9dddf25 Mon Sep 17 00:00:00 2001 From: romesc Date: Tue, 5 Jan 2021 11:38:52 -0800 Subject: [PATCH 01/12] add WIP OVERVIEW.md --- OVERVIEW.md | 123 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 123 insertions(+) create mode 100644 OVERVIEW.md diff --git a/OVERVIEW.md b/OVERVIEW.md new file mode 100644 index 0000000..3945424 --- /dev/null +++ b/OVERVIEW.md @@ -0,0 +1,123 @@ +# Overview + +The `hydra-torch` repo serves three purposes. + +1. It demonstrates a standardization for creating and organizing Hydra configuration classes for the ML/torch community and beyond. + - If someone wants to create their own `hydra-` repo, `hydra-torch` points towards an upper bound on what it should look like. +2. It unifies a collection of classes across projects in one place, ensuring robust testing, version compatibility, and PyPI distributed packaging for each. + - We see many hydra users reimplementing these classes (and not tracking APIs of the configured projects). `hydra-torch` factors this code out. +3. It provides best practices and guidance on how to organize code and utilize Hydra for research, production, and other innovative use cases. + +##### Terminology for this overview: +- The overall effort and repo is named:`hydra-torch` +- **Library:** The library being configured + - e.g. `torch`, `torchvision` +- **Project:** The project corresponding to the library being configured + - e.g. `hydra-configs-torch`, `hydra-configs-torchvision` +- **Package:** The installable package containing the configs + - here, also `hydra-configs-torch`, `hydra-configs-torchvision`. find definition in the project dir's `setup.py`. + + + +### Repo Structure +``` +📂 hydra-torch +└ 📁 configen # source files used when generating package content. all projects have their own subdirectory here. +└ 📁 hydra-configs-torch # a project corresponding to configuring a specific library (contains package definition) +└ 📁 hydra-configs-torchvision # " +└ 📁 hydra-configs- # " +└ 📁 examples # use-cases and tutorials +``` + +Each `hydra-configs-` defines its own package corresponding to a project it provides classes for. That is, `hydra-configs-torch` contains a package (with its own `setup.py`) which provides classes for `torch`. + +[`hydra-configs-projects.txt`](hydra-configs-projects.txt) specifies a list of projects this repo maintains. When adding a project, please update this file. + + +### Packaging + +This repo maintains multiple packages. An important area of consideration for these packages is to organize them such that they are compatible with each other and have an 'expected', intuitive structure. The easiest way to do this is to enforce a standard for all Hydra configs packages to follow. + +Our approach makes use of [Native Namespace Packages](https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages). This allows all packages to share the top level namespace `hydra_configs`. The only real requirement for this package format is to ensure there is no `__init__.py` located in the namespace folder. + +The format for folder structure per project is always: +``` +📂 hydra-configs- # the dir containing the package of configs for +├ 📁 hydra_configs # the namespace we will always use +│ └ 📁 # e.g. 'torchvision' +│ └ 📁 # e.g. 'transforms' +│ ⋮ +│ └ .py # e.g. 'transforms.py' +└ setup.py # configures this package for setuptools +``` + +The beauty of this approach is that users can be sure the importing idiom is reliably: +```python +from hydra_configs..path.to.module import Conf +``` + +This will retain compatibility even with repositories that define configs for hydra outside of the `hydra-torch` repo as long as they follow this pattern. + +##### Metapackage + +For convenience, we also provide a metapackage called `hydra-torch`. This is defined at the top level of the repo in `setup.py`. Effectively, all this means is that a user can either install each project-specific package as needed or they can obtain them all in one shot by simply installing this singular metapackage. + +### Configen + +We are actively developing the tool, [Configen](https://github.com/facebookresearch/hydra/tree/master/tools/configen) to automatically create config classes for Hydra. Much of the work for `hydra-torch` has helped prototype this workflow and it is still rapidly evolving. + +Currently, the workflow looks like the following: +0. Ensure the most recent configen is installed from master. +1. Edit `configen/conf//configen.yaml`, listing the module and its classes from the project library to be configured. + - e.g. in `/configen/conf/torchvision/configen.yaml`: + ```yaml + modules: + - name: torchvision.datasets.mnist # module with classes to gen for + # mnist datasets + classes: # list of classes to gen for + - MNIST + - FashionMNIST + - KMNIST + - EMNIST + - QMNIST + + ``` +2. In the corresponding project directory, `hydra-configs-`, run the command `configen --config-dir ../configen/conf/`. +3. If generation is successful, the configs should be located in: + - `/hydra-configs-/hydra_configs/path/to/module`. + - for our above example, you should see `MNISTConf, ..., QMNISTConf` in the module, `mnist.py` located in: + `hydra-configs-torchvision/hydra_configs/torchvision/datasets` + +>**Note:** This process is still under development. As we encounter blocking factors, we do the appropriate development on Configen to mitigate future issues. + +### Testing + +All configs must be tested to ensure they are compatible with the latest version of Hydra. At a minimum, each config must be able to successfully yield the desired class when passed to `hydra.utils.instantiate()`. If the class has special functionality, consider making evaluating this a goal of the test suite as well. + +Tests are run through pytest. The CI flow is currently: `CircleCI` -> `nox` -> `pytest`. All tests must pass before anything can be merged. + + +### Versions + +##### Regarding Libraries + +Given the rapid development of libraries in the torch ecosystem, it is likely that there will be a need for multiple simultaneous version specific `hydra-configs-` per library. This is especially relevant as API and type annotation is updated moving forward for a particular library. + +For each library, there will be releases for each `MINOR` version starting with the latest version available at the time of creation of the configs project. + 1. There will be a branch/tag for each version for each library's corresponding configs project. These branches/tags will only contain the content relevant to that particular project. + 2. In `master`, each configs project will maintain the latest supported library version. + +###### Package/Branch Names: +The version for the `hydra-configs-` package will be `MAJOR.MINOR.X` where `MAJOR.MINOR` tracks the library versions and `.X` is reserved for revisions of the configs for that particular `MAJOR.MINOR` should they be required. + +e.g. `hydra-configs-torch==1.6.0` corresponds to `torch==1.6` and if updates are needed for patches from either end, only the `PATCH` version will be updated -> `hydra-config-torch==1.6.1` + +#TODO: write out exact tag format once decided upon + +##### Regarding Hydra + +Many of the features that make using configs compelling are supported in Hydra > 1.1. For this reason, we are aiming support starting at 1.1 . Using an older version of Hydra is not advised, but may still work fine in less complex cases. + + +### Releases +Releases will be on a per-project basis for specific versions of a project. Releases should be tied to a specific branch/tag in the repo that passes all tests via CI. We release via PyPI. From 9f51012fb04114e8e32ee0c7472082ccb75aba58 Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Tue, 5 Jan 2021 17:25:26 -0800 Subject: [PATCH 02/12] Update OVERVIEW.md fix formatting --- OVERVIEW.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/OVERVIEW.md b/OVERVIEW.md index 3945424..1caec3d 100644 --- a/OVERVIEW.md +++ b/OVERVIEW.md @@ -22,11 +22,11 @@ The `hydra-torch` repo serves three purposes. ### Repo Structure ``` 📂 hydra-torch -└ 📁 configen # source files used when generating package content. all projects have their own subdirectory here. -└ 📁 hydra-configs-torch # a project corresponding to configuring a specific library (contains package definition) +└ 📁 configen # source files used when generating package content. all projects have their own subdirectory here. +└ 📁 hydra-configs-torch # a project corresponding to configuring a specific library (contains package definition) └ 📁 hydra-configs-torchvision # " -└ 📁 hydra-configs- # " -└ 📁 examples # use-cases and tutorials +└ 📁 hydra-configs- # " +└ 📁 examples # use-cases and tutorials ``` Each `hydra-configs-` defines its own package corresponding to a project it provides classes for. That is, `hydra-configs-torch` contains a package (with its own `setup.py`) which provides classes for `torch`. @@ -43,12 +43,12 @@ Our approach makes use of [Native Namespace Packages](https://packaging.python.o The format for folder structure per project is always: ``` 📂 hydra-configs- # the dir containing the package of configs for -├ 📁 hydra_configs # the namespace we will always use -│ └ 📁 # e.g. 'torchvision' +├ 📁 hydra_configs # the namespace we will always use +│ └ 📁 # e.g. 'torchvision' │ └ 📁 # e.g. 'transforms' │ ⋮ │ └ .py # e.g. 'transforms.py' -└ setup.py # configures this package for setuptools +└ setup.py # configures this package for setuptools ``` The beauty of this approach is that users can be sure the importing idiom is reliably: From 50e4b3e12fcda60168d8f1784cc490a4d1f92109 Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Thu, 7 Jan 2021 07:21:20 -0800 Subject: [PATCH 03/12] Update CONTRIBUTING.md move configen info to contributing --- CONTRIBUTING.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cd15a5a..a8b3e4e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -22,6 +22,38 @@ There are 3 main ways to contribute starting with the most straightfoward option Please download the formatting / linting requirements: `pip install -r requirements/dev.txt`. Please install the pre-commit config for this environment: `pre-commit install`. + +## Using Configen + +We are actively developing the tool, [Configen](https://github.com/facebookresearch/hydra/tree/master/tools/configen) to automatically create config classes for Hydra. Much of the work for `hydra-torch` has helped prototype this workflow and it is still rapidly evolving. + +Currently, the workflow looks like the following: + +1. Ensure the most recent configen is installed from master. + - `pip install git+https://github.com/facebookresearch/hydra/#subdirectory=tools/configen` +2. Edit `configen/conf//configen.yaml`, listing the module and its classes from the project library to be configured. + - e.g. in `/configen/conf/torchvision/configen.yaml`: + ```yaml + modules: + - name: torchvision.datasets.mnist # module with classes to gen for + # mnist datasets + classes: # list of classes to gen for + - MNIST + - FashionMNIST + - KMNIST + - EMNIST + - QMNIST + + ``` +3. In the corresponding project directory, `hydra-configs-`, run the command `configen --config-dir ../configen/conf/`. +4. If generation is successful, the configs should be located in: + - `/hydra-configs-/hydra_configs/path/to/module`. + - for our above example, you should see `MNISTConf, ..., QMNISTConf` in the module, `mnist.py` located in: + `hydra-configs-torchvision/hydra_configs/torchvision/datasets` + +>**Note:** This process is still under development. As we encounter blocking factors, we do the appropriate development on Configen to mitigate future issues. + + ## Pull Requests We actively welcome your pull requests. From a9107e1528f90ddf0cacd915742ef5adbbf9944e Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Thu, 7 Jan 2021 07:43:12 -0800 Subject: [PATCH 04/12] Update OVERVIEW.md still fine tuning --- OVERVIEW.md | 55 +++++++++++++++++------------------------------------ 1 file changed, 17 insertions(+), 38 deletions(-) diff --git a/OVERVIEW.md b/OVERVIEW.md index 1caec3d..ca1f166 100644 --- a/OVERVIEW.md +++ b/OVERVIEW.md @@ -1,21 +1,25 @@ # Overview -The `hydra-torch` repo serves three purposes. +The audience for this library is Hydra users using PyTorch and related libraries (torchvision, ...) +The goals of this repository are: + +1. Provide a maintained and tested implementation of config classes that can be used instantiate and configure various classes from the supported projects. +2. Provide examples and tutorials demonstrating best practices for using Hydra to configure PyTorch deep learning applications. +3. Showcase a recommended approach for creating other similar configuration packages for other libraries. 1. It demonstrates a standardization for creating and organizing Hydra configuration classes for the ML/torch community and beyond. - - If someone wants to create their own `hydra-` repo, `hydra-torch` points towards an upper bound on what it should look like. 2. It unifies a collection of classes across projects in one place, ensuring robust testing, version compatibility, and PyPI distributed packaging for each. - We see many hydra users reimplementing these classes (and not tracking APIs of the configured projects). `hydra-torch` factors this code out. 3. It provides best practices and guidance on how to organize code and utilize Hydra for research, production, and other innovative use cases. ##### Terminology for this overview: - The overall effort and repo is named:`hydra-torch` -- **Library:** The library being configured +- **Library:** The library being configured. - e.g. `torch`, `torchvision` -- **Project:** The project corresponding to the library being configured +- **Project:** The project corresponding to the library being configured. - e.g. `hydra-configs-torch`, `hydra-configs-torchvision` -- **Package:** The installable package containing the configs - - here, also `hydra-configs-torch`, `hydra-configs-torchvision`. find definition in the project dir's `setup.py`. +- **Package:** The installable package containing the configs. Usually corresponding to a specific project. + - here, also `hydra-configs-torch`, `hydra-configs-torchvision`. Find the package definition in the project dir's `setup.py`. @@ -31,14 +35,12 @@ The `hydra-torch` repo serves three purposes. Each `hydra-configs-` defines its own package corresponding to a project it provides classes for. That is, `hydra-configs-torch` contains a package (with its own `setup.py`) which provides classes for `torch`. -[`hydra-configs-projects.txt`](hydra-configs-projects.txt) specifies a list of projects this repo maintains. When adding a project, please update this file. - ### Packaging This repo maintains multiple packages. An important area of consideration for these packages is to organize them such that they are compatible with each other and have an 'expected', intuitive structure. The easiest way to do this is to enforce a standard for all Hydra configs packages to follow. -Our approach makes use of [Native Namespace Packages](https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages). This allows all packages to share the top level namespace `hydra_configs`. The only real requirement for this package format is to ensure there is no `__init__.py` located in the namespace folder. +Our approach makes use of [Native Namespace Packages](https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages). This allows all packages to share the top level namespace `hydra_configs`. The only real requirement for this package format is to ensure there is **no** `__init__.py` located in the namespace folder. The format for folder structure per project is always: ``` @@ -55,40 +57,17 @@ The beauty of this approach is that users can be sure the importing idiom is rel ```python from hydra_configs..path.to.module import Conf ``` +For example: +```python +from hydra_configs.torch.optim import AdamConf +``` + This will retain compatibility even with repositories that define configs for hydra outside of the `hydra-torch` repo as long as they follow this pattern. ##### Metapackage -For convenience, we also provide a metapackage called `hydra-torch`. This is defined at the top level of the repo in `setup.py`. Effectively, all this means is that a user can either install each project-specific package as needed or they can obtain them all in one shot by simply installing this singular metapackage. - -### Configen - -We are actively developing the tool, [Configen](https://github.com/facebookresearch/hydra/tree/master/tools/configen) to automatically create config classes for Hydra. Much of the work for `hydra-torch` has helped prototype this workflow and it is still rapidly evolving. - -Currently, the workflow looks like the following: -0. Ensure the most recent configen is installed from master. -1. Edit `configen/conf//configen.yaml`, listing the module and its classes from the project library to be configured. - - e.g. in `/configen/conf/torchvision/configen.yaml`: - ```yaml - modules: - - name: torchvision.datasets.mnist # module with classes to gen for - # mnist datasets - classes: # list of classes to gen for - - MNIST - - FashionMNIST - - KMNIST - - EMNIST - - QMNIST - - ``` -2. In the corresponding project directory, `hydra-configs-`, run the command `configen --config-dir ../configen/conf/`. -3. If generation is successful, the configs should be located in: - - `/hydra-configs-/hydra_configs/path/to/module`. - - for our above example, you should see `MNISTConf, ..., QMNISTConf` in the module, `mnist.py` located in: - `hydra-configs-torchvision/hydra_configs/torchvision/datasets` - ->**Note:** This process is still under development. As we encounter blocking factors, we do the appropriate development on Configen to mitigate future issues. +For convenience, we also provide a metapackage called `hydra-torch`. This is defined at the top level of the repo in `setup.py`. Effectively, all this means is that a user can either install each project-specific package as needed or they can obtain them all in one shot by installing only the metapackage. ### Testing From ed71eecc778d0220417ca0cd4330c0b9da718acb Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Mon, 11 Jan 2021 10:12:34 -0800 Subject: [PATCH 05/12] Update OVERVIEW.md --- OVERVIEW.md | 27 ++++++++++++--------------- 1 file changed, 12 insertions(+), 15 deletions(-) diff --git a/OVERVIEW.md b/OVERVIEW.md index ca1f166..60d1030 100644 --- a/OVERVIEW.md +++ b/OVERVIEW.md @@ -1,17 +1,12 @@ # Overview -The audience for this library is Hydra users using PyTorch and related libraries (torchvision, ...) +`hydra-torch`'s audience: Hydra users utilizing PyTorch and related libraries (torchvision, ...). The goals of this repository are: -1. Provide a maintained and tested implementation of config classes that can be used instantiate and configure various classes from the supported projects. -2. Provide examples and tutorials demonstrating best practices for using Hydra to configure PyTorch deep learning applications. +1. Provide a maintained and tested implementation of config classes that can be used to instantiate and configure classes belonging to the supported projects. +2. Provide examples and tutorials demonstrating best practices for using Hydra to configure PyTorch. 3. Showcase a recommended approach for creating other similar configuration packages for other libraries. -1. It demonstrates a standardization for creating and organizing Hydra configuration classes for the ML/torch community and beyond. -2. It unifies a collection of classes across projects in one place, ensuring robust testing, version compatibility, and PyPI distributed packaging for each. - - We see many hydra users reimplementing these classes (and not tracking APIs of the configured projects). `hydra-torch` factors this code out. -3. It provides best practices and guidance on how to organize code and utilize Hydra for research, production, and other innovative use cases. - ##### Terminology for this overview: - The overall effort and repo is named:`hydra-torch` - **Library:** The library being configured. @@ -62,8 +57,7 @@ For example: from hydra_configs.torch.optim import AdamConf ``` - -This will retain compatibility even with repositories that define configs for hydra outside of the `hydra-torch` repo as long as they follow this pattern. +This approach ensures interoperability with repositories that define configs for hydra outside of the `hydra-torch` repo as long as they follow this pattern. ##### Metapackage @@ -78,7 +72,7 @@ Tests are run through pytest. The CI flow is currently: `CircleCI` -> `nox` -> ` ### Versions -##### Regarding Libraries +#### Regarding Libraries Given the rapid development of libraries in the torch ecosystem, it is likely that there will be a need for multiple simultaneous version specific `hydra-configs-` per library. This is especially relevant as API and type annotation is updated moving forward for a particular library. @@ -89,14 +83,17 @@ For each library, there will be releases for each `MINOR` version starting with ###### Package/Branch Names: The version for the `hydra-configs-` package will be `MAJOR.MINOR.X` where `MAJOR.MINOR` tracks the library versions and `.X` is reserved for revisions of the configs for that particular `MAJOR.MINOR` should they be required. -e.g. `hydra-configs-torch==1.6.0` corresponds to `torch==1.6` and if updates are needed for patches from either end, only the `PATCH` version will be updated -> `hydra-config-torch==1.6.1` - -#TODO: write out exact tag format once decided upon +> e.g. `hydra-configs-torch==1.6.0` corresponds to `torch==1.6` and if updates are needed for patches from either end, only the `PATCH` version will be updated -> `hydra-config-torch==1.6.1` -##### Regarding Hydra +#### Regarding Hydra Many of the features that make using configs compelling are supported in Hydra > 1.1. For this reason, we are aiming support starting at 1.1 . Using an older version of Hydra is not advised, but may still work fine in less complex cases. ### Releases Releases will be on a per-project basis for specific versions of a project. Releases should be tied to a specific branch/tag in the repo that passes all tests via CI. We release via PyPI. + +#TODO: write out exact tag format once decided upon + +### Contributing +We encourage contribution from all members of the community! This repo is especially ammenable to community contribution as it covers a wide breadth and can be developed in a distributed fashion. Please see [`CONTRIBUTING.md`](CONTRIBUTING.md) to get started! From 13bd9a1ee72462c0d74518998df5de61deac759c Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Mon, 11 Jan 2021 10:13:03 -0800 Subject: [PATCH 06/12] Update OVERVIEW.md --- OVERVIEW.md | 1 + 1 file changed, 1 insertion(+) diff --git a/OVERVIEW.md b/OVERVIEW.md index 60d1030..8770466 100644 --- a/OVERVIEW.md +++ b/OVERVIEW.md @@ -1,6 +1,7 @@ # Overview `hydra-torch`'s audience: Hydra users utilizing PyTorch and related libraries (torchvision, ...). + The goals of this repository are: 1. Provide a maintained and tested implementation of config classes that can be used to instantiate and configure classes belonging to the supported projects. From ed022432df0ade27b255c2950c7dbbb49095f4a2 Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Mon, 11 Jan 2021 10:15:02 -0800 Subject: [PATCH 07/12] Update OVERVIEW.md --- OVERVIEW.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/OVERVIEW.md b/OVERVIEW.md index 8770466..05f8a11 100644 --- a/OVERVIEW.md +++ b/OVERVIEW.md @@ -1,8 +1,8 @@ # Overview -`hydra-torch`'s audience: Hydra users utilizing PyTorch and related libraries (torchvision, ...). +**`hydra-torch` audience:** Hydra users utilizing PyTorch and related libraries (torchvision, ...). -The goals of this repository are: +**`hydra-torch` goals:** 1. Provide a maintained and tested implementation of config classes that can be used to instantiate and configure classes belonging to the supported projects. 2. Provide examples and tutorials demonstrating best practices for using Hydra to configure PyTorch. From 0342b2147cc12e4fcc0376ab10d540f62dcc60da Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Mon, 11 Jan 2021 10:15:58 -0800 Subject: [PATCH 08/12] Update OVERVIEW.md --- OVERVIEW.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/OVERVIEW.md b/OVERVIEW.md index 05f8a11..1f5133d 100644 --- a/OVERVIEW.md +++ b/OVERVIEW.md @@ -8,7 +8,7 @@ 2. Provide examples and tutorials demonstrating best practices for using Hydra to configure PyTorch. 3. Showcase a recommended approach for creating other similar configuration packages for other libraries. -##### Terminology for this overview: +#### Terminology for this overview: - The overall effort and repo is named:`hydra-torch` - **Library:** The library being configured. - e.g. `torch`, `torchvision` From d45b47713d15d795aafa82a19035d5690a67f149 Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Mon, 11 Jan 2021 10:19:25 -0800 Subject: [PATCH 09/12] Update OVERVIEW.md --- OVERVIEW.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/OVERVIEW.md b/OVERVIEW.md index 1f5133d..4caf23f 100644 --- a/OVERVIEW.md +++ b/OVERVIEW.md @@ -84,7 +84,7 @@ For each library, there will be releases for each `MINOR` version starting with ###### Package/Branch Names: The version for the `hydra-configs-` package will be `MAJOR.MINOR.X` where `MAJOR.MINOR` tracks the library versions and `.X` is reserved for revisions of the configs for that particular `MAJOR.MINOR` should they be required. -> e.g. `hydra-configs-torch==1.6.0` corresponds to `torch==1.6` and if updates are needed for patches from either end, only the `PATCH` version will be updated -> `hydra-config-torch==1.6.1` +> e.g. `hydra-configs-torch==1.6.0` corresponds to `torch==1.6` and if updates are needed in response to patches for `hydra` or ``, only the `PATCH` version will be incremented -> `hydra-config-torch==1.6.1` #### Regarding Hydra From d0ac873288a71d8c319ab0ba067e92516fcc1528 Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Mon, 11 Jan 2021 11:06:55 -0800 Subject: [PATCH 10/12] Update CONTRIBUTING.md --- CONTRIBUTING.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a8b3e4e..6ad0f34 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -11,12 +11,12 @@ There are 3 main ways to contribute starting with the most straightfoward option 1. **Filing Issues against Configs:** Noticing a bug that you believe is a problem with the generated config? Please file an issue for the offending config class stating clearly your versions of `hydra-core` and the library being configured (e.g. `torch`). If you believe it is actually a problem with hydra or torch, file the issue in their respective repositories. -> **NOTE:** Please include the manual tag for the project and library version in your issue title. If, for example, there is an issue with `AdamConf` for `torch1.6` which comes from `hydra-configs-torch` v1.6.1, your issue name might look like: +> **NOTE:** Please include a **manual tag** for the project and library version in your issue title. If, for example, there is an issue with `AdamConf` for `torch1.6` which comes from `hydra-configs-torch` v1.6.1, your issue name might look like: **[hydra-configs-torch][1.6.1] AdamConf does not instantiate**. 2. **Example Usecase / Tutorial:** The `hydra-torch` repository not only hosts config packages like `hydra-configs-torch`,`hydra-configs-torchvision`, etc., it also aggregates examples of how to structure projects utilizing hydra and torch. The bar is high for examples that get included, but we will work together as a community to hone in on what the best practices are. Ideally, example usecases will come along with an incremental tutorial that introduces a user to the methodology being followed. If you have an interesting way to use hydra/torch, write up an example and show us in a draft PR! -3. **Maintaining Configs:** After the initial (considerable) setup effort, the goal of this repository is to be self-sustaining meaning code can be autogrenerated when APIs change. In order to contribute to a particular package like `hydra-configs-torch`, please see the [**Projects**](https://github.com/pytorch/hydra-torch/projects) tab to identify outstanding issues per project and configured library version. Before contributing at this level, please familiarize with [configen](https://github.com/facebookresearch/hydra/tree/master/tools/configen). We are actively developing this tool as well. +3. **Maintaining Configs:** After the initial (considerable) setup effort, the goal of this repository is to be self-sustaining meaning code can be autogrenerated when APIs change. In order to contribute to a particular package like `hydra-configs-torch`, please see the [**Projects**](https://github.com/pytorch/hydra-torch/projects) tab to identify outstanding issues per project and configured library version. Before contributing at this level, please familiarize with [configen](https://github.com/facebookresearch/hydra/tree/master/tools/configen). We are actively developing this tool alongside this repository. ## Linting / Formatting Please download the formatting / linting requirements: `pip install -r requirements/dev.txt`. @@ -32,7 +32,7 @@ Currently, the workflow looks like the following: 1. Ensure the most recent configen is installed from master. - `pip install git+https://github.com/facebookresearch/hydra/#subdirectory=tools/configen` 2. Edit `configen/conf//configen.yaml`, listing the module and its classes from the project library to be configured. - - e.g. in `/configen/conf/torchvision/configen.yaml`: + - e.g. in `/configen/conf/torchvision.yaml`: ```yaml modules: - name: torchvision.datasets.mnist # module with classes to gen for @@ -45,7 +45,7 @@ Currently, the workflow looks like the following: - QMNIST ``` -3. In the corresponding project directory, `hydra-configs-`, run the command `configen --config-dir ../configen/conf/`. +3. In the corresponding project directory, `hydra-configs-`, run the command `configen --config-dir ../configen/conf/ --config-name .yaml` . 4. If generation is successful, the configs should be located in: - `/hydra-configs-/hydra_configs/path/to/module`. - for our above example, you should see `MNISTConf, ..., QMNISTConf` in the module, `mnist.py` located in: From f92fcab4fc7c4d3ae8ee1939765ac4c4bdd588cc Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Mon, 11 Jan 2021 11:08:33 -0800 Subject: [PATCH 11/12] Update CONTRIBUTING.md --- CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6ad0f34..cf3b7f5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -11,8 +11,8 @@ There are 3 main ways to contribute starting with the most straightfoward option 1. **Filing Issues against Configs:** Noticing a bug that you believe is a problem with the generated config? Please file an issue for the offending config class stating clearly your versions of `hydra-core` and the library being configured (e.g. `torch`). If you believe it is actually a problem with hydra or torch, file the issue in their respective repositories. -> **NOTE:** Please include a **manual tag** for the project and library version in your issue title. If, for example, there is an issue with `AdamConf` for `torch1.6` which comes from `hydra-configs-torch` v1.6.1, your issue name might look like: -**[hydra-configs-torch][1.6.1] AdamConf does not instantiate**. +> **NOTE:** Please include a **manual tag** for the project and library version in your issue title. If, for example, there is an issue with `AdamConf` for `torch1.6` which comes from `hydra-configs-torch` v1.6.1, your issue name might look like the following: + **[hydra-configs-torch][1.6.1] AdamConf does not instantiate**. 2. **Example Usecase / Tutorial:** The `hydra-torch` repository not only hosts config packages like `hydra-configs-torch`,`hydra-configs-torchvision`, etc., it also aggregates examples of how to structure projects utilizing hydra and torch. The bar is high for examples that get included, but we will work together as a community to hone in on what the best practices are. Ideally, example usecases will come along with an incremental tutorial that introduces a user to the methodology being followed. If you have an interesting way to use hydra/torch, write up an example and show us in a draft PR! From 4eb0b718a6a060a699e28850cc36bee226f7b60c Mon Sep 17 00:00:00 2001 From: Rosario Scalise Date: Wed, 13 Jan 2021 13:09:52 -0800 Subject: [PATCH 12/12] Update CONTRIBUTING.md --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cf3b7f5..e072de2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -31,7 +31,7 @@ Currently, the workflow looks like the following: 1. Ensure the most recent configen is installed from master. - `pip install git+https://github.com/facebookresearch/hydra/#subdirectory=tools/configen` -2. Edit `configen/conf//configen.yaml`, listing the module and its classes from the project library to be configured. +2. Edit `configen/conf/.yaml`, listing the module and its classes from the project library to be configured. - e.g. in `/configen/conf/torchvision.yaml`: ```yaml modules: