Skip to content

Commit

Permalink
docs update (facebookresearch#1595)
Browse files Browse the repository at this point in the history
  • Loading branch information
omry authored May 4, 2021
1 parent 713d5ea commit 43cc401
Show file tree
Hide file tree
Showing 5 changed files with 212 additions and 88 deletions.
1 change: 0 additions & 1 deletion website/docs/experimental/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ id: intro
title: Introduction
sidebar_label: Introduction
---
## Introduction
Experimental features are new features in Hydra that are considered experimental because their API may have not yet
stabilized.

Expand Down
110 changes: 100 additions & 10 deletions website/docs/upgrades/1.0_to_1.1/automatic_schema_matching.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,37 @@ title: Automatic schema-matching
hide_title: true
---

In Hydra 1.0, when config file is loaded, if a config with a matching name and group is present in the `ConfigStore`,
In Hydra 1.0, when a config file is loaded, if a config with a matching name and group is present in the `ConfigStore`,
it is used as the schema for the newly loaded config.

There are several problems with this approach:

- **Inflexible**: This approach can only be used when a schema should validate a single config file.
It does not work if you want to use the same schema to validate multiple config files.
- **Surprising**: This hidden behavior can be surprising. There is no way to tell this is going to happen when looking at a given
- **Unexpected**: This behavior can be unexpected. There is no way to tell this is going to happen when looking at a given
config file.

Hydra 1.1 deprecates this behavior in favor of an explicit config extension via the Defaults List.
This upgrade page aims to provide the TLDR. It is highly recommended that you read the following pages:
This upgrade page aims to provide a summary of the required changes. It is highly recommended that you read the following pages:
- [Background: The Defaults List](../../advanced/defaults_list.md)
- [Background: Extending configs](../../patterns/extending_configs.md)
- [Tutorial: Structured config schema](../../tutorials/structured_config/5_schema.md)

### Migration
The migration involves two steps for each config file.
1. Store use a different name when storing the schema into the config store. Common choices:

## Migration
Before the upgrade, you have two different configs with the same name (a config file, and a Structured Config in the `ConfigStore`).
You need to rename one of them. Depending on the circumstances and your preference you may rename one or the other.
- If you control both configs, you can rename either of them.
- If you only control the config file, rename it.

### Option 1: rename the Structured Config
This option is less disruptive. Use it if you control the Structured Config.
1. Use a different name when storing the schema into the Config Store. Common choices:
- `base_` prefix, e.g. `base_mysql`.
- `_schema` suffix, e.g. `mysql_schema`.
2. Add the schema to the Defaults List in the config file.

2. Add the schema to the Defaults List of the extending config file.

<details><summary>Click to show an example</summary>

#### Hydra 1.0
<div className="row">
Expand All @@ -46,7 +54,7 @@ port: 3306
</div>
<div className="col col--6">
```python title="Schema for db/mysql.yaml"
```python title="db/mysql schema in the ConfigStore"
@dataclass
class MySQLConfig:
host: str
Expand Down Expand Up @@ -78,7 +86,7 @@ port: 3306
</div>
<div className="col col--6">
```python title="Schema for db/mysql.yaml" {8}
```python title="db/mysql schema in the ConfigStore" {8}
@dataclass
class MySQLConfig:
host: str
Expand All @@ -92,3 +100,85 @@ cs.store(group="db",
</div>
</div>

</details>

### Option 2: rename the config file
This option is a bit more disruptive. Use it if you only control the config file.
1. Rename the config file. Common choices are `custom_` or `my_` prefix, e.g. `custom_mysql.yaml`. You can also use a domain specific name like `prod_mysql.yaml`.
2. Add the schema to the Defaults List of the extending config file.
3. Update references to the config name accordingly, e.g. on the command-line `db=mysql` would become `db=custom_mysql`, and in a defaults list `db: mysql` would become `db: custom_mysql`.


<details><summary>Click to show an example</summary>

#### Hydra 1.0
<div className="row">
<div className="col col--6">

```yaml title="db/mysql.yaml"
# @package _group_
host: localhost
port: 3306
```
```yaml title="config.yaml"
defaults:
- db: mysql
```
</div>
<div className="col col--6">
```python title="db/mysql schema in the ConfigStore"
@dataclass
class MySQLConfig:
host: str
port: int

cs = ConfigStore.instance()
cs.store(group="db",
name="mysql",
node=MySQLConfig)

```
</div>
</div>

#### Hydra 1.1
Rename `db/mysql.yaml` to `db/custom_mysql.yaml` and explicitly add the schema to the Defaults List.
<div className="row">

<div className="col col--6">

```yaml title="db/custom_mysql.yaml" {1,2}
defaults:
- mysql

host: localhost
port: 3306
```
```yaml title="config.yaml" {2}
defaults:
- db: custom_mysql
```
</div>
<div className="col col--6">
```python title="db/mysql schema in the ConfigStore"





NO CHANGES






```
</div>
</div>

Don't forget to also update your command line overrides from `db=mysql` to `db=custom_mysql`.
</details>
15 changes: 7 additions & 8 deletions website/docs/upgrades/1.0_to_1.1/hydra_main_config_path.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,24 @@ title: Changes to @hydra.main() and hydra.initialize()

Prior to Hydra 1.1, **@hydra.main()** and **hydra.initialize()** default `config path` was the directory containing the Python app (calling **@hydra.main()** or **hydra.initialize()**).

This can cause surprising behavior, such as:
This can cause unexpected behavior:
- Sibling directories are interpreted as config groups, which can lead to surprising results (See [#1533](https://github.com/facebookresearch/hydra/issues/1533)).
- The directory added automatically can have many files/directories - which will cause **--help** to be very slow as it's scanning for all config groups/config files (See [#759](https://github.com/facebookresearch/hydra/issues/759)).
- The subtree added automatically can have many files/directories - which will cause **--help** to be very slow as it's scanning for all config groups/config files (See [#759](https://github.com/facebookresearch/hydra/issues/759)).

To address these issues, Hydra 1.1 issues a warning if the config_path is not specified.
Your options are as follows:

### Dedicated config directory
Specify a directory like "conf" to use a dedicated config directory relative to the application.
Recommended for applications with config files.
For applications with config files, specify a directory like "conf" to use a dedicated config directory relative to the application.
```python
@hydra.main(config_path="conf")
# or:
hydra.initialize(config_path="conf")
```

### No config directory
Do not add any directory to the config searchpath.
Recommended if you do not want to use configs files defined next to your Python script (Typically applications using only Structured Configs).
For applications that do not define config files next to the Python script (typically applications using only Structured Configs), it is recommended that
you pass `None` as the config_path, indicating that no directory should be added to the config search path.
This will become the default in Hydra 1.2.
```python
@hydra.main(config_path=None)
Expand All @@ -33,8 +32,8 @@ hydra.initialize(config_path=None)

### Using the application directory
Use the directory/module of the Python script.
This was the default behavior up Hydra 1.0.
This is not recommended as it can cause surprising behavior.
This was the default behavior up to Hydra 1.0.
This is not recommended as it can cause the surprising behavior outlined above.

```python
@hydra.main(config_path=".")
Expand Down
35 changes: 35 additions & 0 deletions website/docs/upgrades/intro.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
id: intro
title: Introduction
sidebar_label: Introduction
---

Upgrading to a new Hydra version is usually an easy process.


:::info NOTE
Hydra versioning has only major versions and patch versions. A bump of the first two version digits is considered a major release.
A major release may have multiple followup patch releases that will fix bugs without introducing new functionality.
:::


## Major version upgrades
Hydra will typically provide helpful warnings about required changes, sometimes pointing to an upgrade page that provides more details about the required changes.

For a smooth upgrade experience, please follow these simple rules:
- **Upgrade to the latest patch version first**.
e.g: If you are upgrading from 1.0 to 1.1, be sure to upgrade to the latest 1.0 version first (1.0.6).
- **Address ALL runtime warnings issued by Hydra.**
A warning in one version is likely to become a far less friendly error in the next major version.
- **Do not skip major versions**.
e.g: If you are upgrading from Hydra 1.0 to Hydra 1.2 - Do it by
- Upgrading from 1.0 to 1.1, addressing all the warnings.
- Upgrading from 1.1 to 1.2, addressing all the warnings.

## Patch version upgrades
Patch releases normally contains only bug fixes and are thus safe and easy to upgrade (e.g. **1.0.3** to **1.0.6**).
In rare cases, patch releases will introduce new warnings. Be sure to address them before upgrading to the next major version.

## Dev release upgrades
Development releases are subject to breaking changes without notice. Please be aware that upgrading to a new development release
is more likely to introduce some breakage. No attempt will be made to make upgrading between development releases easy.
Loading

0 comments on commit 43cc401

Please sign in to comment.