Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change YAML version exception into a warning #2385

Merged
merged 28 commits into from
Apr 19, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
393207f
Change exception into warning, add strict_version param, and remove c…
ZanSara Apr 1, 2022
5b27352
Simplify update_json_schema
ZanSara Apr 1, 2022
5833879
Rename unstable into master
ZanSara Apr 1, 2022
3829311
Typo
ZanSara Apr 1, 2022
07454b5
Fix KeyErrors
ZanSara Apr 1, 2022
da633d5
Fix tests
ZanSara Apr 5, 2022
b0f0494
Prevent validate_config from changing the config to validate
ZanSara Apr 6, 2022
9955a09
Fix tests again
ZanSara Apr 6, 2022
9cda077
Fix generate_json_schema.py
ZanSara Apr 6, 2022
202d7a6
Update Documentation & Code Style
github-actions[bot] Apr 6, 2022
575ed86
Fix version validation and add tests
ZanSara Apr 13, 2022
c9d9993
Merge branch 'master' into tame_yaml_version_check
ZanSara Apr 13, 2022
260eed6
Fix mypy
ZanSara Apr 13, 2022
9b09f34
Update Documentation & Code Style
github-actions[bot] Apr 13, 2022
c7ad082
Rename master into ignore
ZanSara Apr 13, 2022
70ec745
Merge branch 'tame_yaml_version_check' of github.com:deepset-ai/hayst…
ZanSara Apr 13, 2022
09caeef
Complete parameter rename
ZanSara Apr 13, 2022
bf9273c
Merge branch 'master' into tame_yaml_version_check
ZanSara Apr 14, 2022
3b3d1b0
Mypy and pylint
ZanSara Apr 14, 2022
b709e4c
Merge branch 'tame_yaml_version_check' of github.com:deepset-ai/hayst…
ZanSara Apr 14, 2022
86bbc23
Update Documentation & Code Style
github-actions[bot] Apr 14, 2022
3daa7fa
mypy
ZanSara Apr 14, 2022
048266a
Update Documentation & Code Style
github-actions[bot] Apr 14, 2022
516c455
mypy
ZanSara Apr 14, 2022
fab6c86
mypy again
ZanSara Apr 14, 2022
fcaa499
Update Documentation & Code Style
github-actions[bot] Apr 14, 2022
f007936
Merge branch 'master' into tame_yaml_version_check
ZanSara Apr 19, 2022
dd274fb
Update Documentation & Code Style
github-actions[bot] Apr 19, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions .github/utils/generate_json_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,4 @@
sys.path.append(".")
from haystack.nodes._json_schema import update_json_schema

update_json_schema(
update_index=True, destination_path=Path(__file__).parent.parent.parent / "haystack" / "json-schemas"
)
update_json_schema(destination_path=Path(__file__).parent.parent.parent / "haystack" / "json-schemas")
11 changes: 7 additions & 4 deletions docs/_src/api/api/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Default value is True.
```python
@classmethod
@abstractmethod
def load_from_config(cls, pipeline_config: Dict, pipeline_name: Optional[str] = None, overwrite_with_env_variables: bool = True)
def load_from_config(cls, pipeline_config: Dict, pipeline_name: Optional[str] = None, overwrite_with_env_variables: bool = True, strict_version_check: bool = False)
```

Load Pipeline from a config dict defining the individual components and how they're tied together to form
Expand Down Expand Up @@ -132,6 +132,7 @@ Here's a sample configuration:
to change index name param for an ElasticsearchDocumentStore, an env
variable 'MYDOCSTORE_PARAMS_INDEX=documents-2021' can be set. Note that an
`_` sign must be used to specify nested hierarchical properties.
- `strict_version_check`: whether to fail in case of a version mismatch (throws a warning otherwise)

<a id="base.BasePipeline.load_from_yaml"></a>

Expand Down Expand Up @@ -561,7 +562,7 @@ Create a Graphviz visualization of the pipeline.

```python
@classmethod
def load_from_yaml(cls, path: Path, pipeline_name: Optional[str] = None, overwrite_with_env_variables: bool = True)
def load_from_yaml(cls, path: Path, pipeline_name: Optional[str] = None, overwrite_with_env_variables: bool = True, strict_version_check: bool = False)
```

Load Pipeline from a YAML file defining the individual components and how they're tied together to form
Expand Down Expand Up @@ -610,14 +611,15 @@ If the pipeline loads correctly regardless, save again the pipeline using `Pipel
to change index name param for an ElasticsearchDocumentStore, an env
variable 'MYDOCSTORE_PARAMS_INDEX=documents-2021' can be set. Note that an
`_` sign must be used to specify nested hierarchical properties.
- `strict_version_check`: whether to fail in case of a version mismatch (throws a warning otherwise)

<a id="base.Pipeline.load_from_config"></a>

#### load\_from\_config

```python
@classmethod
def load_from_config(cls, pipeline_config: Dict, pipeline_name: Optional[str] = None, overwrite_with_env_variables: bool = True)
def load_from_config(cls, pipeline_config: Dict, pipeline_name: Optional[str] = None, overwrite_with_env_variables: bool = True, strict_version_check: bool = False)
```

Load Pipeline from a config dict defining the individual components and how they're tied together to form
Expand Down Expand Up @@ -666,6 +668,7 @@ Here's a sample configuration:
to change index name param for an ElasticsearchDocumentStore, an env
variable 'MYDOCSTORE_PARAMS_INDEX=documents-2021' can be set. Note that an
`_` sign must be used to specify nested hierarchical properties.
- `strict_version_check`: whether to fail in case of a version mismatch (throws a warning otherwise).

<a id="base.Pipeline.save_to_yaml"></a>

Expand Down Expand Up @@ -769,7 +772,7 @@ def __init__(address: str = None, **kwargs)

```python
@classmethod
def load_from_yaml(cls, path: Path, pipeline_name: Optional[str] = None, overwrite_with_env_variables: bool = True, address: Optional[str] = None, **kwargs, ,)
def load_from_yaml(cls, path: Path, pipeline_name: Optional[str] = None, overwrite_with_env_variables: bool = True, address: Optional[str] = None, strict_version_check: bool = False, **kwargs, ,)
```

Load Pipeline from a YAML file defining the individual components and how they're tied together to form
Expand Down
2 changes: 1 addition & 1 deletion docs/_src/tutorials/tutorials/8.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ This tutorial will show you all the tools that Haystack provides to help you cas
!pip install git+https://github.com/deepset-ai/haystack.git#egg=farm-haystack[colab,ocr]

# For Colab/linux based machines
!wget --no-check-certificate https://dl.xpdfreader.com/xpdf-tools-linux-4.03.tar.gz
!wget --no-check-certificate https://dl.xpdfreader.com/xpdf-tools-linux-4.04.tar.gz
!tar -xvf xpdf-tools-linux-4.03.tar.gz && sudo cp xpdf-tools-linux-4.03/bin64/pdftotext /usr/local/bin

# For Macos machines
Expand Down
12 changes: 1 addition & 11 deletions haystack/json-schemas/haystack-pipeline-1.0.0.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,7 @@
"title": "Version",
"description": "Version of the Haystack Pipeline file.",
"type": "string",
"oneOf": [
{
"const": "1.0.0"
},
{
"const": "1.1.0"
},
{
"const": "1.2.0"
}
]
"const": "1.0.0"
},
"components": {
"title": "Components",
Expand Down
Loading