Skip to content

Commit

Permalink
Merge branch 'main' into feature/3932-catalog-from-to-prototype
Browse files Browse the repository at this point in the history
  • Loading branch information
ElenaKhaustova committed Nov 14, 2024
2 parents 1b6be8e + 9464dc7 commit db77436
Show file tree
Hide file tree
Showing 8 changed files with 49 additions and 14 deletions.
8 changes: 4 additions & 4 deletions .github/workflows/no-response.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@ on:
issue_comment:
types: [created]
schedule:
# Schedule for five minutes after the hour, every hour
- cron: '5 * * * *'
# Run every day at 9am (UTC time)
- cron: '0 9 * * *'

jobs:
noResponse:
runs-on: ubuntu-latest
steps:
- uses: lee-dohm/[email protected]
with:
token: ${{ github.token }}
token: ${{ secrets.GITHUB_TOKEN }}
responseRequiredLabel: "support: needs more info"
daysUntilClose: 14
daysUntilClose: 28
closeComment: >-
This issue has been closed due to lack of information. Feel free to re-open this issue if you're facing a similar problem. Please provide as much information as possible so we can help resolve your issue.
2 changes: 2 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## Major features and improvements
* Implemented dict-like interface for `KedroDataCatalog`.
* Implemented lazy dataset initializing for `KedroDataCatalog`.
* Project dependencies on both the default template and on starter templates are now explicitly declared on the `pyproject.toml` file, allowing Kedro projects to work with project management tools like `uv`, `pdm`, and `rye`.

**Note:** ``KedroDataCatalog`` is an experimental feature and is under active development. Therefore, it is possible we'll introduce breaking changes to this class, so be mindful of that if you decide to use it already. Let us know if you have any feedback about the ``KedroDataCatalog`` or ideas for new features.

Expand Down Expand Up @@ -103,6 +104,7 @@ Many thanks to the following Kedroids for contributing PRs to this release:

## Upcoming deprecations for Kedro 0.20.0
* The utility method `get_pkg_version()` is deprecated and will be removed in Kedro 0.20.0.
* `LambdaDataset` is deprecated and will be removed in Kedro 0.20.0.

## Documentation changes
* Improved documentation for configuring dataset parameters in the data catalog
Expand Down
4 changes: 4 additions & 0 deletions docs/source/nodes_and_pipelines/run_a_pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,10 @@ Out[11]: {'v': 0.666666666666667}

We can also use IO to save outputs to a file. In this example, we define a custom `LambdaDataset` that would serialise the output to a file locally:

```{warning}
`LambdaDataset` has been deprecated and will be removed in Kedro `0.20.0`.
```

<details>
<summary><b>Click to expand</b></summary>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,20 @@ requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
requires-python = ">=3.9"
name = "{{ cookiecutter.python_package }}"
readme = "README.md"
dynamic = ["dependencies", "version"]
dynamic = ["version"]
dependencies = [
"ipython>=8.10",
"jupyterlab>=3.0",
"notebook",
"kedro~={{ cookiecutter.kedro_version }}",
"kedro-datasets[pandas-csvdataset]"
]

[project.scripts]
{{ cookiecutter.repo_name }} = "{{ cookiecutter.python_package }}.__main__:main"
"{{ cookiecutter.repo_name }}" = "{{ cookiecutter.python_package }}.__main__:main"

[project.optional-dependencies]
docs = [
Expand All @@ -30,7 +38,6 @@ dev = [
]

[tool.setuptools.dynamic]
dependencies = {file = "requirements.txt"}
version = {attr = "{{ cookiecutter.python_package }}.__version__"}

[tool.setuptools.packages.find]
Expand All @@ -41,7 +48,7 @@ namespaces = false
package_name = "{{ cookiecutter.python_package }}"
project_name = "{{ cookiecutter.project_name }}"
kedro_init_version = "{{ cookiecutter.kedro_version }}"
tools = {{ cookiecutter.tools | default('') | string | replace('\"', '\\\"') }}
tools = "{{ cookiecutter.tools | default('') | string | replace('\"', '\\\"') }}"
example_pipeline = "{{ cookiecutter.example_pipeline }}"
source_dir = "src"

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
ipython>=8.10
jupyterlab>=3.0
notebook
kedro~={{ cookiecutter.kedro_version}}
kedro~={{ cookiecutter.kedro_version }}
kedro-datasets[pandas-csvdataset]
8 changes: 8 additions & 0 deletions kedro/io/lambda_dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@

from __future__ import annotations

import warnings
from typing import Any, Callable

from kedro import KedroDeprecationWarning
from kedro.io.core import AbstractDataset, DatasetError


Expand Down Expand Up @@ -100,6 +102,12 @@ def __init__(
DatasetError: If a method is specified, but is not a Callable.
"""

warnings.warn(
"`LambdaDataset` has been deprecated and will be removed in Kedro 0.20.0.",
KedroDeprecationWarning,
)

for name, value in [
("load", load),
("save", save),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,19 @@ requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
requires-python = ">=3.9"
name = "{{ cookiecutter.python_package }}"
readme = "README.md"
dynamic = ["dependencies", "version"]
dynamic = ["version"]
dependencies = [
"ipython>=8.10",
"jupyterlab>=3.0",
"notebook",
"kedro~={{ cookiecutter.kedro_version }}"
]

[project.scripts]
{{ cookiecutter.repo_name }} = "{{ cookiecutter.python_package }}.__main__:main"
"{{ cookiecutter.repo_name }}" = "{{ cookiecutter.python_package }}.__main__:main"

[project.entry-points."kedro.hooks"]

Expand All @@ -31,9 +38,7 @@ dev = [
"ruff~=0.1.8"
]


[tool.setuptools.dynamic]
dependencies = {file = "requirements.txt"}
version = {attr = "{{ cookiecutter.python_package }}.__version__"}

[tool.setuptools.packages.find]
Expand All @@ -44,7 +49,7 @@ namespaces = false
package_name = "{{ cookiecutter.python_package }}"
project_name = "{{ cookiecutter.project_name }}"
kedro_init_version = "{{ cookiecutter.kedro_version }}"
tools = {{ cookiecutter.tools | default('') | string | replace('\"', '\\\"') }}
tools = "{{ cookiecutter.tools | default('') | string | replace('\"', '\\\"') }}"
example_pipeline = "{{ cookiecutter.example_pipeline }}"
source_dir = "src"

Expand Down
9 changes: 9 additions & 0 deletions tests/io/test_lambda_dataset.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import pytest

from kedro import KedroDeprecationWarning
from kedro.io import DatasetError, LambdaDataset


Expand Down Expand Up @@ -81,6 +82,14 @@ def test_ephemeral_attribute(mocked_dataset):
assert mocked_dataset._EPHEMERAL is False


def test_lambda_dataset_deprecation():
with pytest.warns(
KedroDeprecationWarning,
match=r"\`LambdaDataset\` has been deprecated",
):
_ = LambdaDataset(None, None)


class TestLambdaDatasetLoad:
def test_load_invocation(self, mocker):
"""Test the basic `load` method invocation"""
Expand Down

0 comments on commit db77436

Please sign in to comment.