Skip to content

Commit

Permalink
Formatting
Browse files Browse the repository at this point in the history
Signed-off-by: pryce-turner <[email protected]>
  • Loading branch information
pryce-turner committed Jul 31, 2024
1 parent 3dc30a9 commit 8419de4
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 7 deletions.
14 changes: 8 additions & 6 deletions flytekit/core/data_persistence.py
Original file line number Diff line number Diff line change
Expand Up @@ -461,12 +461,14 @@ def generate_new_custom_path(
stem: typing.Optional[str] = None,
) -> str:
"""
Generates a new path with the raw output prefix and a random string appended to it. Optionally, you can provide
an alternate prefix and a stem. If stem is provided, it will be appended to the path instead of a random string. If alt is provided, it
will replace the first part of the output prefix, e.g. the S3 or GCS bucket.
If wanting to write to a non-random prefix in a non-default S3 bucket, this can be called with
alt="my-alt-bucket" and stem="my-stem" to generate a path like s3://my-alt-bucket/default-prefix-part/my-stem
Generates a new path with the raw output prefix and a random string appended to it.
Optionally, you can provide an alternate prefix and a stem. If stem is provided, it
will be appended to the path instead of a random string. If alt is provided, it will
replace the first part of the output prefix, e.g. the S3 or GCS bucket.
If wanting to write to a non-random prefix in a non-default S3 bucket, this can be
called with alt="my-alt-bucket" and stem="my-stem" to generate a path like
s3://my-alt-bucket/default-prefix-part/my-stem
:param fs: The filesystem to use. If None, the context's raw output filesystem is used.
:param alt: An alternate first member of the prefix to use instead of the default.
Expand Down
5 changes: 4 additions & 1 deletion tests/flytekit/unit/core/test_data_persistence.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,10 @@ def test_generate_new_custom_path():
Test that a new path given alternate bucket and name is generated correctly
"""
random_dir = tempfile.mkdtemp()
fs = FileAccessProvider(local_sandbox_dir=random_dir, raw_output_prefix="s3://my-default-bucket/my-default-prefix/")
fs = FileAccessProvider(
local_sandbox_dir=random_dir,
raw_output_prefix="s3://my-default-bucket/my-default-prefix/"
)
np = fs.generate_new_custom_path(alt="foo-bucket", stem="bar.txt")
assert np == "s3://foo-bucket/my-default-prefix/bar.txt"

Expand Down

0 comments on commit 8419de4

Please sign in to comment.