Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Using ignore_file in S3Bucket.put_directory skips files which should be uploaded #140

Closed
neumann-nico opened this issue Oct 21, 2022 · 0 comments · Fixed by #139
Closed

Comments

@neumann-nico
Copy link
Contributor

neumann-nico commented Oct 21, 2022

When I try to use the ignore_file option in S3Bucket.put_directory it skips files which should be included. As an example I modified one test and simply added an empty .prefectignore file. No files are uploaded then.

async def test_put_directory_respects_local_path(
    s3_bucket: S3Bucket, tmp_path: Path, aws_creds_block
):
    (tmp_path / "file1.txt").write_text("FILE 1")
    (tmp_path / "file2.txt").write_text("FILE 2")
    (tmp_path / "folder1").mkdir()
    (tmp_path / "folder1" / "file3.txt").write_text("FILE 3")
    (tmp_path / "folder1" / "file4.txt").write_text("FILE 4")
    (tmp_path / "folder1" / "folder2").mkdir()
    (tmp_path / "folder1" / "folder2" / "file5.txt").write_text("FILE 5")
    (tmp_path / ".prefectignore").write_text("") # ADDED

    uploaded_file_count = await s3_bucket.put_directory(
        local_path=str(tmp_path / "folder1"),
        ignore_file=str(tmp_path / ".prefectignore"), # ADDED
    )
    assert uploaded_file_count == 3  # uploaded_file_count equals 0 
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant