Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue#7 path refactoring #8

Merged
merged 11 commits into from
Nov 25, 2022
Merged

Issue#7 path refactoring #8

merged 11 commits into from
Nov 25, 2022

Conversation

bruno-f-cruz
Copy link
Collaborator

Closes #7
Implements a more general logic to open files from AWS by always calling the "open" function. If the root folder is remote it will overload the open function from s3fs. E.g.:

from s3fs.core import S3FileSystem
s3fs = S3FileSystem()
s3fs.open(remote_path)

All current streams were updated to implement this change.

commit b8d1796
Merge: 5ad7394 ac671ed
Author: brunocruz <[email protected]>
Date:   Wed Nov 23 17:48:44 2022 +0000

    Merge pull request #6 from emotional-cities/s3b-compliant

    Enable data loading from AWS server

commit ac671ed
Author: bruno-f-cruz <[email protected]>
Date:   Wed Nov 23 17:15:56 2022 +0000

    updated acceleromter IO to be s3f compliant

commit 1487d50
Author: bruno-f-cruz <[email protected]>
Date:   Wed Nov 23 17:03:58 2022 +0000

    Remove unused dependency

commit f03170f
Author: bruno-f-cruz <[email protected]>
Date:   Wed Nov 23 17:02:29 2022 +0000

    updated empatica IO to be s3f compliant

commit 61e7a60
Author: bruno-f-cruz <[email protected]>
Date:   Wed Nov 23 14:10:44 2022 +0000

    updated microphone data to be s3f compliant

commit 3477fd3
Author: bruno-f-cruz <[email protected]>
Date:   Wed Nov 23 14:10:09 2022 +0000

    fix init

commit 3baaca3
Author: bruno-f-cruz <[email protected]>
Date:   Wed Nov 23 13:51:20 2022 +0000

    updated ubx to be s3f compliant

commit cd42217
Author: bruno-f-cruz <[email protected]>
Date:   Wed Nov 23 11:14:27 2022 +0000

    Enable harp stream loading from remote s3f

commit b4889d8
Author: bruno-f-cruz <[email protected]>
Date:   Wed Nov 23 09:47:15 2022 +0000

    Ensure the automatic conversion from input str
    add qol shortcut methods

commit 8a3bb6e
Author: bruno-f-cruz <[email protected]>
Date:   Tue Nov 22 14:47:24 2022 +0000

    Initial sketch of Path class to handle s3fs remote

commit 8ac20f6
Author: bruno-f-cruz <[email protected]>
Date:   Tue Nov 22 14:19:27 2022 +0000

    Correct case-sensitive default paths

commit 8ce683e
Author: bruno-f-cruz <[email protected]>
Date:   Tue Nov 22 11:47:41 2022 +0000

    Improve path generation and file io error handling
@bruno-f-cruz bruno-f-cruz linked an issue Nov 25, 2022 that may be closed by this pull request
@bruno-f-cruz bruno-f-cruz merged commit 579b762 into main Nov 25, 2022
@bruno-f-cruz bruno-f-cruz deleted the issue#7-path-refactoring branch November 25, 2022 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handle S3fs by using "open" syntax instead of using complex paths
1 participant