Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce data vault as storage backend abstraction #6899

Merged
merged 32 commits into from
Mar 20, 2023
Merged

Introduce data vault as storage backend abstraction #6899

merged 32 commits into from
Mar 20, 2023

Commits on Mar 2, 2023

  1. Implement https remote file system

    @frcroth
    frcroth committed Mar 2, 2023
    Configuration menu
    Copy the full SHA
    6a42651 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2023

  1. Implement gcs remote file system

    @frcroth
    frcroth committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    d660449 View commit details
    Browse the repository at this point in the history

  2. Implement s3 remote file system

    @frcroth
    frcroth committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    88aed0c View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into new-filesystem

    @frcroth
    frcroth committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    f091c89 View commit details
    Browse the repository at this point in the history

  4. Rename to DataVault

    @frcroth
    frcroth committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    9e66e9b View commit details
    Browse the repository at this point in the history

  5. Add file system data vault for local datasets

    @frcroth
    frcroth committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    1b01bd9 View commit details
    Browse the repository at this point in the history

  6. Remove S3FileSystem, clean up FileSystemsHolder

    @frcroth
    frcroth committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    9941e02 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2023

  1. Fix and expand test for data vault

    @frcroth
    frcroth committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    ccec35a View commit details
    Browse the repository at this point in the history

  2. Remove FileSystemStore

    @frcroth
    frcroth committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    8f47066 View commit details
    Browse the repository at this point in the history

  3. Remove HttpsFileSystem

    @frcroth
    frcroth committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    327ab4a View commit details
    Browse the repository at this point in the history

  4. Fix some issues with GoogleCloudDataVault

    @frcroth
    frcroth committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    a948287 View commit details
    Browse the repository at this point in the history

  5. Support legacy credentials for https, support range with tryget

    @frcroth
    frcroth committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    2206a8f View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2023

  1. Rename FileSystemsHolder, use Long Range

    @frcroth
    frcroth committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    699441e View commit details
    Browse the repository at this point in the history

  2. Remove S3Utilities

    @frcroth
    frcroth committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    6118033 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2023

  1. Flatmap instead of map

    @frcroth
    frcroth committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    e0fc3d0 View commit details
    Browse the repository at this point in the history

  2. Simplify s3 credentials passing

    @frcroth
    frcroth committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    88d92b5 View commit details
    Browse the repository at this point in the history

  3. Change signature to readBytes

    @frcroth
    frcroth committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    201448c View commit details
    Browse the repository at this point in the history

  4. Remove credential in vault path

    @frcroth
    frcroth committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    2fb78c8 View commit details
    Browse the repository at this point in the history

  5. Remove key argument in readBytes, use underlying path instead

    @frcroth
    frcroth committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    9ba4b89 View commit details
    Browse the repository at this point in the history

  6. Adapt interface for FileSystemVaultPath

    @frcroth
    frcroth committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    32d3b8c View commit details
    Browse the repository at this point in the history

  7. FIx s3 data vault with new keyless interface

    @frcroth
    frcroth committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    2aaf5db View commit details
    Browse the repository at this point in the history

  8. Cache vaults and not paths

    @frcroth
    frcroth committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    ecd9a16 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'master' into new-filesystem

    @frcroth
    frcroth authored Mar 14, 2023
    Configuration menu
    Copy the full SHA
    5352bb6 View commit details
    Browse the repository at this point in the history

  10. Fix DataVaultTestSuite

    @frcroth
    frcroth committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    ce31bd4 View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2023

  1. Fix GCS range reading when not starting at 0

    @frcroth
    frcroth committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    c236008 View commit details
    Browse the repository at this point in the history

  2. Use slash for vault paths in a better way

    @frcroth
    frcroth committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    943ce67 View commit details
    Browse the repository at this point in the history

  3. Add test to clarify behavior with starting slash

    @frcroth
    frcroth committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    912ab90 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into new-filesystem

    @frcroth
    frcroth authored Mar 20, 2023
    Configuration menu
    Copy the full SHA
    51d2d6d View commit details
    Browse the repository at this point in the history

  5. Add another test to describe behavior

    @frcroth
    frcroth committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    3e71046 View commit details
    Browse the repository at this point in the history

  6. Fix handling of s3 short style uri

    @frcroth
    frcroth committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    512e664 View commit details
    Browse the repository at this point in the history

  7. Remove comment relating to old usage

    @frcroth
    frcroth committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    5dd3172 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'master' into new-filesystem

    @frcroth
    frcroth authored Mar 20, 2023
    Configuration menu
    Copy the full SHA
    c73dcf2 View commit details
    Browse the repository at this point in the history