Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem: processing workflow has separate code paths for filesystem and minio watchers #867

Open
djjuhasz opened this issue Feb 22, 2024 · 1 comment

Comments

@djjuhasz
Copy link
Collaborator

djjuhasz commented Feb 22, 2024

Describe the problem

The processing workflow and bundle activity use conditional branches to manage differences between the filesystem watcher and minio watcher implementations. The branching logic is hard to understand, requires some code duplication and makes the assumption that only a filesystem watcher can provide a "blob" that is a directory.

Possible solutions

The differences between the two watchers could be managed by moving the difference in implementation for copying the blob data to a local "processing" directory into their individual implementations instead of requiring branching logic (see "Avoiding conditionals by obeying the Liskov Substitution Principle" - https://sandimetz.com/99bottles).

Additional context

@djjuhasz
Copy link
Collaborator Author

djjuhasz commented Feb 22, 2024

This is complicated by the fact that the current watcher.Service.Download() method takes an io.Writer parameter as the destination to write the byte stream. An io.Writer stream works for a single file, as we expect from a minioWatcher, but doesn't work for a filesystem directory as we might receive from a filesystemWatcher.

djjuhasz added a commit that referenced this issue Feb 23, 2024
Fixes #867

- Update existing watcher Download() method and interface signatures to
  take a destination filepath (string) instead of an io.Writer stream
- Add a `filewatcher.Download()` method
- Create a `minio.Download()` method and move implementation specific
  code from `serviceImpl.Download()` to it
- Update mocks
djjuhasz added a commit that referenced this issue Feb 23, 2024
Refs #867

- Update existing watcher Download() method and interface signatures to
  take a destination filepath (string) instead of an io.Writer stream
- Add a `filewatcher.Download()` method
- Create a `minio.Download()` method and move implementation specific
  code from `serviceImpl.Download()` to it
- Update mocks
djjuhasz added a commit that referenced this issue Feb 23, 2024
Refs #867

- Update existing watcher Download() method and interface signatures to
  take a destination filepath (string) instead of an io.Writer stream
- Add a `filewatcher.Download()` method
- Create a `minio.Download()` method and move implementation specific
  code from `serviceImpl.Download()` to it
- Update mocks
djjuhasz added a commit that referenced this issue Feb 28, 2024
Refs #867

- Update existing watcher Download() method and interface signatures to
  take a destination filepath (string) instead of an io.Writer stream
- Add a `filewatcher.Download()` method
- Create a `minio.Download()` method and move implementation specific
  code from `serviceImpl.Download()` to it
- Update mocks
djjuhasz added a commit that referenced this issue Feb 29, 2024
Refs #867

- Update existing watcher Download() method and interface signatures to
  take a destination filepath (string) instead of an io.Writer stream
- Add a `filewatcher.Download()` method
- Create a `minio.Download()` method and move implementation specific
  code from `serviceImpl.Download()` to it
- Update mocks
djjuhasz added a commit that referenced this issue Feb 29, 2024
Refs #867

- Update existing watcher Download() method and interface signatures to
  take a destination filepath (string) instead of an io.Writer stream
- Add a `filewatcher.Download()` method
- Create a `minio.Download()` method and move implementation specific
  code from `serviceImpl.Download()` to it
- Update mocks
djjuhasz added a commit that referenced this issue Mar 1, 2024
Refs #867

- Update the existing watcher `Download()` method and interface
  signatures to take a destination filepath (string) instead of an
  `io.Writer` stream
- Add a `filewatcher.Download()` method
- Create a `minio.Download()` method and move implementation specific
  code from `serviceImpl.Download()` to it
- Update mocks
djjuhasz added a commit that referenced this issue Mar 1, 2024
Refs #867

- Update the existing watcher `Download()` method and interface
  signatures to take a destination filepath (string) instead of an
  `io.Writer` stream
- Add a `filewatcher.Download()` method
- Create a `minio.Download()` method and move implementation specific
  code from `serviceImpl.Download()` to it
- Add `MinioConfig.URL` field to set bucket config by URL for testing
- Add unit tests
- Update mocks
djjuhasz added a commit that referenced this issue Mar 1, 2024
Refs #867

- Update the existing watcher `Download()` method and interface
  signatures to take a destination filepath (string) instead of an
  `io.Writer` stream
- Add a `filewatcher.Download()` method
- Create a `minio.Download()` method and move implementation specific
  code from `serviceImpl.Download()` to it
- Add `MinioConfig.URL` field to set bucket config by URL for testing
- Add unit tests
- Update mocks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant