Open-source, document and explain the S3 plugin/integration #327
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Wow, that was a lot of words. I don't like the current s3index script, but I had to choose between "leave the user on their own and don't explain how to actually index samples" or "write a shitty script that is still better than nothing or doing it manually" But I'm happy to announce that this PR is almost ready! Just need to re-read the code when it's not 3 AM, fix the style issues (and bugs, if any), fix alll my grammar issues, and get nazywam to review it. |
msm-code
changed the title
4 tasks
Merged
nazywam
reviewed
Feb 10, 2023
docs/utils/s3index.md
Outdated
| @@ -0,0 +1,56 @@ | |||
| # s3index.py | |||
|
|
|||
| Can be used to index files from s3. | |||
There was a problem hiding this comment.
Suggested change
| Can be used to index files from s3. | |
| Can be used to index files from S3. |
docs/utils/s3index.md
Outdated
|
|
||
| Can be used to index files from s3. | ||
|
|
||
| This script was created to accompany the [S3 integration guide](../how-to/integrate-with-s3.md). It will download files from s3 temporarily and index |
There was a problem hiding this comment.
Suggested change
| This script was created to accompany the [S3 integration guide](../how-to/integrate-with-s3.md). It will download files from s3 temporarily and index | |
| This script was created to accompany the [S3 integration guide](../how-to/integrate-with-s3.md). It will download files from S3 temporarily and index |
docs/utils/s3index.md
Outdated
|
|
||
| Only --workdir and s3-related parameters are required: | ||
|
|
||
| ``` |
docs/utils/s3index.md
Outdated
|
|
||
| ## Example | ||
|
|
||
| Only --workdir and s3-related parameters are required: |
There was a problem hiding this comment.
Suggested change
| Only --workdir and s3-related parameters are required: | |
| Only --workdir and S3-related parameters are required: |
src/plugins/s3_plugin.py
Outdated
| response.release_conn() | ||
| return tmp.name | ||
|
|
||
| def clean(self): |
There was a problem hiding this comment.
Suggested change
| def clean(self): | |
| def clean(self) -> None: |
nazywam
approved these changes
Feb 10, 2023
docs/how-to/integrate-with-s3.md
Outdated
| looks at `9535be65f6d2f315971e53440e4e1` and completely ignores the path. | ||
| * Next, the plugin downloads that filename from a configured S3 bucket - in our | ||
| example, the plugin will get a file called `9535be65f6d2f315971e53440e4e1`. | ||
| * This file is used for the requested operation, like scanning with Yara, |
There was a problem hiding this comment.
Suggested change
| * This file is used for the requested operation, like scanning with Yara, | |
| * This file is used for the requested operation, like scanning with YARA, |
Co-authored-by: Michał Praszmo <[email protected]>
|
Thanks for a thorough review! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Your checklist for this pull request
What is the current behaviour?
No public documentation about s3 support
What is the new behaviour?
S3 support is documented. S3 plugin is open-sourced. Documentation is better.
Test plan
Literally half of the PR is a test plan guide.
Closing issues
fixes #326