-
Notifications
You must be signed in to change notification settings - Fork 545
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[redhat|policy] Provide a URL message for S3 bucket based uploads #3891
base: main
Are you sure you want to change the base?
[redhat|policy] Provide a URL message for S3 bucket based uploads #3891
Conversation
Congratulations! One of the builds has completed. 🍾 You can install the built RPMs by following these steps:
Please note that the RPMs should be used only in a testing environment. |
5e10fe9
to
e9deb52
Compare
Signed-off-by: Trevor Benson <[email protected]>
e9deb52
to
c1fe344
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Did a very quick test with non-S3 uploads and worked as expected. I'll test more thoroughly in a couple of days
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code LGTM, but I wonder if we wouldn't be better served if we instead change the logic that was generating the original problematic messaging in #3890. It would strike me as a little strange for the RH api path the be appended on to whatever endpoint/bucket path I'd otherwise be expecting.
rh_case_api = "/support/v1/cases/%s/attachments" | ||
return f"{endpoint}/{bucket}" + rh_case_api % self.case_id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's make this all f-string formatting, mixing the two syntax is a little awkward.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll adjust this accordingly once I have additional feedback to #3891 (comment)
Unless I overlooked something this function is what ends up generating both parts of the message: the missing case id; the upload to Red Hat Secure FTP.
Only included this as I think @jcastill mentioned RH potentially using an S3 bucket in the future. I presume he may be adding an RH_S3_HOST (aka an S3 Endpoint) in the work he is doing separately. I want to be sure I understand your request. When both a case id and a bucket prefix are provided the resulting sos-collector tar archive will contain the case id, but the archive itself is only found inside the provided s3 object prefix? Or are you suggesting we drop the rh_case_api altogether and if the user provides a case id without providing an object prefix the tar file, with a case ID in its name, ends up in the root of the bucket? Thanks in advance for any clarification. |
Let me answer this way, please let me know if I'm not making sense here. As an end-user creating an sos report and uploading it to an S3 location that I've defined in my config file or in the
On that last point that means I'd expect, with or without a case id, this path from #3890:
and not
If RH starts using an S3 bucket, we'd address any handling of that within the Red Hat upload methods so that it doesn't impact a user uploading to a non-RH S3 location in any way. We really need to separate upload from policy, and I'll try and spend some time on that this week and next - but for the purposes of this PR I'm mainly just concerned about the above - predictable and expected locations within a user-specified S3 destination. |
Makes sense and clarifies the case of no prefix with a case id, which will result in the path: I'll make the adjustments and force push a new commit for review. |
I thought providing an sos/sos/policies/distros/__init__.py Lines 547 to 567 in ace5715
The upload_s3 method doesn't rely on upload_url currently, so get_upload_url returning a string w/ case_id appended shows the Attempting upload to correctly, but its dropped by the upload process. While appending the case_id to the object prefix seemed a simple fix to this I found on a freshly installed rhel 8.9 system get_upload_url() method of RHELPolicy is called multiple times, or recursively. This lead to the case_id being appended more than once when appending it directly to the s3 object prefix instead of the upload_url.
I'm going to convert this PR to a draft while determining how best to implement the requests and if aligning s3 more closely to the other upload protocols logic is possible. |
Please place an 'X' inside each '[]' to confirm you adhere to our Contributor Guidelines
Closes #3890