Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add additional options to control storage endpoints on import #84

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

kuzpactor
Copy link
Contributor

Similar to #43, PR adds options to control storage endpoints, but this time at the import part.

I guess it also finally closes #42.

@kuzpactor kuzpactor requested a review from a team as a code owner August 20, 2023 13:25
@kuzpactor
Copy link
Contributor Author

Also found (what seems to be) a bug in Compute ImageService.Create method: it won't accept URIs with ports, but this is exact form that is returned by endpoint resolver API.
1c850b3 works around that by throwing away the colon and the rest of the string (presumably port).

If you care, here are the details of backend error:
==> file.base (yandex-import): Uploading file testing.qcow2 to bucket kuzpactor-service/packer-import-1692542680.qcow2...
==> file.base (yandex-import): URI: "https://storage.il.nebius.cloud:443/kuzpactor-service/packer-import-1692542680.qcow2"
==> file.base (yandex-import): Error creating Yandex Compute Image
2023/08/20 16:44:42 packer-plugin-yandex plugin: 2023/08/20 16:44:42 error: server-request-id = 774a62cb-30ac-48ae-9c37-5ecfe287129a server-trace-id = aa1f2ea3dba95312:2629a01afaf93b0:aa1f2ea3dba95312:1 client-request-id = e556be88-1eff-4af3-abd6-bd84b2d8fa97 client-trace-id = 6353f961-9098-4240-b03d-5e330191f0bf rpc error: code = InvalidArgument desc = Request validation error: URI validating error: invalid host/port.

Copy link
Contributor

@lbajolet-hashicorp lbajolet-hashicorp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @kuzpactor,

Thanks for opening this PR, the code looks good to me, I suggested adding a check during the Configure step for the post-processors, since this is intended to catch configuration errors before we even attempt to run the post-processor.

In the case of conflicting options such as the ones you're submitting here, I think it would be a good spot to do this check (or potentially warn that one will be ignored).

I'll let you decide what to do on those comments, and I'll re-review once you've had a chance to update.

Thanks again for the contribution!

p.config.StorageRegion = defaultStorageRegion
}

if p.config.StorageEndpointAutoresolve {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably also check during Configure that this is not set alongside StorageEndpoint, the two look mutually exclusive, so we can error if both are set in the same config.

// StorageEndpoint custom Yandex Object Storage endpoint to upload image, Default `storage.yandexcloud.net`.
StorageEndpoint string `mapstructure:"storage_endpoint" required:"false"`
// StorageEndpointAutoresolve auto resolve storage endpoint via YC Public API ListEndpoints call. Option has
// precedence over 'storage_endpoint' option.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we reject using both at the same time through a check in Configure (which I think would make sense), this comment should be amended to state that this and the storage_endpoint options are mutually exclusive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make object storage and region configurable
2 participants