Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Wasm OCI image #3564

Merged
merged 36 commits into from
Jun 28, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
d7e1e63
support Wasm OCI image
zhaohuabing Jun 17, 2024
df6bee1
set up test registry
zhaohuabing Jun 12, 2024
e079dc2
add test for registry authn
zhaohuabing Jun 14, 2024
d16ca5d
fix lint
zhaohuabing Jun 14, 2024
ed5311a
fix e2e
zhaohuabing Jun 14, 2024
689266a
fix e2e
zhaohuabing Jun 14, 2024
409967d
add test for unauthed private image
zhaohuabing Jun 14, 2024
f243f54
fix e2e
zhaohuabing Jun 14, 2024
6ff1ac3
fix e2e
zhaohuabing Jun 15, 2024
6ddc332
fix lint
zhaohuabing Jun 15, 2024
0bc5532
refactor
zhaohuabing Jun 15, 2024
1cfccbb
add max failed attempts limit
zhaohuabing Jun 15, 2024
10ce3b9
remove retries
zhaohuabing Jun 15, 2024
d84a113
clean up e2e tests
zhaohuabing Jun 15, 2024
cad02e1
add e2e test for wrong password
zhaohuabing Jun 17, 2024
0d81b8a
Update api/v1alpha1/authorization_types.go
zhaohuabing Jun 18, 2024
171a937
Update api/v1alpha1/wasm_types.go
zhaohuabing Jun 18, 2024
0b86837
remove unnecessary replace
zhaohuabing Jun 18, 2024
d32d325
Merge remote-tracking branch 'upstream/main' into wasm-oci-image
zhaohuabing Jun 18, 2024
b4917f9
remove set package
zhaohuabing Jun 18, 2024
8a69562
fix gen check
zhaohuabing Jun 18, 2024
fedbe2f
add test for failed attempts
zhaohuabing Jun 20, 2024
d6efea3
address comments
zhaohuabing Jun 20, 2024
3c61c51
address comments
zhaohuabing Jun 20, 2024
2e96e1c
Merge remote-tracking branch 'upstream/main' into wasm-oci-image
zhaohuabing Jun 20, 2024
e8a3fce
minor wording
zhaohuabing Jun 20, 2024
56aea0e
move sha256 inside code source
zhaohuabing Jun 21, 2024
591b49d
address comments
zhaohuabing Jun 21, 2024
9b8e42c
fix e2e
zhaohuabing Jun 21, 2024
9d71021
fix flaky test
zhaohuabing Jun 22, 2024
9a16d9f
change comments
zhaohuabing Jun 25, 2024
94ed656
address comments
zhaohuabing Jun 27, 2024
8ac7991
Merge remote-tracking branch 'upstream/main' into wasm-oci-image
zhaohuabing Jun 27, 2024
0cdc304
address comments
zhaohuabing Jun 27, 2024
138e978
Merge remote-tracking branch 'upstream/main' into wasm-oci-image
zhaohuabing Jun 27, 2024
86016ee
fail the eep translation if the wasm cache failed to start
zhaohuabing Jun 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion api/v1alpha1/authorization_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ type Authorization struct {

// AuthorizationRule defines a single authorization rule.
type AuthorizationRule struct {
// Name is a user-friendly name for the rule. It's just for display purposes.
// Name is a user-friendly name for the rule.
// If not specified, Envoy Gateway will generate a unique name for the rule.n
// +optional
Name *string `json:"name,omitempty"`

Expand Down
63 changes: 40 additions & 23 deletions api/v1alpha1/wasm_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,16 @@ type Wasm struct {
// Name is a unique name for this Wasm extension. It is used to identify the
// Wasm extension if multiple extensions are handled by the same vm_id and root_id.
// It's also used for logging/debugging.
Name string `json:"name"`

// VMID is an ID that will be used along with a hash of the wasm code to
// determine which VM will be used to load the Wasm extension. All extensions
// that have the same vm_id and code will use the same VM.
// If not specified, EG will generate a unique name for the Wasm extension.
//
// Note that sharing a VM between plugins can reduce memory utilization and
// make sharing of data easier, but it may have security implications.
// VMID *string `json:"vmID,omitempty"`
// +optional
Name *string `json:"name,omitempty"`

// RootID is a unique ID for a set of extensions in a VM which will share a
// RootContext and Contexts if applicable (e.g., an Wasm HttpFilter and an Wasm AccessLog).
// If left blank, all extensions with a blank root_id with the same vm_id will share Context(s).
// RootID must match the root_id parameter used to register the Context in the Wasm code.
//
// Note: RootID must match the root_id parameter used to register the Context in the Wasm code.
RootID *string `json:"rootID,omitempty"`

// Code is the wasm code for the extension.
Expand Down Expand Up @@ -59,6 +55,10 @@ type Wasm struct {
}

// WasmCodeSource defines the source of the wasm code.
// +union
//
// +kubebuilder:validation:XValidation:rule="self.type == 'HTTP' ? has(self.http) : !has(self.http)",message="If type is HTTP, http field needs to be set."
// +kubebuilder:validation:XValidation:rule="self.type == 'Image' ? has(self.image) : !has(self.image)",message="If type is Image, image field needs to be set."
type WasmCodeSource struct {
// Type is the type of the source of the wasm code.
// Valid WasmCodeSourceType values are "HTTP" or "Image".
Expand All @@ -81,8 +81,21 @@ type WasmCodeSource struct {

// SHA256 checksum that will be used to verify the wasm code.
//
// If not specified, Envoy Gateway will not verify the downloaded wasm code.
// kubebuilder:validation:Pattern=`^[a-f0-9]{64}$`
SHA256 string `json:"sha256"`
// +optional
SHA256 *string `json:"sha256"`
zhaohuabing marked this conversation as resolved.
Show resolved Hide resolved

// PullPolicy is the policy to use when pulling the Wasm module by either the HTTP or Image source.
// This field is only applicable when the SHA256 field is not set.
//
// If not specified, the default policy is IfNotPresent except for OCI images whose tag is latest.
//
// Note: EG does not update the Wasm module every time an Envoy proxy requests
// the Wasm module even if the pull policy is set to Always.
// It only updates the Wasm module when the EnvoyExtension resource version changes.
// +optional
PullPolicy *ImagePullPolicy `json:"pullPolicy,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldnt this be inside Image ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also used for HTTP code source.

	// PullPolicy is the policy to use when pulling the Wasm module by either the HTTP or Image source.
	// This field is only applicable when the SHA256 field is not set.
	//
	// If not specified, the default policy is IfNotPresent except for OCI images whose tag is latest.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ is a little confusing to understand

Copy link
Member Author

@zhaohuabing zhaohuabing Jun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If SHA is not specified, we need to know whether the Wasm code need updating when the EEP changes, regardless its code source is HTTP or OCI registry.

}

// WasmCodeSourceType specifies the types of sources for the wasm code.
Expand All @@ -100,31 +113,35 @@ const (
// HTTPWasmCodeSource defines the HTTP URL containing the wasm code.
type HTTPWasmCodeSource struct {
// URL is the URL containing the wasm code.
// +kubebuilder:validation:Pattern=`^((https?:)(\/\/\/?)([\w]*(?::[\w]*)?@)?([\d\w\.-]+)(?::(\d+))?)?([\/\\\w\.()-]*)?(?:([?][^#]*)?(#.*)?)*`
URL string `json:"url"`
}

// ImageWasmCodeSource defines the OCI image containing the wasm code.
type ImageWasmCodeSource struct {
// URL is the URL of the OCI image.
// URL can be in the format of `registry/image:tag` or `registry/image@sha256:digest`.
zhaohuabing marked this conversation as resolved.
Show resolved Hide resolved
// The image format must follow the Wasm Artifact Image Specification or the Compat Specification.
// - Wasm Artifact Image Specification: https://github.com/solo-io/wasm/blob/master/spec/spec.md
// - Compat Specification: https://github.com/solo-io/wasm/blob/master/spec/spec-compat.md
URL string `json:"url"`

// PullSecretRef is a reference to the secret containing the credentials to pull the image.
PullSecretRef gwapiv1b1.SecretObjectReference `json:"pullSecret"`

// PullPolicy is the policy to use when pulling the image.
// If not specified, the default policy is IfNotPresent for images whose tag is not latest,
// and Always for images whose tag is latest.
// Only support Kubernetes Secret resource from the same namespace.
// +kubebuilder:validation:XValidation:message="only support Secret kind.",rule="self.kind == 'Secret'"
// +optional
// PullPolicy *PullPolicy `json:"pullPolicy,omitempty"`
PullSecretRef *gwapiv1b1.SecretObjectReference `json:"pullSecretRef,omitempty"`
}

// PullPolicy defines the policy to use when pulling an OIC image.
/* type PullPolicy string
// ImagePullPolicy defines the policy to use when pulling an OIC image.
// +kubebuilder:validation:Enum=IfNotPresent;Always
type ImagePullPolicy string

const (
// PullPolicyIfNotPresent will only pull the image if it does not already exist.
PullPolicyIfNotPresent PullPolicy = "IfNotPresent"
// ImagePullPolicyIfNotPresent will only pull the image if it does not already exist in the EG cache.
ImagePullPolicyIfNotPresent ImagePullPolicy = "IfNotPresent"

// PullPolicyAlways will always pull the image.
PullPolicyAlways PullPolicy = "Always"
)*/
// ImagePullPolicyAlways will pull the image when the EnvoyExtension resource version changes.
// Note: EG does not update the Wasm module every time an Envoy proxy requests the Wasm module.
ImagePullPolicyAlways ImagePullPolicy = "Always"
)
21 changes: 20 additions & 1 deletion api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 6 additions & 3 deletions charts/gateway-helm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,12 @@ To uninstall the chart:
| deployment.ports[1].name | string | `"ratelimit"` | |
| deployment.ports[1].port | int | `18001` | |
| deployment.ports[1].targetPort | int | `18001` | |
| deployment.ports[2].name | string | `"metrics"` | |
| deployment.ports[2].port | int | `19001` | |
| deployment.ports[2].targetPort | int | `19001` | |
| deployment.ports[2].name | string | `"wasm"` | |
| deployment.ports[2].port | int | `18002` | |
| deployment.ports[2].targetPort | int | `18002` | |
| deployment.ports[3].name | string | `"metrics"` | |
| deployment.ports[3].port | int | `19001` | |
| deployment.ports[3].targetPort | int | `19001` | |
| deployment.replicas | int | `1` | |
| global.images.envoyGateway.image | string | `nil` | |
| global.images.envoyGateway.pullPolicy | string | `nil` | |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,7 @@ spec:
properties:
url:
description: URL is the URL containing the wasm code.
pattern: ^((https?:)(\/\/\/?)([\w]*(?::[\w]*)?@)?([\d\w\.-]+)(?::(\d+))?)?([\/\\\w\.()-]*)?(?:([?][^#]*)?(#.*)?)*
type: string
required:
- url
Expand All @@ -337,9 +338,10 @@ spec:

Note that the image must be accessible from the Envoy Gateway.
properties:
pullSecret:
description: PullSecretRef is a reference to the secret
containing the credentials to pull the image.
pullSecretRef:
description: |-
PullSecretRef is a reference to the secret containing the credentials to pull the image.
Only support Kubernetes Secret resource from the same namespace.
properties:
group:
default: ""
Expand Down Expand Up @@ -382,18 +384,42 @@ spec:
required:
- name
type: object
x-kubernetes-validations:
- message: only support Secret kind.
rule: self.kind == 'Secret'
url:
description: URL is the URL of the OCI image.
description: |-
URL is the URL of the OCI image.
URL can be in the format of `registry/image:tag` or `registry/image@sha256:digest`.
The image format must follow the Wasm Artifact Image Specification or the Compat Specification.
- Wasm Artifact Image Specification: https://github.com/solo-io/wasm/blob/master/spec/spec.md
- Compat Specification: https://github.com/solo-io/wasm/blob/master/spec/spec-compat.md
type: string
required:
- pullSecret
- url
type: object
pullPolicy:
description: |-
PullPolicy is the policy to use when pulling the Wasm module by either the HTTP or Image source.
This field is only applicable when the SHA256 field is not set.


If not specified, the default policy is IfNotPresent except for OCI images whose tag is latest.


Note: EG does not update the Wasm module every time an Envoy proxy requests
the Wasm module even if the pull policy is set to Always.
It only updates the Wasm module when the EnvoyExtension resource version changes.
enum:
- IfNotPresent
- Always
type: string
sha256:
description: |-
SHA256 checksum that will be used to verify the wasm code.


If not specified, Envoy Gateway will not verify the downloaded wasm code.
kubebuilder:validation:Pattern=`^[a-f0-9]{64}$`
type: string
type:
Expand All @@ -410,9 +436,13 @@ spec:
Valid WasmCodeSourceType values are "HTTP" or "Image".
type: string
required:
- sha256
- type
type: object
x-kubernetes-validations:
- message: If type is HTTP, http field needs to be set.
rule: 'self.type == ''HTTP'' ? has(self.http) : !has(self.http)'
- message: If type is Image, image field needs to be set.
rule: 'self.type == ''Image'' ? has(self.image) : !has(self.image)'
config:
description: |-
Config is the configuration for the Wasm extension.
Expand All @@ -433,17 +463,19 @@ spec:
Name is a unique name for this Wasm extension. It is used to identify the
Wasm extension if multiple extensions are handled by the same vm_id and root_id.
It's also used for logging/debugging.
If not specified, EG will generate a unique name for the Wasm extension.
type: string
rootID:
description: |-
RootID is a unique ID for a set of extensions in a VM which will share a
RootContext and Contexts if applicable (e.g., an Wasm HttpFilter and an Wasm AccessLog).
If left blank, all extensions with a blank root_id with the same vm_id will share Context(s).
RootID must match the root_id parameter used to register the Context in the Wasm code.


Note: RootID must match the root_id parameter used to register the Context in the Wasm code.
type: string
required:
- code
- name
type: object
maxItems: 16
type: array
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,9 @@ spec:
- Deny
type: string
name:
description: Name is a user-friendly name for the rule.
It's just for display purposes.
description: |-
Name is a user-friendly name for the rule.
If not specified, Envoy Gateway will generate a unique name for the rule.n
type: string
principal:
description: Principal specifies the client identity of
Expand Down
5 changes: 4 additions & 1 deletion charts/gateway-helm/values.tmpl.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ deployment:
- name: ratelimit
port: 18001
targetPort: 18001
- name: wasm
port: 18002
targetPort: 18002
- name: metrics
port: 19001
targetPort: 19001
Expand Down Expand Up @@ -76,4 +79,4 @@ certgen:
ttlSecondsAfterFinished: 30
rbac:
annotations: {}
labels: {}
labels: {}
Loading
Loading