From a52df42adee589321abead8e27cf3bc4dc7c6383 Mon Sep 17 00:00:00 2001 From: Rakavitha Kodhandapani Date: Fri, 25 Oct 2024 12:59:15 +0530 Subject: [PATCH 01/14] first draft of the securing endpoints --- docs-gb/SUMMARY.md | 1 + docs-gb/models/securing-endpoints.md | 73 ++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+) create mode 100644 docs-gb/models/securing-endpoints.md diff --git a/docs-gb/SUMMARY.md b/docs-gb/SUMMARY.md index 51631b8886..a447d478ab 100644 --- a/docs-gb/SUMMARY.md +++ b/docs-gb/SUMMARY.md @@ -78,6 +78,7 @@ * [rClone](models/rclone.md) * [Parameterized Models](models/parameterized-models/README.md) * [Pandas Query](models/parameterized-models/pandasquery.md) + * [Securing Endpoints](models/securing-endpoints.md) * [Metrics](metrics/README.md) * [Usage](metrics/usage.md) * [Operational](metrics/operational.md) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md new file mode 100644 index 0000000000..66b1d06ff0 --- /dev/null +++ b/docs-gb/models/securing-endpoints.md @@ -0,0 +1,73 @@ +# Securing model endpoints + +You can secure the endpoints of a model that you deployed in a Kubernetes cluster using a service mesh. You can configure multiple layers of security within an Istio Gateway. For instance, you can configure [TLS for HTTPS at the gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-tls-ingress-gateway-for-a-single-host) level, enable [mutual TLS (mTLS) to secure internal communication](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-mutual-tls-ingress-gateway), and apply [AuthorizationPolicies](https://istio.io/latest/docs/reference/config/security/authorization-policy/) and [RequestAuthentication](https://istio.io/latest/docs/reference/config/security/request_authentication/) policies to enforce both authentication and authorization controls. + +## Prerequaites +* [Deploy a model] +* [Configure a gateway] +* [Create a virtual service to expose the REST and gRPC endpoints] +* Configure a OIDC provider to authenticate `https://$MESH_IP/v2`. Obtain the `issuer` url, `jwksUri`, and the `Access token` from the OIDC provider. + +In the following example, you can secure the endpoint such that any requests to the end point without the access token are denied. + +To secure the enpoints of a model, you need to: +1. Create a RequestAuthentication resource `ingress-jwt-auth` in the namespace `istio-system`. + ```yaml + apiVersion: security.istio.io/v1beta1 + kind: RequestAuthentication + metadata: + name: ingress-jwt-auth + namespace: istio-system # This is the namespace where Istio Ingress Gateway usually resides + spec: + selector: + matchLabels: + istio: istio-ingressgateway # Apply to Istio Ingress Gateway pods + jwtRules: + - issuer: "https://fc4dba59-f6ea-4f05-9fd2-37ff194947ba.app.skycloak.io/realms/core2" + jwksUri: "https://fc4dba59-f6ea-4f05-9fd2-37ff194947ba.app.skycloak.io/realms/core2/protocol/openid-connect/certs" + ``` + +2. Create Authetication policy `deny-empty-jwt` in the namespace `istio-system`. + ```yaml + apiVersion: security.istio.io/v1beta1 + kind: AuthorizationPolicy + metadata: + name: core-v2-ingress + namespace: istio-system + spec: + action: DENY + rules: + - from: + - source: + notRequestPrincipals: + - '*' + to: + - operation: + paths: + - /v2/* + selector: + matchLabels: + app: istio-ingressgateway # Applies to Istio Ingress Gateway pods + ``` +3. To verify that the requests without an access token are denied send this request: + ```bash + curl -i http://$MESH_IP/models/iris/infer \ + -H "Content-Type: application/json" \ + -d '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}' + ``` + The output is similar to: + ```bash + + ``` + Now, send the same request with an access token: + ```bash + curl -i http://34.90.95.128/v2/models/iris/infer \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $ACCESS_TOKEN" \ + -d '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}' + ``` + The output is similar to: + ```bash + + ``` + From 692aeb54fa7e9d3b8161b49a12470f34df4d65c6 Mon Sep 17 00:00:00 2001 From: Rakavitha Kodhandapani Date: Fri, 25 Oct 2024 17:45:18 +0530 Subject: [PATCH 02/14] added the output --- docs-gb/models/securing-endpoints.md | 49 +++++++++++++++++++++------- 1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index 66b1d06ff0..40fabcc4cf 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -3,15 +3,15 @@ You can secure the endpoints of a model that you deployed in a Kubernetes cluster using a service mesh. You can configure multiple layers of security within an Istio Gateway. For instance, you can configure [TLS for HTTPS at the gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-tls-ingress-gateway-for-a-single-host) level, enable [mutual TLS (mTLS) to secure internal communication](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-mutual-tls-ingress-gateway), and apply [AuthorizationPolicies](https://istio.io/latest/docs/reference/config/security/authorization-policy/) and [RequestAuthentication](https://istio.io/latest/docs/reference/config/security/request_authentication/) policies to enforce both authentication and authorization controls. ## Prerequaites -* [Deploy a model] -* [Configure a gateway] -* [Create a virtual service to expose the REST and gRPC endpoints] -* Configure a OIDC provider to authenticate `https://$MESH_IP/v2`. Obtain the `issuer` url, `jwksUri`, and the `Access token` from the OIDC provider. +* [Deploy a model](/kubernetes/service-meshes/istio.md) +* [Configure a gateway](/kubernetes/service-meshes/istio.md) +* [Create a virtual service to expose the REST and gRPC endpoints](/kubernetes/service-meshes/istio.md) +* Configure a OIDC provider to authenticate. Obtain the `issuer` url, `jwksUri`, and the `Access token` from the OIDC provider. In the following example, you can secure the endpoint such that any requests to the end point without the access token are denied. -To secure the enpoints of a model, you need to: -1. Create a RequestAuthentication resource `ingress-jwt-auth` in the namespace `istio-system`. +To secure the endpoints of a model, you need to: +1. Create a `RequestAuthentication` resource named `ingress-jwt-auth` in the `istio-system namespace`. Replace `` and `` with your OIDC provider’s specific issuer URL and JWKS (JSON Web Key Set) URI. ```yaml apiVersion: security.istio.io/v1beta1 kind: RequestAuthentication @@ -23,8 +23,8 @@ To secure the enpoints of a model, you need to: matchLabels: istio: istio-ingressgateway # Apply to Istio Ingress Gateway pods jwtRules: - - issuer: "https://fc4dba59-f6ea-4f05-9fd2-37ff194947ba.app.skycloak.io/realms/core2" - jwksUri: "https://fc4dba59-f6ea-4f05-9fd2-37ff194947ba.app.skycloak.io/realms/core2/protocol/openid-connect/certs" + - issuer: + jwksUri: ``` 2. Create Authetication policy `deny-empty-jwt` in the namespace `istio-system`. @@ -51,23 +51,48 @@ To secure the enpoints of a model, you need to: ``` 3. To verify that the requests without an access token are denied send this request: ```bash - curl -i http://$MESH_IP/models/iris/infer \ + curl -i http://$MESH_IP/v2/models/iris/infer \ -H "Content-Type: application/json" \ + -H "seldon-model":iris \ -d '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}' ``` The output is similar to: ```bash - + HTTP/1.1 403 Forbidden + content-length: 19 + content-type: text/plain + date: Fri, 25 Oct 2024 11:14:33 GMT + server: istio-envoy + connection: close + Closing connection 0 + RBAC: access denied ``` Now, send the same request with an access token: ```bash - curl -i http://34.90.95.128/v2/models/iris/infer \ + curl -i http://$MESH_IP/v2/models/iris/infer \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $ACCESS_TOKEN" \ + -H "seldon-model":iris \ -d '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}' ``` The output is similar to: ```bash - + HTTP/1.1 200 OK + ce-endpoint: iris_1 + ce-id: 2fb8a086-ee22-4285-9826-9d38111cbb9e + ce-inferenceservicename: mlserver + ce-modelid: iris_1 + ce-namespace: seldon-mesh + ce-requestid: 2fb8a086-ee22-4285-9826-9d38111cbb9e + ce-source: io.seldon.serving.deployment.mlserver.seldon-mesh + ce-specversion: 0.3 + ce-type: io.seldon.serving.inference.response + content-length: 213 + content-type: application/json + date: Fri, 25 Oct 2024 11:44:49 GMT + server: envoy + x-request-id: csdo9cbc2nks73dtlk3g + x-envoy-upstream-service-time: 9 + x-seldon-route: :iris_1: ``` From ae3a53f32cbdc335656654d8b6d4e54a0b67ed78 Mon Sep 17 00:00:00 2001 From: Rakavitha Kodhandapani Date: Fri, 25 Oct 2024 17:50:12 +0530 Subject: [PATCH 03/14] updated the policy name --- docs-gb/models/securing-endpoints.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index 40fabcc4cf..a6a38d37a4 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -32,7 +32,7 @@ To secure the endpoints of a model, you need to: apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: - name: core-v2-ingress + name: deny-empty-jwt namespace: istio-system spec: action: DENY From 8d43055b79a29f6d70c117b671eca8e96738934d Mon Sep 17 00:00:00 2001 From: Rakavitha Kodhandapani Date: Fri, 25 Oct 2024 17:59:31 +0530 Subject: [PATCH 04/14] added a note --- docs-gb/models/securing-endpoints.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index a6a38d37a4..db00af9040 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -7,6 +7,9 @@ You can secure the endpoints of a model that you deployed in a Kubernetes cluste * [Configure a gateway](/kubernetes/service-meshes/istio.md) * [Create a virtual service to expose the REST and gRPC endpoints](/kubernetes/service-meshes/istio.md) * Configure a OIDC provider to authenticate. Obtain the `issuer` url, `jwksUri`, and the `Access token` from the OIDC provider. +{% hint style="info" %} +**Note** There are many types of authorization policies that you can configure to enable access control on workloads in the mesh. +{% endhint %} In the following example, you can secure the endpoint such that any requests to the end point without the access token are denied. @@ -27,7 +30,7 @@ To secure the endpoints of a model, you need to: jwksUri: ``` -2. Create Authetication policy `deny-empty-jwt` in the namespace `istio-system`. +2. Create an authorization policy `deny-empty-jwt` in the namespace `istio-system`. ```yaml apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy From ac9bdb3eafc539c07fdc2109d00b6cc850cd3711 Mon Sep 17 00:00:00 2001 From: Paul Bridi Date: Mon, 28 Oct 2024 13:55:32 +0000 Subject: [PATCH 05/14] Added context, minor grammar edits --- docs-gb/models/securing-endpoints.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index db00af9040..c1d7a082a0 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -1,8 +1,12 @@ # Securing model endpoints -You can secure the endpoints of a model that you deployed in a Kubernetes cluster using a service mesh. You can configure multiple layers of security within an Istio Gateway. For instance, you can configure [TLS for HTTPS at the gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-tls-ingress-gateway-for-a-single-host) level, enable [mutual TLS (mTLS) to secure internal communication](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-mutual-tls-ingress-gateway), and apply [AuthorizationPolicies](https://istio.io/latest/docs/reference/config/security/authorization-policy/) and [RequestAuthentication](https://istio.io/latest/docs/reference/config/security/request_authentication/) policies to enforce both authentication and authorization controls. +Often, in enterprise use-cases, users will want to define who is able to hit the endpoints exposed for deployed models or pipelines. Seldon Core itself does not provide that functionality, but it does integrate easily with various service meshes that support this requirement. Seldon Core 2 is service mesh agnostic, but the example below will demonstrate an approach to setting up authentication and authorization in order to secure a model endpoint using Istio. -## Prerequaites +## Securing Endpoints with Istio + +Service meshes offer a flexible way of defining authentication and authorization rules for your models. With Istio, for example, you can configure multiple layers of security within an Istio Gateway, such as a [TLS for HTTPS at the gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-tls-ingress-gateway-for-a-single-host) level, [mutual TLS (mTLS) for secure internal communication](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-mutual-tls-ingress-gateway), as well as [AuthorizationPolicies](https://istio.io/latest/docs/reference/config/security/authorization-policy/) and [RequestAuthentication](https://istio.io/latest/docs/reference/config/security/request_authentication/) policies to enforce both authentication and authorization controls. + +## Prerequisites * [Deploy a model](/kubernetes/service-meshes/istio.md) * [Configure a gateway](/kubernetes/service-meshes/istio.md) * [Create a virtual service to expose the REST and gRPC endpoints](/kubernetes/service-meshes/istio.md) @@ -11,7 +15,7 @@ You can secure the endpoints of a model that you deployed in a Kubernetes cluste **Note** There are many types of authorization policies that you can configure to enable access control on workloads in the mesh. {% endhint %} -In the following example, you can secure the endpoint such that any requests to the end point without the access token are denied. +In the following example, you can secure the endpoint such that any requests to the endpoint without the access token are denied. To secure the endpoints of a model, you need to: 1. Create a `RequestAuthentication` resource named `ingress-jwt-auth` in the `istio-system namespace`. Replace `` and `` with your OIDC provider’s specific issuer URL and JWKS (JSON Web Key Set) URI. From 617f1b2e236ea5d6904728a46aa5b7add9efe433 Mon Sep 17 00:00:00 2001 From: paulb-seldon <141156400+paulb-seldon@users.noreply.github.com> Date: Mon, 28 Oct 2024 14:44:54 +0000 Subject: [PATCH 06/14] Update docs-gb/models/securing-endpoints.md Co-authored-by: Rajakavitha Kodhandapani --- docs-gb/models/securing-endpoints.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index c1d7a082a0..a94cd63162 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -1,6 +1,6 @@ # Securing model endpoints -Often, in enterprise use-cases, users will want to define who is able to hit the endpoints exposed for deployed models or pipelines. Seldon Core itself does not provide that functionality, but it does integrate easily with various service meshes that support this requirement. Seldon Core 2 is service mesh agnostic, but the example below will demonstrate an approach to setting up authentication and authorization in order to secure a model endpoint using Istio. +In enterprise use cases, you may need to control who can access the endpoints for deployed models or pipelines. While Seldon Core 2 doesn’t natively provide this functionality, it integrates with various service meshes that support these requirements. Though Seldon Core 2 is service-mesh agnostic, the example on this page demonstrates how to set up authentication and authorization to secure a model endpoint using the Istio service mesh. ## Securing Endpoints with Istio From e7057dbffd9215f8bbe9df2e5b55da30e86fbece Mon Sep 17 00:00:00 2001 From: Rajakavitha Kodhandapani Date: Wed, 6 Nov 2024 09:57:15 +0530 Subject: [PATCH 07/14] incorporate review suggestions --- docs-gb/models/securing-endpoints.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index a94cd63162..0ed1cdbcde 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -1,6 +1,6 @@ # Securing model endpoints -In enterprise use cases, you may need to control who can access the endpoints for deployed models or pipelines. While Seldon Core 2 doesn’t natively provide this functionality, it integrates with various service meshes that support these requirements. Though Seldon Core 2 is service-mesh agnostic, the example on this page demonstrates how to set up authentication and authorization to secure a model endpoint using the Istio service mesh. +In enterprise use cases, you may need to control who can access the endpoints for deployed models or pipelines. You can leverage existing authentication mechanisms in your cluster or environment, such as service mesh-level controls, or use cloud provider solutions like Apigee on GCP, Amazon API Gateway on AWS, or a provider-agnostic gateway like Gravitee. Seldon Core 2 integrates with various service meshes that support these requirements. Though Seldon Core 2 is service-mesh agnostic, the example on this page demonstrates how to set up authentication and authorization to secure a model endpoint using the Istio service mesh. ## Securing Endpoints with Istio From 6d470bf8598d15067475b74c8403d504d64d5a0f Mon Sep 17 00:00:00 2001 From: Rajakavitha Kodhandapani Date: Wed, 6 Nov 2024 10:27:38 +0530 Subject: [PATCH 08/14] fixing the links --- docs-gb/models/securing-endpoints.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index 0ed1cdbcde..bf2ee002e2 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -7,9 +7,9 @@ In enterprise use cases, you may need to control who can access the endpoints fo Service meshes offer a flexible way of defining authentication and authorization rules for your models. With Istio, for example, you can configure multiple layers of security within an Istio Gateway, such as a [TLS for HTTPS at the gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-tls-ingress-gateway-for-a-single-host) level, [mutual TLS (mTLS) for secure internal communication](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-mutual-tls-ingress-gateway), as well as [AuthorizationPolicies](https://istio.io/latest/docs/reference/config/security/authorization-policy/) and [RequestAuthentication](https://istio.io/latest/docs/reference/config/security/request_authentication/) policies to enforce both authentication and authorization controls. ## Prerequisites -* [Deploy a model](/kubernetes/service-meshes/istio.md) -* [Configure a gateway](/kubernetes/service-meshes/istio.md) -* [Create a virtual service to expose the REST and gRPC endpoints](/kubernetes/service-meshes/istio.md) +* [Deploy a model](../kubernetes/service-meshes/istio.md) +* [Configure a gateway](../kubernetes/service-meshes/istio.md) +* [Create a virtual service to expose the REST and gRPC endpoints](../kubernetes/service-meshes/istio.md) * Configure a OIDC provider to authenticate. Obtain the `issuer` url, `jwksUri`, and the `Access token` from the OIDC provider. {% hint style="info" %} **Note** There are many types of authorization policies that you can configure to enable access control on workloads in the mesh. From 2dca09b6f80f2e68a11f76314b81f27aefb29b76 Mon Sep 17 00:00:00 2001 From: Rajakavitha Kodhandapani Date: Fri, 8 Nov 2024 17:21:48 +0530 Subject: [PATCH 09/14] added an example for all models --- docs-gb/kubernetes/service-meshes/istio.md | 94 +++++++++++++++++++++- 1 file changed, 93 insertions(+), 1 deletion(-) diff --git a/docs-gb/kubernetes/service-meshes/istio.md b/docs-gb/kubernetes/service-meshes/istio.md index d1fbd42a15..4bbdd67878 100644 --- a/docs-gb/kubernetes/service-meshes/istio.md +++ b/docs-gb/kubernetes/service-meshes/istio.md @@ -49,7 +49,7 @@ spec: privateKey: /etc/istio/ingressgateway-certs/tls.key serverCertificate: /etc/istio/ingressgateway-certs/tls.crt --- -apiVersion: networking.istio.io/v1beta1 +apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: iris-route @@ -84,6 +84,98 @@ spec: seldon-model: iris ``` +### Create a Virtual Service for all models + +You can create a virtual service for all models in the `seldon-mesh` namespace. For example, you can set HTTP routes for `model-a`, `model-b`, and `model-c` with a unique route based on its URI prefix (/model-a), directing HTTP traffic to the appropriate service. +You can also set the gRPC routes for each model with uri prefixes for `inference.GRPCInferenceService`, directing gRPC traffic to different ports such as 50051, 50052, 50053 for each model. + +```yaml +apiVersion: networking.istio.io/v1alpha3 +kind: VirtualService +metadata: + name: seldon-models-route + namespace: seldon-mesh +spec: + gateways: + - istio-system/seldon-gateway + hosts: + - '*' + http: + - match: + - uri: + prefix: /model-a + name: model-a-http + route: + - destination: + host: model-a.seldon-mesh.svc.cluster.local + headers: + request: + set: + seldon-model: model-a + - match: + - uri: + prefix: /model-b + name: model-b-http + route: + - destination: + host: model-b.seldon-mesh.svc.cluster.local + headers: + request: + set: + seldon-model: model-b + - match: + - uri: + prefix: /model-c + name: model-c-http + route: + - destination: + host: model-c.seldon-mesh.svc.cluster.local + headers: + request: + set: + seldon-model: model-c + - match: + - uri: + prefix: /inference.GRPCInferenceService/model-a + name: model-a-grpc + route: + - destination: + host: model-a.seldon-mesh.svc.cluster.local + port: + number: 50051 + headers: + request: + set: + seldon-model: model-a + - match: + - uri: + prefix: /inference.GRPCInferenceService/model-b + name: model-b-grpc + route: + - destination: + host: model-b.seldon-mesh.svc.cluster.local + port: + number: 50052 + headers: + request: + set: + seldon-model: model-b + - match: + - uri: + prefix: /inference.GRPCInferenceService/model-c + name: model-c-grpc + route: + - destination: + host: model-c.seldon-mesh.svc.cluster.local + port: + number: 50053 + headers: + request: + set: + seldon-model: model-c + +``` + ## Traffic Split * Two Iris Models From 9a48ecdad3e1def1fcec9ea47b82e0773aad272b Mon Sep 17 00:00:00 2001 From: Rajakavitha Kodhandapani Date: Fri, 29 Nov 2024 13:45:37 +0530 Subject: [PATCH 10/14] removed the example to create a vs for all models --- docs-gb/kubernetes/service-meshes/istio.md | 92 ---------------------- 1 file changed, 92 deletions(-) diff --git a/docs-gb/kubernetes/service-meshes/istio.md b/docs-gb/kubernetes/service-meshes/istio.md index 4bbdd67878..4a16db5b5b 100644 --- a/docs-gb/kubernetes/service-meshes/istio.md +++ b/docs-gb/kubernetes/service-meshes/istio.md @@ -84,98 +84,6 @@ spec: seldon-model: iris ``` -### Create a Virtual Service for all models - -You can create a virtual service for all models in the `seldon-mesh` namespace. For example, you can set HTTP routes for `model-a`, `model-b`, and `model-c` with a unique route based on its URI prefix (/model-a), directing HTTP traffic to the appropriate service. -You can also set the gRPC routes for each model with uri prefixes for `inference.GRPCInferenceService`, directing gRPC traffic to different ports such as 50051, 50052, 50053 for each model. - -```yaml -apiVersion: networking.istio.io/v1alpha3 -kind: VirtualService -metadata: - name: seldon-models-route - namespace: seldon-mesh -spec: - gateways: - - istio-system/seldon-gateway - hosts: - - '*' - http: - - match: - - uri: - prefix: /model-a - name: model-a-http - route: - - destination: - host: model-a.seldon-mesh.svc.cluster.local - headers: - request: - set: - seldon-model: model-a - - match: - - uri: - prefix: /model-b - name: model-b-http - route: - - destination: - host: model-b.seldon-mesh.svc.cluster.local - headers: - request: - set: - seldon-model: model-b - - match: - - uri: - prefix: /model-c - name: model-c-http - route: - - destination: - host: model-c.seldon-mesh.svc.cluster.local - headers: - request: - set: - seldon-model: model-c - - match: - - uri: - prefix: /inference.GRPCInferenceService/model-a - name: model-a-grpc - route: - - destination: - host: model-a.seldon-mesh.svc.cluster.local - port: - number: 50051 - headers: - request: - set: - seldon-model: model-a - - match: - - uri: - prefix: /inference.GRPCInferenceService/model-b - name: model-b-grpc - route: - - destination: - host: model-b.seldon-mesh.svc.cluster.local - port: - number: 50052 - headers: - request: - set: - seldon-model: model-b - - match: - - uri: - prefix: /inference.GRPCInferenceService/model-c - name: model-c-grpc - route: - - destination: - host: model-c.seldon-mesh.svc.cluster.local - port: - number: 50053 - headers: - request: - set: - seldon-model: model-c - -``` - ## Traffic Split * Two Iris Models From 5e9b0d38099a0ce3e85a8d84a3c077112b50e785 Mon Sep 17 00:00:00 2001 From: Rajakavitha Kodhandapani Date: Fri, 29 Nov 2024 13:57:44 +0530 Subject: [PATCH 11/14] fixed formatting --- docs-gb/models/securing-endpoints.md | 69 +++++++++++++++------------- 1 file changed, 36 insertions(+), 33 deletions(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index bf2ee002e2..a3d8fba167 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -19,43 +19,46 @@ In the following example, you can secure the endpoint such that any requests to To secure the endpoints of a model, you need to: 1. Create a `RequestAuthentication` resource named `ingress-jwt-auth` in the `istio-system namespace`. Replace `` and `` with your OIDC provider’s specific issuer URL and JWKS (JSON Web Key Set) URI. - ```yaml - apiVersion: security.istio.io/v1beta1 - kind: RequestAuthentication - metadata: - name: ingress-jwt-auth - namespace: istio-system # This is the namespace where Istio Ingress Gateway usually resides - spec: - selector: - matchLabels: - istio: istio-ingressgateway # Apply to Istio Ingress Gateway pods - jwtRules: - - issuer: - jwksUri: - ``` + +```yaml +apiVersion: security.istio.io/v1beta1 +kind: RequestAuthentication +metadata: + name: ingress-jwt-auth + namespace: istio-system # This is the namespace where Istio Ingress Gateway usually resides +spec: + selector: + matchLabels: + istio: istio-ingressgateway # Apply to Istio Ingress Gateway pods + jwtRules: + - issuer: + jwksUri: +``` 2. Create an authorization policy `deny-empty-jwt` in the namespace `istio-system`. - ```yaml - apiVersion: security.istio.io/v1beta1 - kind: AuthorizationPolicy - metadata: - name: deny-empty-jwt - namespace: istio-system - spec: - action: DENY - rules: - - from: + +```yaml +apiVersion: security.istio.io/v1beta1 +kind: AuthorizationPolicy +metadata: + name: deny-empty-jwt + namespace: istio-system +spec: + action: DENY + rules: + - from: - source: notRequestPrincipals: - - '*' - to: - - operation: - paths: - - /v2/* - selector: - matchLabels: - app: istio-ingressgateway # Applies to Istio Ingress Gateway pods - ``` + - '*' # Denies requests without a valid JWT principal + to: + - operation: + paths: + - /v2/* # Applies to requests with this path pattern + selector: + matchLabels: + app: istio-ingressgateway # Applies to Istio Ingress Gateway pods +``` + 3. To verify that the requests without an access token are denied send this request: ```bash curl -i http://$MESH_IP/v2/models/iris/infer \ From 2d01638224c836a16b2d8acabb74ee07c458fb75 Mon Sep 17 00:00:00 2001 From: Rajakavitha Kodhandapani Date: Fri, 29 Nov 2024 14:03:50 +0530 Subject: [PATCH 12/14] formatting changes --- docs-gb/models/securing-endpoints.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index a3d8fba167..6f49ed9a1e 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -34,6 +34,7 @@ spec: - issuer: jwksUri: ``` +Create the resource using `kubectl apply -f ingress-jwt-auth.yaml`. 2. Create an authorization policy `deny-empty-jwt` in the namespace `istio-system`. @@ -58,6 +59,7 @@ spec: matchLabels: app: istio-ingressgateway # Applies to Istio Ingress Gateway pods ``` +Create the resource using `kubectl apply -f deny-empty-jwt.yaml`. 3. To verify that the requests without an access token are denied send this request: ```bash From ba071a4a95c98dfa396c621cff5af4f781d23e23 Mon Sep 17 00:00:00 2001 From: Rajakavitha Kodhandapani Date: Fri, 29 Nov 2024 14:06:47 +0530 Subject: [PATCH 13/14] Update securing-endpoints.md --- docs-gb/models/securing-endpoints.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index 6f49ed9a1e..8ec8dab7bf 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -6,7 +6,7 @@ In enterprise use cases, you may need to control who can access the endpoints fo Service meshes offer a flexible way of defining authentication and authorization rules for your models. With Istio, for example, you can configure multiple layers of security within an Istio Gateway, such as a [TLS for HTTPS at the gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-tls-ingress-gateway-for-a-single-host) level, [mutual TLS (mTLS) for secure internal communication](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-mutual-tls-ingress-gateway), as well as [AuthorizationPolicies](https://istio.io/latest/docs/reference/config/security/authorization-policy/) and [RequestAuthentication](https://istio.io/latest/docs/reference/config/security/request_authentication/) policies to enforce both authentication and authorization controls. -## Prerequisites +**Prerequisites** * [Deploy a model](../kubernetes/service-meshes/istio.md) * [Configure a gateway](../kubernetes/service-meshes/istio.md) * [Create a virtual service to expose the REST and gRPC endpoints](../kubernetes/service-meshes/istio.md) From b282ccea5338418cb550e8d781cab3209cb19473 Mon Sep 17 00:00:00 2001 From: Rajakavitha Kodhandapani Date: Thu, 5 Dec 2024 15:33:38 +0530 Subject: [PATCH 14/14] added a link to the services meshes main docs page --- docs-gb/models/securing-endpoints.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-gb/models/securing-endpoints.md b/docs-gb/models/securing-endpoints.md index 8ec8dab7bf..debce9b854 100644 --- a/docs-gb/models/securing-endpoints.md +++ b/docs-gb/models/securing-endpoints.md @@ -1,6 +1,6 @@ # Securing model endpoints -In enterprise use cases, you may need to control who can access the endpoints for deployed models or pipelines. You can leverage existing authentication mechanisms in your cluster or environment, such as service mesh-level controls, or use cloud provider solutions like Apigee on GCP, Amazon API Gateway on AWS, or a provider-agnostic gateway like Gravitee. Seldon Core 2 integrates with various service meshes that support these requirements. Though Seldon Core 2 is service-mesh agnostic, the example on this page demonstrates how to set up authentication and authorization to secure a model endpoint using the Istio service mesh. +In enterprise use cases, you may need to control who can access the endpoints for deployed models or pipelines. You can leverage existing authentication mechanisms in your cluster or environment, such as service mesh-level controls, or use cloud provider solutions like Apigee on GCP, Amazon API Gateway on AWS, or a provider-agnostic gateway like Gravitee. Seldon Core 2 integrates with various [service meshes](../kubernetes/service-meshes/) that support these requirements. Though Seldon Core 2 is service-mesh agnostic, the example on this page demonstrates how to set up authentication and authorization to secure a model endpoint using the Istio service mesh. ## Securing Endpoints with Istio