fix(docs): first draft of the securing endpoints

[Rajakavitha1]

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ paulb-seldon
✅ Rajakavitha1
❌ Rakavitha Kodhandapani

---

Rakavitha Kodhandapani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[paulb-seldon]

Have made an edit to add context to the beginning of the page.

Another suggestion I have is to add a bit more content about what we are doing ahead of each of the steps. Instead of saying 'to secure model endpoints' ahead of the steps, I think we need to expand a bit more (I think it's fine in the context / intro, but when we go more into the details, we should explain exactly what we are doing). I.e. what is the auth policy doing, what is the authentication policy as shown doing.

Also ahead of each step can we call out what the placeholders are maybe we just need that for $MESH_IP?

[Rajakavitha1]

Hi @paulb-seldon I just rephrased the paragraph to adhere to the writing guidelines.

[Rajakavitha1]

Hi @paulb-seldon @lc525 and @cherrymu I have tested and validated the steps documented in the guide.

NAME                                        STATUS   ROLES    AGE   VERSION
gke-rajie-core-default-pool-808de050-ds6t   Ready    <none>   41h   v1.30.5-gke.1014003
gke-rajie-core-default-pool-808de050-g74t   Ready    <none>   41h   v1.30.5-gke.1014003
gke-rajie-core-default-pool-808de050-q6sh   Ready    <none>   41h   v1.30.5-gke.1014003
➜  ~ kubectl -f ingress-jwt-auth.yaml
error: unknown shorthand flag: 'f' in -f
See 'kubectl --help' for usage.
➜  ~ kubectl apply -f ingress-jwt-auth.yaml
requestauthentication.security.istio.io/ingress-jwt-auth created
➜  ~ kubectl apply -f deny-empty-jwt.yaml  
Warning: configured AuthorizationPolicy will deny all traffic to TCP ports under its scope due to the use of only HTTP attributes in a DENY rule; it is recommended to explicitly specify the port
authorizationpolicy.security.istio.io/deny-empty-jwt created
➜  ~  seldon model infer iris --inference-host 35.204.190.15:80 \
  '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
Error: Post "http://35.204.190.15:80/v2/models/iris/infer": dial tcp 35.204.190.15:80: connect: connection refused
➜  ~ seldon model infer iris --inference-host 34.141.255.75:80 \
  '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
{
	"model_name": "iris_1",
	"model_version": "1",
	"id": "08b1910a-ed60-48eb-b034-1720d2c3d645",
	"parameters": {},
	"outputs": [
		{
			"name": "predict",
			"shape": [
				1,
				1
			],
			"datatype": "INT64",
			"parameters": {
				"content_type": "np"
			},
			"data": [
				2
			]
		}
	]
}
➜  ~  curl -i http://34.141.255.75/v2/models/iris/infer \
 -H "Content-Type: application/json" \
 -H "seldon-model":iris \
 -d '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
HTTP/1.1 200 OK
ce-endpoint: iris_1
ce-id: c9045202-6bad-466a-bf06-7c6eb4c85f88
ce-inferenceservicename: mlserver
ce-modelid: iris_1
ce-namespace: seldon-mesh
ce-requestid: c9045202-6bad-466a-bf06-7c6eb4c85f88
ce-source: io.seldon.serving.deployment.mlserver.seldon-mesh
ce-specversion: 0.3
ce-type: io.seldon.serving.inference.response
content-length: 213
content-type: application/json
date: Fri, 29 Nov 2024 03:56:10 GMT
server: envoy
x-request-id: ct4jmmjc2nks739innrg
x-envoy-upstream-service-time: 8
x-seldon-route: :iris_1:

{"model_name":"iris_1","model_version":"1","id":"c9045202-6bad-466a-bf06-7c6eb4c85f88","parameters":{},"outputs":[{"name":"predict","shape":[1,1],"datatype":"INT64","parameters":{"content_type":"np"},"data":[2]}]}%                                                                                               ➜  ~  curl -i http://35.204.190.15/v2/models/iris/infer \
 -H "Content-Type: application/json" \
 -H "seldon-model":iris \
 -d '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
curl: (7) Failed to connect to 35.204.190.15 port 80 after 191 ms: Couldn't connect to server
➜  ~ seldon model infer iris --inference-host 34.141.255.75:80 \
  '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
{
	"model_name": "iris_1",
	"model_version": "1",
	"id": "a76e4ea9-9138-4c26-bf5c-7844992e155b",
	"parameters": {},
	"outputs": [
		{
			"name": "predict",
			"shape": [
				1,
				1
			],
			"datatype": "INT64",
			"parameters": {
				"content_type": "np"
			},
			"data": [
				2
			]
		}
	]
}
➜  ~ kubectl apply -f virtual-service.yaml 
virtualservice.networking.istio.io/iris-route created
➜  ~ seldon model infer iris --inference-host 34.141.255.75:80 \
  '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
{
	"model_name": "iris_1",
	"model_version": "1",
	"id": "d13e5568-1f43-48af-8bf3-c527334a7b6d",
	"parameters": {},
	"outputs": [
		{
			"name": "predict",
			"shape": [
				1,
				1
			],
			"datatype": "INT64",
			"parameters": {
				"content_type": "np"
			},
			"data": [
				2
			]
		}
	]
}
➜  ~ seldon model infer iris --inference-host 35.204.190.15:80 \                 
  '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
Error: Post "http://35.204.190.15:80/v2/models/iris/infer": dial tcp 35.204.190.15:80: connect: connection refused
➜  ~ seldon model infer iris --inference-host 34.141.255.75:80 \
  '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
{
	"model_name": "iris_1",
	"model_version": "1",
	"id": "11eae1dc-dda5-4b22-92b0-1af20bd74d6f",
	"parameters": {},
	"outputs": [
		{
			"name": "predict",
			"shape": [
				1,
				1
			],
			"datatype": "INT64",
			"parameters": {
				"content_type": "np"
			},
			"data": [
				2
			]
		}
	]
}
➜  ~ curl -i http://34.141.255.75/v2/models/iris/infer \
 -H "Content-Type: application/json" \
 -H "seldon-model":iris \
 -d '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'

HTTP/1.1 200 OK
ce-endpoint: iris_1
ce-id: 41dea464-a171-4dda-aeb7-682603e9fa59
ce-inferenceservicename: mlserver
ce-modelid: iris_1
ce-namespace: seldon-mesh
ce-requestid: 41dea464-a171-4dda-aeb7-682603e9fa59
ce-source: io.seldon.serving.deployment.mlserver.seldon-mesh
ce-specversion: 0.3
ce-type: io.seldon.serving.inference.response
content-length: 213
content-type: application/json
date: Fri, 29 Nov 2024 07:17:18 GMT
server: envoy
x-request-id: ct4mkvrc2nks739inntg
x-envoy-upstream-service-time: 9
x-seldon-route: :iris_1:

{"model_name":"iris_1","model_version":"1","id":"41dea464-a171-4dda-aeb7-682603e9fa59","parameters":{},"outputs":[{"name":"predict","shape":[1,1],"datatype":"INT64","parameters":{"content_type":"np"},"data":[2]}]}%                                                                                               ➜  ~ kubectl get svc istio-ingressgateway -n istio-system
NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                                      AGE
istio-ingressgateway   LoadBalancer   34.118.224.41   35.204.190.15   15021:31959/TCP,80:32567/TCP,443:32475/TCP   41h
➜  ~ kubectl apply -f istio-seldon-gateway.yaml
gateway.networking.istio.io/seldon-gateway created
➜  ~ curl -i http://34.141.255.75/v2/models/iris/infer \ 
 -H "Content-Type: application/json" \
 -H "seldon-model":iris \
 -d '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'

HTTP/1.1 200 OK
ce-endpoint: iris_1
ce-id: e380fbca-55c4-4cb7-b1cd-57576346248d
ce-inferenceservicename: mlserver
ce-modelid: iris_1
ce-namespace: seldon-mesh
ce-requestid: e380fbca-55c4-4cb7-b1cd-57576346248d
ce-source: io.seldon.serving.deployment.mlserver.seldon-mesh
ce-specversion: 0.3
ce-type: io.seldon.serving.inference.response
content-length: 213
content-type: application/json
date: Fri, 29 Nov 2024 07:21:47 GMT
server: envoy
x-request-id: ct4mn2rc2nks739innu0
x-envoy-upstream-service-time: 8
x-seldon-route: :iris_1:

{"model_name":"iris_1","model_version":"1","id":"e380fbca-55c4-4cb7-b1cd-57576346248d","parameters":{},"outputs":[{"name":"predict","shape":[1,1],"datatype":"INT64","parameters":{"content_type":"np"},"data":[2]}]}%                                                                                               ➜  ~ curl -i http://35.204.190.15/v2/models/iris/infer \
 -H "Content-Type: application/json" \
 -H "seldon-model":iris \
 -d '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'

HTTP/1.1 403 Forbidden
content-length: 19
content-type: text/plain
date: Fri, 29 Nov 2024 07:22:34 GMT
server: istio-envoy
connection: close

RBAC: access denied% ```

[paulb-seldon]

Looks good, thank you for this work, and testing it out after the initial documentation!

[Rajakavitha1]

merging after confirming with @paulb-seldon