Upgrades: Golang 1.19, Knative v1.9, K8s v1.25.3, Istio 1.16.0, metallb 0.3.19 #667

anshalshukla · 2023-02-11T16:58:14Z

List of upgrades:

Golang to 1.19
Knative to 1.9
k8s to 1.25
Istio to 1.16
metallb to 0.3.19

WARNING: After this PR, vHive is incompatible with k8s <1.24

leokondrashov · 2023-02-14T10:51:00Z

I've ran the PR branch through the vHive quickstart guide. Looks fine, in the end invoker runs as expected.

Should there be any other checks before merge?

scripts/cluster/setup_master_node.sh

docs/developers_guide.md

CHANGELOG.md

scripts/cluster/setup_master_node.sh

ustiugov · 2023-02-20T10:18:50Z

@anshalshukla please fix the linter checks. also, please see the error that poped up in the CRI test.

scripts/github_runner/Dockerfile.cri_dev_env

scripts/cluster/setup_master_node.sh

scripts/setup_zipkin.sh

ustiugov · 2023-02-23T06:57:26Z

@anshalshukla please investigate the failing unit test (it probably also causes the CRI tests to fail). Also, please squash your commits that belong to the same logical changes into one using git rebase

cvetkovic

LGTM!

anshalshukla · 2023-02-28T11:27:56Z

While trying to test CRI test using this. I face the following error while running create_one_node_cluster.sh

Istio has been successfully downloaded into the istio-1.16.0 folder on your system.

Next Steps:
See https://istio.io/latest/docs/setup/install/ to add Istio to your Kubernetes cluster.

To configure the istioctl client tool for your workstation,
add the /users/anshal/vHive/istio-1.16.0/bin directory to your environment path variable with:
	 export PATH="$PATH:/users/anshal/vHive/istio-1.16.0/bin"

Begin the Istio pre-installation check by running:
	 istioctl x precheck 

Need more information? Visit https://istio.io/latest/docs/setup/install/ 
! values.global.jwtPolicy is deprecated; use Values.global.jwtPolicy=third-party-jwt. See https://istio.io/latest/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for more information instead
✔ Istio core installed                                                          
✘ Istiod encountered an error: failed to wait for resource: resources not ready after 5m0s: timed out waiting for the condition
  
✘ Ingress gateways encountered an error: failed to wait for resource: resources not ready after 5m0s: timed out waiting for the condition
  
  
- Pruning removed resources                                                     Error: failed to install manifests: errors occurred during operation
customresourcedefinition.apiextensions.k8s.io/certificates.networking.internal.knative.dev created
customresourcedefinition.apiextensions.k8s.io/configurations.serving.knative.dev created
customresourcedefinition.apiextensions.k8s.io/clusterdomainclaims.networking.internal.knative.dev created
customresourcedefinition.apiextensions.k8s.io/domainmappings.serving.knative.dev created
customresourcedefinition.apiextensions.k8s.io/ingresses.networking.internal.knative.dev created
customresourcedefinition.apiextensions.k8s.io/metrics.autoscaling.internal.knative.dev created
customresourcedefinition.apiextensions.k8s.io/podautoscalers.autoscaling.internal.knative.dev created
customresourcedefinition.apiextensions.k8s.io/revisions.serving.knative.dev created
customresourcedefinition.apiextensions.k8s.io/routes.serving.knative.dev created
customresourcedefinition.apiextensions.k8s.io/serverlessservices.networking.internal.knative.dev created
customresourcedefinition.apiextensions.k8s.io/services.serving.knative.dev created
customresourcedefinition.apiextensions.k8s.io/images.caching.internal.knative.dev created
namespace/knative-serving created

The command eventually ends with a pending external IP for istio-ingressgateway

service/broker-filter created
deployment.apps/mt-broker-ingress created
service/broker-ingress created
deployment.apps/mt-broker-controller created
horizontalpodautoscaler.autoscaling/broker-ingress-hpa created
horizontalpodautoscaler.autoscaling/broker-filter-hpa created
NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                      AGE
istio-ingressgateway   LoadBalancer   10.110.66.203   <pending>     15021:31083/TCP,80:31577/TCP,443:30548/TCP   5m15s

While this is not the case when the CRI test suite runs via the GitHub actions. See this.

On searching for the possible reasons I found a couple of possible reasons for this behaviour.

Lack of RAM. See this
Failed scheduling. See this

I think the problem is with the scheduling because on running kubectl get pods -A. The response shows pods as pending:

NAMESPACE          NAME                                                                          READY   STATUS    RESTARTS   AGE
istio-system       cluster-local-gateway-fffb9f589-xrvk6                                         0/1     Pending   0          32m
istio-system       istio-ingressgateway-778db64bb6-7w5dw                                         0/1     Pending   0          32m
istio-system       istiod-85bf857c79-2pxjt                                                       0/1     Pending   0          37m
knative-eventing   eventing-controller-6b5b744bfd-t86rt                                          0/1     Pending   0          27m
knative-eventing   eventing-webhook-75cdd7c68-fs9k4                                              0/1     Pending   0          27m
knative-eventing   imc-controller-565df566f8-qgvv7                                               0/1     Pending   0          27m
knative-eventing   imc-dispatcher-5bf6c7d945-mxgxq                                               0/1     Pending   0          27m
knative-eventing   mt-broker-controller-575d4c9f77-qm852                                         0/1     Pending   0          27m
knative-eventing   mt-broker-filter-746ddf5785-svsb8                                             0/1     Pending   0          27m
knative-eventing   mt-broker-ingress-7bff548b5b-hv4ll                                            0/1     Pending   0          27m
knative-serving    activator-74ffb7c9b6-js29l                                                    0/1     Pending   0          27m
knative-serving    autoscaler-8586b7f7df-9727m                                                   0/1     Pending   0          27m
knative-serving    controller-5d4fcf8bcf-fhcwb                                                   0/1     Pending   0          27m
knative-serving    default-domain-tmqth                                                          0/1     Pending   0          27m
knative-serving    domain-mapping-55d685d97b-62mn2                                               0/1     Pending   0          27m
knative-serving    domainmapping-webhook-5c5b5979cb-v5lhg                                        0/1     Pending   0          27m
knative-serving    net-istio-controller-777b6b4d89-56rct                                         0/1     Pending   0          27m
knative-serving    net-istio-webhook-78665d59fd-dt6b5                                            0/1     Pending   0          27m
knative-serving    webhook-6894666d8-qvjnh                                                       0/1     Pending   0          27m
kube-system        calico-kube-controllers-6f4474fd5b-wv6dl                                      0/1     Pending   0          37m
kube-system        canal-cmlct                                                                   2/2     Running   0          37m
kube-system        coredns-565d847f94-22w27                                                      1/1     Running   0          37m
kube-system        coredns-565d847f94-s8sh8                                                      1/1     Running   0          37m
kube-system        etcd-node-1.anshal-149188.ntu-cloud-pg0.utah.cloudlab.us                      1/1     Running   0          37m
kube-system        kube-apiserver-node-1.anshal-149188.ntu-cloud-pg0.utah.cloudlab.us            1/1     Running   0          37m
kube-system        kube-controller-manager-node-1.anshal-149188.ntu-cloud-pg0.utah.cloudlab.us   1/1     Running   0          37m
kube-system        kube-proxy-qbw5b                                                              1/1     Running   0          37m
kube-system        kube-scheduler-node-1.anshal-149188.ntu-cloud-pg0.utah.cloudlab.us            1/1     Running   0          37m
metallb-system     controller-588f6c844d-s5f44                                                   0/1     Pending   0          37m
registry           docker-registry-pod-7k8rc                                                     0/1     Pending   0          27m

Also if I run kubectl taint --all node-role.kubernetes.io/control-plane:NoSchedule- and wait for a few minutes, eventualy all the pods get to running state. Then to proceed to next setup of setting up local-registry I had to install go, which is weird because I had used go build to compile vhive. Even after installing go the go run ./examples/registry/populate_registry.go -imageFile ./examples/registry/images.txt fails with the following error:

INFO[0000] Reading the images from the file: ./examples/registry/images.txt 
INFO[0000] Pulling images from docker://docker.io to docker://docker-registry.registry.svc.cluster.local:5000 
INFO[0000] Pulling vhiveease/helloworld:var_workload    
FATA[0000] Failed to pull image vhiveease/helloworld:var_workload: exec: "skopeo": executable file not found in $PATH

Then I tried deploying functions which was successful. Here it again differs with the GitHub actions workflow, which failed while function deployment. Eventually on running the cri tests it failed with the following log:

go: downloading github.com/stretchr/testify v1.8.0
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading gopkg.in/yaml.v3 v3.0.1
=== RUN   TestSingleInvoke
--- PASS: TestSingleInvoke (3.05s)
=== RUN   TestSingleInvokeLocal
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:61
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestSingleInvokeLocal
        	Messages:   	Failed to get response from function
--- FAIL: TestSingleInvokeLocal (0.03s)
=== RUN   TestParallelInvoke
--- PASS: TestParallelInvoke (0.13s)
=== RUN   TestParallelInvokeLocal
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocal
        	Messages:   	Failed to get response from function
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:178
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestParallelInvokeLocalua
        	Messages:   	Failed to get response from function
--- FAIL: TestParallelInvokeLocal (0.19s)
=== RUN   TestAutoscaler
=== RUN   TestAutoscaler/Scale_fn_with_concurrency_1
=== RUN   TestAutoscaler/Scale_from_0
--- PASS: TestAutoscaler (206.28s)
    --- PASS: TestAutoscaler/Scale_fn_with_concurrency_1 (3.28s)
    --- PASS: TestAutoscaler/Scale_from_0 (203.00s)
=== RUN   TestMultipleFuncInvoke
    cri_test.go:166: 
        	Error Trace:	/users/anshal/vHive/cri/cri_test.go:166
        	            				/users/anshal/vHive/cri/cri_test.go:131
        	            				/users/anshal/vHive/cri/asm_amd64.s:1571
        	Error:      	Received unexpected error:
        	            	rpc error: code = Unimplemented desc = 
        	Test:       	TestMultipleFuncInvoke
        	Messages:   	Failed to get response from function
--- FAIL: TestMultipleFuncInvoke (3.09s)
=== RUN   TestBench
First invocation took 1026 ms
Second invocation took 8 ms
--- PASS: TestBench (1.17s)
FAIL
coverage: 0.0% of statements
FAIL	github.com/vhive-serverless/vhive/cri	213.975s
FAIL

Studying the logs of local testing and GitHub actions I'm unable to pin point the problem. Logs of actions suggest that there is a setup problem with k8s or kn because of which we are unable to deploy functions but locally the k8s and kn seem to be working fine without any error.

I tried the same steps with the main branch and although the istio setup completed properly eventually the cri-tests fails with mostly the same errors.

Signed-off-by: Anshal Shukla <[email protected]>

anshalshukla · 2023-04-18T10:07:12Z

function-images/tests/save_load_minio/server/go.mod


 replace github.com/vhive-serverless/vhive/function-images/tests/save_load_minio/proto => ../proto_gen

 require (
-	github.com/containerd/containerd v1.4.3
-	github.com/vhive-serverless/vhive/function-images/tests/save_load_minio/proto v0.0.0-00010101000000-000000000000
+	github.com/containerd/containerd v1.6.20


Can we upgrade this to 1.6.20 without the firecracker-containerd upgrade?

and refactored deprecated API of the dependencies (io/utils) Signed-off-by: Dmitrii Ustiugov <[email protected]>

Signed-off-by: Dmitrii Ustiugov <[email protected]>

ustiugov

ready to merge. Thanks a lot to @anshalshukla and @CuriousGeorgiy and the team.

anshalshukla mentioned this pull request Feb 11, 2023

Upgrade Knative to v1.9 #661

Closed

cvetkovic requested changes Feb 16, 2023

View reviewed changes

scripts/cluster/setup_master_node.sh Outdated Show resolved Hide resolved

docs/developers_guide.md Outdated Show resolved Hide resolved

CHANGELOG.md Outdated Show resolved Hide resolved

cvetkovic requested changes Feb 16, 2023

View reviewed changes

scripts/cluster/setup_master_node.sh Show resolved Hide resolved

anshalshukla requested a review from cvetkovic February 17, 2023 14:35

anshalshukla changed the title ~~Knative v1.9, K8s v1.24.0 update~~ Knative v1.9, K8s v1.25.6, Istio 1.16.0 update Feb 17, 2023

ustiugov force-pushed the release-1.9 branch from 50648bd to 542a704 Compare February 20, 2023 10:09

cvetkovic requested changes Feb 20, 2023

View reviewed changes

scripts/github_runner/Dockerfile.cri_dev_env Outdated Show resolved Hide resolved

scripts/cluster/setup_master_node.sh Show resolved Hide resolved

cvetkovic requested changes Feb 20, 2023

View reviewed changes

scripts/setup_zipkin.sh Outdated Show resolved Hide resolved

anshalshukla force-pushed the release-1.9 branch 2 times, most recently from c6eb9c0 to 130ba9f Compare February 23, 2023 06:08

anshalshukla changed the title ~~Knative v1.9, K8s v1.25.6, Istio 1.16.0 update~~ Knative v1.9, K8s v1.25.3, Istio 1.16.0 update Feb 23, 2023

anshalshukla requested review from cvetkovic and leokondrashov February 23, 2023 07:41

cvetkovic previously approved these changes Feb 23, 2023

View reviewed changes

ustiugov force-pushed the release-1.9 branch from 130ba9f to e91a817 Compare February 23, 2023 13:05

anshalshukla dismissed cvetkovic’s stale review via f984ded February 23, 2023 14:11

anshalshukla force-pushed the release-1.9 branch 3 times, most recently from f9018b0 to eed1a73 Compare February 25, 2023 15:57

ustiugov force-pushed the release-1.9 branch from 5e0a7c4 to 79dc5cc Compare February 27, 2023 03:51

anshalshukla force-pushed the release-1.9 branch from 0dbcc0a to 9dde372 Compare February 28, 2023 04:14

anshalshukla force-pushed the release-1.9 branch 4 times, most recently from a935877 to e8f53b1 Compare March 1, 2023 20:54

anshalshukla force-pushed the release-1.9 branch from a3724b8 to 2a63d80 Compare March 20, 2023 18:00

ustiugov force-pushed the release-1.9 branch from 3b14ceb to ef9020b Compare April 13, 2023 05:00

anshalshukla force-pushed the release-1.9 branch 3 times, most recently from 61b7dbf to e65fffc Compare April 15, 2023 09:09

anshalshukla added 7 commits April 15, 2023 03:15

Knative upgrade to v1.9

3cc1cd5

Signed-off-by: Anshal Shukla <[email protected]>

Upgrade k8s to v1.25.3 and istio to v1.16.0

ae6a5f1

Signed-off-by: Anshal Shukla <[email protected]>

Taint control plane to allow pod scheduling

f7c8fef

Signed-off-by: Anshal Shukla <[email protected]>

Update metallb version and calico config

62b8f02

Signed-off-by: Anshal Shukla <[email protected]>

Update changelog

78760a0

Signed-off-by: Anshal Shukla <[email protected]>

Move knative-serving images to ghcr.io

1d21f41

Signed-off-by: Anshal Shukla <[email protected]>

Fix linting errors

ee098e2

Signed-off-by: Anshal Shukla <[email protected]>

anshalshukla force-pushed the release-1.9 branch from e65fffc to babb981 Compare April 15, 2023 09:16

ustiugov force-pushed the release-1.9 branch 8 times, most recently from 23cd1fe to c943da8 Compare April 18, 2023 09:06

ustiugov changed the title ~~Knative v1.9, K8s v1.25.3, Istio 1.16.0, metallb 0.3.19 update~~ Upgrades: Golang 1.19, Knative v1.9, K8s v1.25.3, Istio 1.16.0, metallb 0.3.19 Apr 18, 2023

ustiugov force-pushed the release-1.9 branch 2 times, most recently from 6900aed to 8f6bddf Compare April 18, 2023 09:51

anshalshukla commented Apr 18, 2023

View reviewed changes

anshalshukla and others added 2 commits April 18, 2023 19:25

Updated to Golang 1.19

1952e55

and refactored deprecated API of the dependencies (io/utils) Signed-off-by: Dmitrii Ustiugov <[email protected]>

updated changelog for v1.5

4baa063

Signed-off-by: Dmitrii Ustiugov <[email protected]>

ustiugov force-pushed the release-1.9 branch from 8f6bddf to 4baa063 Compare April 18, 2023 11:26

updated CI badges

fd7edb4

Signed-off-by: Dmitrii Ustiugov <[email protected]>

ustiugov approved these changes Apr 18, 2023

View reviewed changes

ustiugov merged commit ead2d63 into vhive-serverless:main Apr 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrades: Golang 1.19, Knative v1.9, K8s v1.25.3, Istio 1.16.0, metallb 0.3.19 #667

Upgrades: Golang 1.19, Knative v1.9, K8s v1.25.3, Istio 1.16.0, metallb 0.3.19 #667

anshalshukla commented Feb 11, 2023 •

edited by ustiugov

Loading

leokondrashov commented Feb 14, 2023

ustiugov commented Feb 20, 2023

ustiugov commented Feb 23, 2023

cvetkovic left a comment

anshalshukla commented Feb 28, 2023 •

edited

Loading

anshalshukla Apr 18, 2023

ustiugov left a comment

Upgrades: Golang 1.19, Knative v1.9, K8s v1.25.3, Istio 1.16.0, metallb 0.3.19 #667

Upgrades: Golang 1.19, Knative v1.9, K8s v1.25.3, Istio 1.16.0, metallb 0.3.19 #667

Conversation

anshalshukla commented Feb 11, 2023 • edited by ustiugov Loading

leokondrashov commented Feb 14, 2023

ustiugov commented Feb 20, 2023

ustiugov commented Feb 23, 2023

cvetkovic left a comment

Choose a reason for hiding this comment

anshalshukla commented Feb 28, 2023 • edited Loading

anshalshukla Apr 18, 2023

Choose a reason for hiding this comment

ustiugov left a comment

Choose a reason for hiding this comment

anshalshukla commented Feb 11, 2023 •

edited by ustiugov

Loading

anshalshukla commented Feb 28, 2023 •

edited

Loading