agent and apollo crashed

[cometta]

these two pods keep on crashing, the rest of the pods are fine. May i know is this path http://<name_here>-prefect-ui-hasura:3000/v1alpha1/graphql suppose to return result? i checked the port 3000 is open in hasura but this path is no accessible

wget: server returned error: HTTP/1.1 404 Not Found

[szelenka]

From what I understand, the hasura service feeds data to apollo, and the agent needs apollo running for it to query the database to see if any Flows are pending execution.

Doing a wget GET to http://<name_here>-prefect-ui-hasura:3000/v1alpha1/graphql should return a 404, I believe this is expecting a GraphQL query through a HTTP POST message.

Is it possible to view the startup logs of your Apollo Pod?

[cometta]

@szelenka below is the output logs in apollo pod

-staging-prefect-ui-apollo-797b7c46cb-q998b --namespace prefectio

> @ serve /apollo
> node dist/index.js

2020-06-23T00:21:40.628Z Building schema...

you are right about the POST for 3000.

Warning  Unhealthy  9h (x4 over 9h)      kubelet, docker-desktop  Readiness probe failed: dial tcp 10.1.0.....:4200: connect: connection refused

[szelenka]

In this case, the Pod is not starting because the Apollo service isn't responding to the readiness probe. You could try disabling the Readiness probe to investigate what's going on.

The logs seem to imply it's in the middle of building the schema, which takes awhile on first run. One thing to check would be to ensure enough resources are allocated to the Pod (CPU & memory) so it can execute the steps before the Readiness probes fail and terminate the Pod.