Sometimes the api-pod does not respond any more

[FamilieRo]

From time to time the api pod stops to respond. So everything like attaching volumes, cli etc. gets a timeout.
e.g.
kubectl mayastor get volumes -n openebs
Failed to list volumes. Error error in request: request timed out

In the logs of the api pod you will find something like

.[2m2024-12-19T13:08:39.383948Z.[0m .[31mERROR.[0m .[1;31mgrpc::operations::volume::client.[0m.[31m: .[1;31merror.[0m.[31m: Unavailable: status: Unavailable, message: "error trying to connect: deadline has elapsed", details: [], metadata: MetadataMap { headers: {} }.[0m

I have attached a full log file for you to analyse.

Removing the api pod brings up a new one and everything works directly.

The error occurs regardless of whether workload happens on the cluster or not.

openebs-api-rest-676998984d-svxhr-api-rest.log

[tiagolobocastro]

Could you please try to increase the cpu limits for the rest just in case?
Example:
mayastor.apis.rest.resources.limits.cpu=500m

And could you also share logs from agents-core pod, agent-core container?

[FamilieRo]

I have increased the cpu limit. At the moment no more timeouts. I will observe the behaviour.

However, I'm on holiday for a few days and will upload more logs if necessary.