fix(log): add custom error handler for Kubernetes API errors

[ritwikranjan]

Description

This pull request includes changes to improve error handling and logging in the Kubernetes watcher and to simplify error checking in the endpoint reconciler. The most important changes include adding a custom error handler for the Kubernetes watcher, importing necessary packages, and simplifying error handling logic.

Improvements to error handling and logging:

• pkg/k8s/watcher_linux.go: Added a custom error handler retinaK8sErrorHandler to log specific Kubernetes API server errors and tag them for easier identification.
• pkg/k8s/watcher_linux.go: Registered the custom error handler in the init function to ensure it is used by the watcher.

Code simplification:

• pkg/controllers/operator/cilium-crds/endpoint/endpoint_controller.go: Simplified the error checking logic in handlePodUpsert by removing redundant error check.

Checklist

• I have read the contributing documentation.
• I signed and signed-off the commits (git commit -S -s ...). See this documentation on signing commits.
• I have correctly attributed the author(s) of the code.
• I have tested the changes locally.
• I have followed the project's style guidelines.
• I have updated the documentation, if necessary.
• I have added tests, if applicable.

Testing

I removed permission for retina agent to read nodes and services. I can see the completer error as as our custom message coming from retina.

time="2024-11-26T16:05:33Z" level=error msg="Potentially Network Error coming from K8s API Server failing to watch Services" actualError="pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:232: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User \"system:serviceaccount:kube-system:retina-agent\" cannot list resource \"services\" in API group \"\" at the cluster scope" subsys=k8s-watcher

---

Please refer to the CONTRIBUTING.md file for more information on how to contribute to this project.