Automatically inject expected ODH annotations to InferenceGraph and InferenceServices

[israel-hdez]

NOTE: Requires PR #324 before merging (fulfilled)

Description

On creation of InferenceGraph or InferenceService resources, the following default annotations will be added:

• serving.knative.openshift.io/enablePassthrough: true
• sidecar.istio.io/inject: true
• sidecar.istio.io/rewriteAppHTTPProbers: true

The annotations are added only for Serverless mode, and only if they are missing.

Fixes https://issues.redhat.com/browse/RHOAIENG-15337

How Has This Been Tested?

• Deploy odh-model-controller from this PR, either by creating a custom build of the operator, or by using dev-flags
• Follow the InferenceGraph example at https://kserve.github.io/website/latest/modelserving/inference_graph/image_pipeline/#deploy-the-inferenceservices (you will need to configure pytorch runtime)
• When you create the InferenceGraph resource, confirm that the three default annotations (mentioned up) are automatically injected to the created InferenceGraph resource.
• Delete the InferenceGraph resource, and re-create it in Raw mode. Confirm that the default annotations are not injected.
• Delete the InferenceGraph resource, and re-create it in Serverless mode, while manually specifying one (or some) of the annotations with a non-default value (e.g. serving.knative.openshift.io/enablePassthrough: false). Confirm that the manually specified annotations are honored, while the others are injected to the default values.

Similar tests can be performed on InferenceService resources.

Merge criteria:

• The commits are squashed in a cohesive manner and have meaningful messages.
• Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has manually tested the changes and verified that the changes work

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[israel-hdez]

@spolti This PR is now fully ready, although we need first to merge the migration #324

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: israel-hdez, spolti

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [israel-hdez,spolti]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[spolti]

/lgtm