Skip to content

Commit

Permalink
CNF-11559: [Part 3/3] Hypershift PAO adoption (#1057)
Browse files Browse the repository at this point in the history 
* pao:hypershift: do not add mco scheme

On hypershift we don't have mco, trying to add it will cause for exit
with an error

* pao:hypershift: implement the components handler interface

This is the core of the reconciliation loop, which creates/updates/deletes the relevant components.
Similar to what we already have on OCP.

There's a gap about how to retrieve the `containerruntimeconfig` object
which is needed by the controller and is not accessible in the hosted
control plane namespace. This will be handle on a separate PR.

Signed-off-by: Talor Itzhak <[email protected]>

* pao:hypershift: stub implementation for status writer interface

status reporting is not supported on hypershift yet.
we provide a fake implementation in order to conforms the interface.

Signed-off-by: Talor Itzhak <[email protected]>

* pao:hypershift: check if in hypershift first

Checking the feature gate first will result with an error on hypershift since it's not yet supported.

Signed-off-by: Talor Itzhak <[email protected]>

* pao:hypershift: add management client

Adding a management client to be used on a hypershift platform.

Signed-off-by: Talor Itzhak <[email protected]>

* pao:hypershift: configure controller

Add the necessary configuration for the controller, so it would be able to run on a hypershift platform.

1. It needs to configure predicates for ConfigMaps instead of actual objects.
2. It needs to setup different clients for the management cluster and hosted cluster.

Signed-off-by: Talor Itzhak <[email protected]>

* pao:hypershift: generelize reconciliation loop

Modify the reconciliation flow so it could fit for both HCP and OCP.

Signed-off-by: Talor Itzhak <[email protected]>

* e2e:config: adapt suite for hypershift

Adding the relevant code for configuring performanceProfile over hypershift

Signed-off-by: Talor Itzhak <[email protected]>

* hypershift:client: do not assign to nil map

Signed-off-by: Talor Itzhak <[email protected]>

* hypershift:client: return error when configMap has invalid data state

Signed-off-by: Talor Itzhak <[email protected]>

* pao:hypershift: move label validation to helper function

Signed-off-by: Talor Itzhak <[email protected]>

* vendor: bump deps

Bump needed for handling hypershift APIs.
This is the output formatted and generated by the \`go mod tidy\`

Signed-off-by: Talor Itzhak <[email protected]>

* vendor:featuregate: adopt feature gate api changes

The package that holds featuregate geatures has changed in the last vendor bump.
This commit uses the new paths.

Signed-off-by: Talor Itzhak <[email protected]>

* pao:log: small refining

- Move logs to V(4)
- Fix typos
- Remove redundant spacing
- Consistent HyperShift naming

Signed-off-by: Talor Itzhak <[email protected]>

---------

Signed-off-by: Talor Itzhak <[email protected]>
  • Loading branch information
@Tal-or
Tal-or authored Jun 9, 2024
1 parent d9adde3 commit 4d6cb12
Show file tree
Hide file tree
Showing 232 changed files with 29,747 additions and 5,852 deletions.
1 change: 1 addition & 0 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,7 @@ pao-functests-mixedcpus:
pao-functests-hypershift:
@echo "Cluster Version"
hack/show-cluster-version.sh
hack/run-test.sh -t "./test/e2e/performanceprofile/functests/0_config" -p "-vv -r --fail-fast --flake-attempts=2 --junit-report=report.xml" -m "Running Functional Tests over Hypershift"

.PHONY: cluster-clean-pao
cluster-clean-pao:
Expand Down
52 changes: 51 additions & 1 deletion cmd/cluster-node-tuning-operator/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (
"github.com/spf13/pflag"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/meta"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
apiruntime "k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
Expand All @@ -34,17 +35,23 @@ import (
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
"k8s.io/client-go/rest"
"k8s.io/klog/v2"
"net/http"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/cache"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/cluster"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
"sigs.k8s.io/controller-runtime/pkg/webhook"

tunedv1 "github.com/openshift/cluster-node-tuning-operator/pkg/apis/tuned/v1"
ntoclient "github.com/openshift/cluster-node-tuning-operator/pkg/client"
"github.com/openshift/cluster-node-tuning-operator/pkg/config"
"github.com/openshift/cluster-node-tuning-operator/pkg/metrics"
"github.com/openshift/cluster-node-tuning-operator/pkg/operator"
"github.com/openshift/cluster-node-tuning-operator/pkg/performanceprofile/cmd/render"
"github.com/openshift/cluster-node-tuning-operator/pkg/performanceprofile/controller/performanceprofile/hypershift"
hcpcomponents "github.com/openshift/cluster-node-tuning-operator/pkg/performanceprofile/controller/performanceprofile/hypershift/components"
hcpstatus "github.com/openshift/cluster-node-tuning-operator/pkg/performanceprofile/controller/performanceprofile/hypershift/status"
"github.com/openshift/cluster-node-tuning-operator/pkg/tuned/cmd/operand"
tunedrender "github.com/openshift/cluster-node-tuning-operator/pkg/tuned/cmd/render"
"github.com/openshift/cluster-node-tuning-operator/pkg/util"
Expand All @@ -65,9 +72,11 @@ var (
func init() {
ctrl.SetLogger(zap.New())

if !config.InHyperShift() {
utilruntime.Must(mcov1.AddToScheme(scheme))
}
utilruntime.Must(clientgoscheme.AddToScheme(scheme))
utilruntime.Must(tunedv1.AddToScheme(scheme))
utilruntime.Must(mcov1.AddToScheme(scheme))
utilruntime.Must(apiconfigv1.Install(scheme))
utilruntime.Must(performancev1alpha1.AddToScheme(scheme))
utilruntime.Must(performancev1.AddToScheme(scheme))
Expand Down Expand Up @@ -179,6 +188,7 @@ func operatorRun() {
}
if err = (&paocontroller.PerformanceProfileReconciler{
Client: mgr.GetClient(),
ManagementClient: mgr.GetClient(),
Recorder: mgr.GetEventRecorderFor("performance-profile-controller"),
FeatureGate: fg,
ComponentsHandler: handler.NewHandler(mgr.GetClient(), mgr.GetScheme()),
Expand All @@ -194,6 +204,46 @@ func operatorRun() {
if err = (&performancev2.PerformanceProfile{}).SetupWebhookWithManager(mgr); err != nil {
klog.Exitf("unable to create PerformanceProfile v2 webhook: %v", err)
}
} else {
// HyperShift configuration
restConfig, err := ntoclient.GetInClusterConfig()
if err != nil {
klog.Exitf("unable to create get InClusterConfiguration while creating PerformanceProfile controller: %v", err)
}

fOps := func(opts *cluster.Options) {
operatorNamespace := config.OperatorNamespace()
opts.Cache.Namespaces = []string{operatorNamespace}
opts.Scheme = mgr.GetScheme()
opts.MapperProvider = func(c *rest.Config, httpClient *http.Client) (meta.RESTMapper, error) {
return mgr.GetRESTMapper(), nil
}
opts.NewClient = func(config *rest.Config, options client.Options) (client.Client, error) {
c, err := client.New(config, options)
if err != nil {
return nil, err
}
return hypershift.NewControlPlaneClient(c, operatorNamespace), nil
}
}
managementCluster, err := cluster.New(restConfig, fOps)
if err != nil {
klog.Exitf("unable to create ManagementCluster while creating PerformanceProfile controller: %v", err)
}

if err := mgr.Add(managementCluster); err != nil {
klog.Exitf("unable to add ManagementCluster to manger while creating PerformanceProfile controller: %v", err)
}
if err = (&paocontroller.PerformanceProfileReconciler{
// dataPlaneClient
Client: mgr.GetClient(),
ManagementClient: managementCluster.GetClient(),
Recorder: managementCluster.GetEventRecorderFor("performance-profile-controller"),
ComponentsHandler: hcpcomponents.NewHandler(managementCluster.GetClient(), mgr.GetClient(), mgr.GetScheme()),
StatusWriter: hcpstatus.NewWriter(managementCluster.GetClient(), mgr.GetClient()),
}).SetupWithManagerForHypershift(mgr, managementCluster); err != nil {
klog.Exitf("unable to create PerformanceProfile controller: %v", err)
}
}
if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
klog.Exitf("manager exited with non-zero code: %v", err)
Expand Down
64 changes: 35 additions & 29 deletions go.mod
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
module github.com/openshift/cluster-node-tuning-operator

go 1.22
go 1.22.0

toolchain go1.22.4

require (
github.com/RHsyseng/operator-utils v0.0.0-20200213165520-1a022eb07a43
Expand All @@ -15,10 +17,11 @@ require (
github.com/onsi/gomega v1.33.0
github.com/openshift-kni/debug-tools v0.1.12
github.com/openshift-kni/k8sreporter v1.0.5
github.com/openshift/api v0.0.0-20240422085825-2624175e9673
github.com/openshift/api v0.0.0-20240530231226-9d1c2e5ff5a8
github.com/openshift/build-machinery-go v0.0.0-20231128094528-1e9b1b0595c8
github.com/openshift/client-go v0.0.0-20240415214935-be70f772f157
github.com/openshift/custom-resource-status v1.1.3-0.20220503160415-f2fdb4999d87
github.com/openshift/hypershift/api v0.0.0-20240604072534-cd2d5291e2b7
github.com/openshift/library-go v0.0.0-20240419113445-f1541d628746
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.19.0
Expand All @@ -27,17 +30,18 @@ require (
github.com/spf13/pflag v1.0.6-0.20210604193023-d5e0c0615ace
gopkg.in/fsnotify.v1 v1.4.7
gopkg.in/ini.v1 v1.67.0
k8s.io/api v0.29.4
k8s.io/api v0.30.1
k8s.io/apiextensions-apiserver v0.29.4
k8s.io/apimachinery v0.29.4
k8s.io/apimachinery v0.30.1
k8s.io/client-go v0.29.4
k8s.io/code-generator v0.29.4
k8s.io/klog v1.0.0
k8s.io/klog/v2 v2.120.1
k8s.io/kubelet v0.29.4
k8s.io/kubernetes v1.29.4
k8s.io/utils v0.0.0-20240310230437-4693a0247e57
k8s.io/utils v0.0.0-20240502163921-fe8a2dddb1d0
kubevirt.io/qe-tools v0.1.8
sigs.k8s.io/cluster-api v1.7.2
sigs.k8s.io/controller-runtime v0.17.3
sigs.k8s.io/controller-tools v0.11.3
sigs.k8s.io/yaml v1.4.0
Expand All @@ -50,9 +54,8 @@ require (
github.com/Microsoft/go-winio v0.6.0 // indirect
github.com/NYTimes/gziphandler v1.1.1 // indirect
github.com/StackExchange/wmi v1.2.1 // indirect
github.com/ajeddeloh/go-json v0.0.0-20200220154158-5ae607161559 // indirect
github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.0.0-20230305170008-8188dc5388df // indirect
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/blang/semver/v4 v4.0.0 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
Expand All @@ -72,12 +75,12 @@ require (
github.com/euank/go-kmsg-parser v2.0.0+incompatible // indirect
github.com/evanphx/json-patch v5.9.0+incompatible // indirect
github.com/evanphx/json-patch/v5 v5.9.0 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/frankban/quicktest v1.14.4 // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/ghodss/yaml v1.0.1-0.20190212211648-25d852aebe32 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/zapr v1.3.0 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-openapi/analysis v0.21.4 // indirect
Expand All @@ -90,21 +93,22 @@ require (
github.com/go-openapi/swag v0.23.0 // indirect
github.com/go-openapi/validate v0.22.0 // indirect
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
github.com/gobuffalo/flect v0.3.0 // indirect
github.com/gobuffalo/flect v1.0.2 // indirect
github.com/godbus/dbus/v5 v5.1.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/glog v1.2.1 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/btree v1.1.2 // indirect
github.com/google/cadvisor v0.49.1 // indirect
github.com/google/cel-go v0.17.7 // indirect
github.com/google/cel-go v0.18.2 // indirect
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/pprof v0.0.0-20240416155748-26353dc0451f // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.1 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.2 // indirect
github.com/imdario/mergo v0.3.16 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jaypipes/pcidb v1.0.0 // indirect
Expand All @@ -113,8 +117,7 @@ require (
github.com/karrick/godirwalk v1.17.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.17 // indirect
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
Expand All @@ -127,44 +130,45 @@ require (
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
github.com/oklog/ulid v1.3.1 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/opencontainers/runc v1.1.12 // indirect
github.com/opencontainers/runtime-spec v1.0.3-0.20220909204839-494a5a6aca78 // indirect
github.com/opencontainers/selinux v1.11.0 // indirect
github.com/prometheus/client_model v0.5.0 // indirect
github.com/prometheus/common v0.45.0 // indirect
github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/common v0.47.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/robfig/cron v1.2.0 // indirect
github.com/seccomp/libseccomp-golang v0.10.0 // indirect
github.com/stoewer/go-strcase v1.2.0 // indirect
github.com/stoewer/go-strcase v1.3.0 // indirect
github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635 // indirect
github.com/vincent-petithory/dataurl v1.0.0 // indirect
github.com/vishvananda/netlink v1.1.0 // indirect
github.com/vishvananda/netns v0.0.4 // indirect
go.etcd.io/etcd/api/v3 v3.5.10 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.5.10 // indirect
go.etcd.io/etcd/client/v3 v3.5.10 // indirect
go.etcd.io/etcd/api/v3 v3.5.13 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.5.13 // indirect
go.etcd.io/etcd/client/v3 v3.5.13 // indirect
go.mongodb.org/mongo-driver v1.11.1 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.48.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.48.0 // indirect
go.opentelemetry.io/otel v1.23.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.19.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.19.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.20.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.20.0 // indirect
go.opentelemetry.io/otel/metric v1.23.0 // indirect
go.opentelemetry.io/otel/sdk v1.21.0 // indirect
go.opentelemetry.io/otel/trace v1.23.0 // indirect
go.opentelemetry.io/proto/otlp v1.0.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
go4.org v0.0.0-20230225012048-214862532bf5 // indirect
golang.org/x/crypto v0.22.0 // indirect
golang.org/x/exp v0.0.0-20230510235704-dd950f8aeaea // indirect
golang.org/x/crypto v0.23.0 // indirect
golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect
golang.org/x/mod v0.17.0 // indirect
golang.org/x/net v0.24.0 // indirect
golang.org/x/net v0.25.0 // indirect
golang.org/x/oauth2 v0.19.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.19.0 // indirect
golang.org/x/term v0.19.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/sys v0.20.0 // indirect
golang.org/x/term v0.20.0 // indirect
golang.org/x/text v0.15.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/tools v0.20.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
Expand Down Expand Up @@ -192,7 +196,7 @@ require (
k8s.io/kube-openapi v0.0.0-20240411171206-dc4e619f62f3 // indirect
k8s.io/kube-scheduler v0.0.0 // indirect
k8s.io/mount-utils v0.0.0 // indirect
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.28.0 // indirect
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.29.0 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/kube-storage-version-migrator v0.0.6-0.20230721195810-5c8923c5ff96 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
Expand Down Expand Up @@ -232,6 +236,8 @@ replace (

// All of the pinned dependencies below are basically issues that should be fixed either in this project or upstream.
replace (
github.com/ajeddeloh/go-json => github.com/coreos/go-json v0.0.0-20200220154158-5ae607161559
github.com/google/cel-go => github.com/google/cel-go v0.17.8
// TODO: ProcessorCore.Index is removed entirely in jaypipes/ghw > v0.11.0; investigate how to fix this.
github.com/jaypipes/ghw => github.com/jaypipes/ghw v0.8.1-0.20210605191321-eb162add542b
// The following is needed to make "go vet" pass, otherwise it fails on the vendor directory.
Expand Down
Loading

0 comments on commit 4d6cb12

Please sign in to comment.