Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provide option to set MaxConcurrentReconciles #203

Merged
merged 2 commits into from
Aug 19, 2022
+39 −11
Merged

provide option to set MaxConcurrentReconciles #203

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0c2496b
provide option to set MaxConcurrentReconciles
Madhu-1 Aug 16, 2022
9be5a13
doc: document command line arguments
Madhu-1 Aug 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions cmd/manager/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import (
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/healthz"
"sigs.k8s.io/controller-runtime/pkg/log/zap"

Expand Down Expand Up @@ -61,12 +62,14 @@ func main() {
var enableLeaderElection bool
var probeAddr string
var reclaimSpaceTimeout time.Duration
var maxConcurrentReconciles int
flag.StringVar(&metricsAddr, "metrics-bind-address", ":8080", "The address the metric endpoint binds to.")
flag.StringVar(&probeAddr, "health-probe-bind-address", ":8081", "The address the probe endpoint binds to.")
flag.BoolVar(&enableLeaderElection, "leader-elect", false,
"Enable leader election for controller manager. "+
"Enabling this will ensure there is only one active controller manager.")
flag.DurationVar(&reclaimSpaceTimeout, "reclaim-space-timeout", defaultTimeout, "Timeout for reclaimspace operation")
flag.IntVar(&maxConcurrentReconciles, "max-concurrent-reconciles", 100, "Maximum number of concurrent reconciles")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
flag.IntVar(&maxConcurrentReconciles, "max-concurrent-reconciles", 100, "Maximum number of concurrent reconciles")
flag.IntVar(&maxConcurrentReconciles, "max-concurrent-reconciles", 100, "Maximum number of concurrent reconciles")

Do we really need 100 as default value ?
most controllers have just a single loop.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, having 100 in default doesn't cause any harm, and also, it doesn't make sense to have different configurations per controller. This helps to have concurrent reconciles, which fasten the process when we have more requests.

opts := zap.Options{
Development: true,
}
Expand All @@ -90,11 +93,14 @@ func main() {

connPool := connection.NewConnectionPool()

ctrlOptions := controller.Options{
MaxConcurrentReconciles: maxConcurrentReconciles,
}
if err = (&controllers.CSIAddonsNodeReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
ConnPool: connPool,
}).SetupWithManager(mgr); err != nil {
}).SetupWithManager(mgr, ctrlOptions); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "CSIAddonsNode")
os.Exit(1)
}
Expand All @@ -104,7 +110,7 @@ func main() {
Scheme: mgr.GetScheme(),
ConnPool: connPool,
Timeout: reclaimSpaceTimeout,
}).SetupWithManager(mgr); err != nil {
}).SetupWithManager(mgr, ctrlOptions); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "ReclaimSpaceJob")
os.Exit(1)
}
Expand All @@ -113,21 +119,21 @@ func main() {
Scheme: mgr.GetScheme(),
Connpool: connPool,
Timeout: time.Minute * 3,
}).SetupWithManager(mgr); err != nil {
}).SetupWithManager(mgr, ctrlOptions); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "NetworkFence")
os.Exit(1)
}
if err = (&controllers.ReclaimSpaceCronJobReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
}).SetupWithManager(mgr); err != nil {
}).SetupWithManager(mgr, ctrlOptions); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "ReclaimSpaceCronJob")
os.Exit(1)
}
if err = (&controllers.PersistentVolumeClaimReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
}).SetupWithManager(mgr); err != nil {
}).SetupWithManager(mgr, ctrlOptions); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "PersistentVolumeClaim")
os.Exit(1)
}
Expand Down
4 changes: 3 additions & 1 deletion controllers/csiaddons/csiaddonsnode_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import (
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/predicate"
)
Expand Down Expand Up @@ -149,10 +150,11 @@ func (r *CSIAddonsNodeReconciler) Reconcile(ctx context.Context, req ctrl.Reques
}

// SetupWithManager sets up the controller with the Manager.
func (r *CSIAddonsNodeReconciler) SetupWithManager(mgr ctrl.Manager) error {
func (r *CSIAddonsNodeReconciler) SetupWithManager(mgr ctrl.Manager, ctrlOptions controller.Options) error {
return ctrl.NewControllerManagedBy(mgr).
For(&csiaddonsv1alpha1.CSIAddonsNode{}).
WithEventFilter(predicate.GenerationChangedPredicate{}).
WithOptions(ctrlOptions).
Complete(r)
}

Expand Down
4 changes: 3 additions & 1 deletion controllers/csiaddons/networkfence_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import (
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/predicate"

Expand Down Expand Up @@ -186,10 +187,11 @@ func (nf *NetworkFenceInstance) updateStatus(ctx context.Context,
}

// SetupWithManager sets up the controller with the Manager.
func (r *NetworkFenceReconciler) SetupWithManager(mgr ctrl.Manager) error {
func (r *NetworkFenceReconciler) SetupWithManager(mgr ctrl.Manager, ctrlOptions controller.Options) error {
return ctrl.NewControllerManagedBy(mgr).
For(&csiaddonsv1alpha1.NetworkFence{}).
WithEventFilter(predicate.GenerationChangedPredicate{}).
WithOptions(ctrlOptions).
Complete(r)
}

Expand Down
4 changes: 3 additions & 1 deletion controllers/csiaddons/persistentvolumeclaim_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import (
"k8s.io/apimachinery/pkg/types"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/event"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/predicate"
Expand Down Expand Up @@ -168,7 +169,7 @@ func (r *PersistentVolumeClaimReconciler) Reconcile(ctx context.Context, req ctr
}

// SetupWithManager sets up the controller with the Manager.
func (r *PersistentVolumeClaimReconciler) SetupWithManager(mgr ctrl.Manager) error {
func (r *PersistentVolumeClaimReconciler) SetupWithManager(mgr ctrl.Manager, ctrlOptions controller.Options) error {
err := mgr.GetFieldIndexer().IndexField(
context.Background(),
&csiaddonsv1alpha1.ReclaimSpaceCronJob{},
Expand Down Expand Up @@ -204,6 +205,7 @@ func (r *PersistentVolumeClaimReconciler) SetupWithManager(mgr ctrl.Manager) err
For(&corev1.PersistentVolumeClaim{}).
Owns(&csiaddonsv1alpha1.ReclaimSpaceCronJob{}).
WithEventFilter(pred).
WithOptions(ctrlOptions).
Complete(r)
}

Expand Down
4 changes: 3 additions & 1 deletion controllers/csiaddons/reclaimspacecronjob_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import (
ref "k8s.io/client-go/tools/reference"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/log"
)

Expand Down Expand Up @@ -190,7 +191,7 @@ func (r *ReclaimSpaceCronJobReconciler) Reconcile(ctx context.Context, req ctrl.
}

// SetupWithManager sets up the controller with the Manager.
func (r *ReclaimSpaceCronJobReconciler) SetupWithManager(mgr ctrl.Manager) error {
func (r *ReclaimSpaceCronJobReconciler) SetupWithManager(mgr ctrl.Manager, ctrlOptions controller.Options) error {
if err := mgr.GetFieldIndexer().IndexField(context.Background(), &csiaddonsv1alpha1.ReclaimSpaceJob{}, jobOwnerKey, func(rawObj client.Object) []string {
// extract the owner from job object.
job, ok := rawObj.(*csiaddonsv1alpha1.ReclaimSpaceJob)
Expand All @@ -213,6 +214,7 @@ func (r *ReclaimSpaceCronJobReconciler) SetupWithManager(mgr ctrl.Manager) error
return ctrl.NewControllerManagedBy(mgr).
For(&csiaddonsv1alpha1.ReclaimSpaceCronJob{}).
Owns(&csiaddonsv1alpha1.ReclaimSpaceJob{}).
WithOptions(ctrlOptions).
Complete(r)
}

Expand Down
4 changes: 3 additions & 1 deletion controllers/csiaddons/reclaimspacejob_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ import (
"k8s.io/apimachinery/pkg/types"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/predicate"
)
Expand Down Expand Up @@ -160,10 +161,11 @@ func (r *ReclaimSpaceJobReconciler) Reconcile(ctx context.Context, req ctrl.Requ
}

// SetupWithManager sets up the controller with the Manager.
func (r *ReclaimSpaceJobReconciler) SetupWithManager(mgr ctrl.Manager) error {
func (r *ReclaimSpaceJobReconciler) SetupWithManager(mgr ctrl.Manager, ctrlOptions controller.Options) error {
return ctrl.NewControllerManagedBy(mgr).
For(&csiaddonsv1alpha1.ReclaimSpaceJob{}).
WithEventFilter(predicate.GenerationChangedPredicate{}).
WithOptions(ctrlOptions).
Complete(r)
}

Expand Down
14 changes: 13 additions & 1 deletion docs/deploy-controller.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@

The CSI-Addons Controller can be deployed by different ways:

## Configuration

**Available command line arguments:**

| Option | Default value | Description |
| ----------------------------- | --------------- | --------------------------------------------- |
| `--metrics-bind-address` | `:8080` | The address the metric endpoint binds to. |
| `--health-probe-bind-address` | `:8081` | The address the probe endpoint binds to. |
| `--leader-elect` | `false` | Enable leader election for controller manager.|
| `--reclaim-space-timeout` | `3m` | Timeout for reclaimspace operation |
| `--max-concurrent-reconciles` | 100 | Maximum number of concurrent reconciles |

## Installation for versioned deployments

The CSI-Addons Controller can also be installed using the yaml files in `deploy/controller`.
Expand All @@ -21,7 +33,7 @@ customresourcedefinition.apiextensions.k8s.io/reclaimspacecronjobs.csiaddons.ope
customresourcedefinition.apiextensions.k8s.io/reclaimspacejobs.csiaddons.openshift.io created

$ kubectl create -f rbac.yaml
...
...
serviceaccount/csi-addons-controller-manager created
role.rbac.authorization.k8s.io/csi-addons-leader-election-role created
clusterrole.rbac.authorization.k8s.io/csi-addons-manager-role created
Expand Down