Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds provisioner limits based on cpu and mem #817

Merged
merged 27 commits into from
Nov 24, 2021
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
632e737
Limit provisioner by cpu and memory
suket22 Nov 19, 2021
db8dbb1
Update pkg/apis/provisioning/v1alpha5/provisioner.go
suket22 Nov 19, 2021
5a7759a
Update pkg/controllers/provisioning/provisioner.go
suket22 Nov 19, 2021
5c5b3cf
Update pkg/controllers/provisioning/resourcecounter.go
suket22 Nov 19, 2021
e00778a
Separate controller for counting resources
suket22 Nov 22, 2021
4a6cd5e
Separate controller for counting resources
suket22 Nov 22, 2021
7633fc9
Etarn fixes
ellistarn Nov 23, 2021
9310ccd
Merge pull request #1 from ellistarn/limitsImpl
suket22 Nov 23, 2021
217942f
Some more fixes - don't default the status
suket22 Nov 23, 2021
2f291aa
Remove extra logging statement
suket22 Nov 23, 2021
9d67667
Fix defaults, fix binding errors
suket22 Nov 23, 2021
3285786
More minor fixes
suket22 Nov 23, 2021
4531d68
Remove extra patch from rbac
suket22 Nov 23, 2021
00a10d2
Addressing comments on the PR
suket22 Nov 23, 2021
2e20056
Compare provisionerSpecs before stopping existing Provisioners
suket22 Nov 23, 2021
7134769
Fix failing build!
suket22 Nov 23, 2021
64c1714
Merge branch 'main' into limitsImpl
suket22 Nov 23, 2021
555d18d
Don't reassign the provisioner in the launcher
suket22 Nov 24, 2021
f866907
Addressing more comments on the PR
suket22 Nov 24, 2021
74d6bd9
Adds basic unit test
suket22 Nov 24, 2021
afa1796
Update pkg/controllers/counter/controller.go
suket22 Nov 24, 2021
9974847
Update pkg/apis/provisioning/v1alpha5/provisioner.go
suket22 Nov 24, 2021
dfbc043
Update pkg/controllers/counter/controller.go
suket22 Nov 24, 2021
3036eea
More refactoring
suket22 Nov 24, 2021
ca36e42
Merge branch 'main' into limitsImpl
suket22 Nov 24, 2021
26c1831
Update pkg/controllers/provisioning/suite_test.go
suket22 Nov 24, 2021
2591f0a
Apply suggestions from code review
suket22 Nov 24, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions charts/karpenter/crds/karpenter.sh_provisioners.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,21 @@ spec:
description: Labels are layered with Requirements and applied to every
node.
type: object
limits:
description: Limits define a set of bounds that Karpenter obeys when
provisioning capacity.
properties:
resources:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: Resources contains all the allocatable resources
that Karpenter supports for limiting.
type: object
type: object
provider:
description: Provider contains fields specific to your cloudprovider.
type: object
Expand Down Expand Up @@ -177,6 +192,15 @@ spec:
the number of nodes
format: date-time
type: string
resources:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: Resources is the list of resources that have been provisioned.
type: object
type: object
type: object
served: true
Expand Down
2 changes: 1 addition & 1 deletion charts/karpenter/templates/controller/rbac.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ rules:
- patch
- get
- list
- patch
suket22 marked this conversation as resolved.
Show resolved Hide resolved
- update
- watch
- apiGroups:
- coordination.k8s.io
Expand Down
2 changes: 2 additions & 0 deletions cmd/controller/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"github.com/awslabs/karpenter/pkg/cloudprovider"
"github.com/awslabs/karpenter/pkg/cloudprovider/registry"
"github.com/awslabs/karpenter/pkg/controllers"
"github.com/awslabs/karpenter/pkg/controllers/counter"
"github.com/awslabs/karpenter/pkg/controllers/metrics"
"github.com/awslabs/karpenter/pkg/controllers/node"
"github.com/awslabs/karpenter/pkg/controllers/provisioning"
Expand Down Expand Up @@ -91,6 +92,7 @@ func main() {
terminator,
node.NewController(manager.GetClient()),
metrics.NewController(manager.GetClient(), cloudProvider),
counter.NewController(ctx, manager.GetClient()),
suket22 marked this conversation as resolved.
Show resolved Hide resolved
).Start(ctx); err != nil {
panic(fmt.Sprintf("Unable to start manager, %s", err.Error()))
}
Expand Down
84 changes: 39 additions & 45 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,91 +10,85 @@ require (
github.com/go-logr/zapr v0.4.0
github.com/imdario/mergo v0.3.12
github.com/mitchellh/hashstructure/v2 v2.0.2
github.com/onsi/ginkgo v1.16.4
github.com/onsi/gomega v1.13.0
github.com/onsi/ginkgo v1.16.5
github.com/onsi/gomega v1.16.0
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/prometheus/client_golang v1.11.0
go.uber.org/multierr v1.7.0
golang.org/x/time v0.0.0-20210611083556-38a9dc6acbc6
golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac
golang.org/x/tools v0.1.8-0.20211014194737-fc98fb2abd48 // indirect
k8s.io/api v0.20.7
k8s.io/apimachinery v0.20.7
k8s.io/client-go v0.20.7
knative.dev/pkg v0.0.0-20210628225612-51cfaabbcdf6
sigs.k8s.io/controller-runtime v0.8.3
gopkg.in/inf.v0 v0.9.1 // indirect
k8s.io/api v0.21.4
k8s.io/apimachinery v0.21.4
k8s.io/client-go v0.21.4
knative.dev/pkg v0.0.0-20211120133512-d016976f2567
sigs.k8s.io/controller-runtime v0.9.7
)

require (
cloud.google.com/go v0.72.0 // indirect
cloud.google.com/go v0.97.0 // indirect
contrib.go.opencensus.io/exporter/ocagent v0.7.1-0.20200907061046-05415f1de66d // indirect
contrib.go.opencensus.io/exporter/prometheus v0.3.0 // indirect
contrib.go.opencensus.io/exporter/stackdriver v0.13.5 // indirect
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 // indirect
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d // indirect
contrib.go.opencensus.io/exporter/prometheus v0.4.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/blang/semver/v4 v4.0.0 // indirect
github.com/blendle/zapdriver v1.3.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.3.0 // indirect
github.com/cespare/xxhash/v2 v2.1.1 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/evanphx/json-patch v4.9.0+incompatible // indirect
github.com/evanphx/json-patch/v5 v5.5.0 // indirect
github.com/evanphx/json-patch v4.11.0+incompatible // indirect
github.com/evanphx/json-patch/v5 v5.6.0 // indirect
github.com/fsnotify/fsnotify v1.4.9 // indirect
github.com/go-kit/log v0.1.0 // indirect
github.com/go-logfmt/logfmt v0.5.0 // indirect
github.com/go-logr/logr v0.4.0 // indirect
github.com/gobuffalo/flect v0.2.2 // indirect
github.com/gobuffalo/flect v0.2.4 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/go-cmp v0.5.6 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/uuid v1.2.0 // indirect
github.com/googleapis/gax-go/v2 v2.0.5 // indirect
github.com/googleapis/gnostic v0.5.1 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.14.8 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/googleapis/gnostic v0.5.5 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/json-iterator/go v1.1.11 // indirect
github.com/kelseyhightower/envconfig v1.4.0 // indirect
github.com/konsorten/go-windows-terminal-sequences v1.0.3 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.1 // indirect
github.com/nxadm/tail v1.4.8 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.26.0 // indirect
github.com/prometheus/common v0.32.1 // indirect
github.com/prometheus/procfs v0.6.0 // indirect
github.com/prometheus/statsd_exporter v0.20.0 // indirect
github.com/sirupsen/logrus v1.6.0 // indirect
github.com/prometheus/statsd_exporter v0.21.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
go.opencensus.io v0.23.0 // indirect
go.uber.org/atomic v1.8.0 // indirect
go.uber.org/atomic v1.9.0 // indirect
go.uber.org/automaxprocs v1.4.0 // indirect
go.uber.org/zap v1.17.0 // indirect
golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a // indirect
golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d // indirect
golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c // indirect
go.uber.org/zap v1.19.1 // indirect
golang.org/x/net v0.0.0-20211101193420-4a448f8816b3 // indirect
golang.org/x/oauth2 v0.0.0-20211028175245-ba495a64dcb5 // indirect
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e // indirect
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 // indirect
golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359 // indirect
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d // indirect
golang.org/x/text v0.3.7 // indirect
gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect
google.golang.org/api v0.36.0 // indirect
google.golang.org/api v0.60.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20210416161957-9910b6c460de // indirect
google.golang.org/grpc v1.38.0 // indirect
google.golang.org/protobuf v1.26.0 // indirect
gopkg.in/alecthomas/kingpin.v2 v2.2.6 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
google.golang.org/genproto v0.0.0-20211021150943-2b146023228c // indirect
google.golang.org/grpc v1.42.0 // indirect
google.golang.org/protobuf v1.27.1 // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
k8s.io/apiextensions-apiserver v0.20.7 // indirect
k8s.io/component-base v0.20.7 // indirect
k8s.io/apiextensions-apiserver v0.21.4 // indirect
k8s.io/component-base v0.21.4 // indirect
k8s.io/klog v1.0.0 // indirect
k8s.io/klog/v2 v2.4.0 // indirect
k8s.io/kube-openapi v0.0.0-20201113171705-d219536bb9fd // indirect
k8s.io/utils v0.0.0-20210111153108-fddb29f9d009 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.0.3 // indirect
sigs.k8s.io/yaml v1.2.0 // indirect
k8s.io/klog/v2 v2.8.0 // indirect
k8s.io/kube-openapi v0.0.0-20210305001622-591a79e4bda7 // indirect
k8s.io/utils v0.0.0-20210802155522-efc7438f0176 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.1.2 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)
Loading