Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add allocation binpacking duration metric #687

Merged
merged 2 commits into from
Sep 20, 2021
Merged

add allocation binpacking duration metric #687

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
eb97962
add allocation binpacking duration metric
cjerad Sep 17, 2021
e7213e5
simplify metric logic
cjerad Sep 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions pkg/controllers/allocation/binpacking/packer.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,23 +18,41 @@ import (
"context"
"math"
"sort"
"time"

"github.com/awslabs/karpenter/pkg/apis/provisioning/v1alpha3"
"github.com/awslabs/karpenter/pkg/cloudprovider"
"github.com/awslabs/karpenter/pkg/controllers/allocation/scheduling"
"github.com/awslabs/karpenter/pkg/metrics"
"github.com/awslabs/karpenter/pkg/utils/apiobject"
"github.com/awslabs/karpenter/pkg/utils/resources"
"github.com/prometheus/client_golang/prometheus"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
"knative.dev/pkg/logging"
"sigs.k8s.io/controller-runtime/pkg/client"
crmetrics "sigs.k8s.io/controller-runtime/pkg/metrics"
)

var (
// MaxInstanceTypes defines the number of instance type options to return to the cloud provider
MaxInstanceTypes = 20

packTimeHistogram = prometheus.NewHistogram(
prometheus.HistogramOpts{
Namespace: metrics.KarpenterNamespace,
Subsystem: "allocation_controller",
Name: "binpacking_duration_seconds",
Help: "Duration of binpacking process in seconds.",
Buckets: metrics.DurationBuckets(),
},
)
)

func init() {
crmetrics.Registry.MustRegister(packTimeHistogram)
}

type packer struct{}

// Packer helps pack the pods and calculates efficient placement on the instances.
Expand Down Expand Up @@ -63,6 +81,11 @@ type Packing struct {
// It follows the First Fit Decreasing bin packing technique, reference-
// https://en.wikipedia.org/wiki/Bin_packing_problem#First_Fit_Decreasing_(FFD)
func (p *packer) Pack(ctx context.Context, schedule *scheduling.Schedule, instances []cloudprovider.InstanceType) []*Packing {
startTime := time.Now()
defer func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reason to wrap this in a func rather than just doing:

startTime := time.Now()
defer packTimeHistogram.Observe(time.Since(startTime).Seconds())

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to Defer, Panic, and Recover from the Go blog:

A deferred function’s arguments are evaluated when the defer statement is evaluated.

So the proposed code will not record the elapsed time because only the .Observe() call is defered; time.Since(startTime).Seconds() will be evaluated immediately.

packTimeHistogram.Observe(time.Since(startTime).Seconds())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool pattern!

}()

// Sort pods in decreasing order by the amount of CPU requested, if
// CPU requested is equal compare memory requested.
sort.Sort(sort.Reverse(ByResourcesRequested{SortablePods: schedule.Pods}))
Expand Down