Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cluster Autoscaler AWS cloudprovider tests are failing #608

Closed
bskiba opened this issue Feb 2, 2018 · 3 comments
Closed

Cluster Autoscaler AWS cloudprovider tests are failing #608

bskiba opened this issue Feb 2, 2018 · 3 comments
Labels
area/cluster-autoscaler area/provider/aws Issues or PRs related to aws provider

Comments

@bskiba
Copy link
Member

bskiba commented Feb 2, 2018

I believe this is a flake (example: https://travis-ci.org/kubernetes/autoscaler/jobs/336570454)
k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws tests are sometimes failing. Logs from the failed test below

E0202 14:19:07.636800   11264 runtime.go:66] Observed a panic: "\n\nmock: Unexpected Method Call\n-----------------------------\n\nDescribeAutoScalingGroupsPages(*autoscaling.DescribeAutoScalingGroupsInput,func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool)\n\t\t0: {\n  AutoScalingGroupNames: [\"coolasg\"],\n  MaxRecords: 100\n}\n\t\t1: (func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool)(0x12e7a50)\n\nThe closest call I have is: \n\nDescribeAutoScalingGroupsPages(*autoscaling.DescribeAutoScalingGroupsInput,mock.AnythingOfTypeArgument)\n\t\t0: {\n  AutoScalingGroupNames: [\"coolasg\"],\n  MaxRecords: 100\n}\n\t\t1: \"func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool\"\n\n\n" (
mock: Unexpected Method Call
-----------------------------
DescribeAutoScalingGroupsPages(*autoscaling.DescribeAutoScalingGroupsInput,func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool)
		0: {
  AutoScalingGroupNames: ["coolasg"],
  MaxRecords: 100
}
		1: (func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool)(0x12e7a50)
The closest call I have is: 
DescribeAutoScalingGroupsPages(*autoscaling.DescribeAutoScalingGroupsInput,mock.AnythingOfTypeArgument)
		0: {
  AutoScalingGroupNames: ["coolasg"],
  MaxRecords: 100
}
		1: "func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool"
)
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:72
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51
/home/travis/.gimme/versions/go1.8.3.linux.amd64/src/runtime/asm_amd64.s:514
/home/travis/.gimme/versions/go1.8.3.linux.amd64/src/runtime/panic.go:489
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/github.com/stretchr/testify/mock/mock.go:311
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/github.com/stretchr/testify/mock/mock.go:288
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/aws_cloud_provider_test.go:40
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/auto_scaling.go:89
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/auto_scaling_groups.go:154
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/auto_scaling_groups.go:53
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134
/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88
/home/travis/.gimme/versions/go1.8.3.linux.amd64/src/runtime/asm_amd64.s:2197
panic: 
mock: Unexpected Method Call
-----------------------------
DescribeAutoScalingGroupsPages(*autoscaling.DescribeAutoScalingGroupsInput,func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool)
		0: {
  AutoScalingGroupNames: ["coolasg"],
  MaxRecords: 100
}
		1: (func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool)(0x12e7a50)
The closest call I have is: 
DescribeAutoScalingGroupsPages(*autoscaling.DescribeAutoScalingGroupsInput,mock.AnythingOfTypeArgument)
		0: {
  AutoScalingGroupNames: ["coolasg"],
  MaxRecords: 100
}
		1: "func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool"
 [recovered]
	panic: 
mock: Unexpected Method Call
-----------------------------
DescribeAutoScalingGroupsPages(*autoscaling.DescribeAutoScalingGroupsInput,func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool)
		0: {
  AutoScalingGroupNames: ["coolasg"],
  MaxRecords: 100
}
		1: (func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool)(0x12e7a50)
The closest call I have is: 
DescribeAutoScalingGroupsPages(*autoscaling.DescribeAutoScalingGroupsInput,mock.AnythingOfTypeArgument)
		0: {
  AutoScalingGroupNames: ["coolasg"],
  MaxRecords: 100
}
		1: "func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool"
goroutine 48 [running]:
k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x126
panic(0x1405a20, 0xc4203050c0)
	/home/travis/.gimme/versions/go1.8.3.linux.amd64/src/runtime/panic.go:489 +0x2cf
k8s.io/autoscaler/cluster-autoscaler/vendor/github.com/stretchr/testify/mock.(*Mock).MethodCalled(0xc4202aec40, 0x21adbda, 0x1e, 0xc420354320, 0x2, 0x2, 0x4, 0x10, 0xc420042b30)
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/github.com/stretchr/testify/mock/mock.go:311 +0x654
k8s.io/autoscaler/cluster-autoscaler/vendor/github.com/stretchr/testify/mock.(*Mock).Called(0xc4202aec40, 0xc420354320, 0x2, 0x2, 0x10, 0x153cd60, 0x1)
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/github.com/stretchr/testify/mock/mock.go:288 +0x14c
k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws.(*AutoScalingMock).DescribeAutoScalingGroupsPages(0xc4202aec40, 0xc420344180, 0xc420304440, 0x227c6b8, 0x0)
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/aws_cloud_provider_test.go:40 +0xa9
k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws.(*autoScalingWrapper).getAutoscalingGroupsByNames(0xc420311c00, 0xc420304410, 0x1, 0x1, 0x1, 0x1, 0x0, 0x227c6b8, 0x50)
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/auto_scaling.go:89 +0x1c2
k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws.(*asgCache).regenerate(0xc420311bd0, 0x1765240, 0xc420311bf8)
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/auto_scaling_groups.go:154 +0x33a
k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws.newASGCache.func1()
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/auto_scaling_groups.go:53 +0x7d
k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc4201e1b20)
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x5e
k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc4201e1b20, 0x34630b8a000, 0x0, 0xc42053ed01, 0xc4201a5980)
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134 +0xbd
k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/wait.Until(0xc4201e1b20, 0x34630b8a000, 0xc4201a5980)
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 +0x4d
created by k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws.newASGCache
	/home/travis/gopath/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/auto_scaling_groups.go:56 +0x21d
FAIL	k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws	0.201s
@bskiba
Copy link
Member Author

bskiba commented Feb 6, 2018

I think this happens in tests added by #462. @negz would you mind taking a look?

@negz
Copy link
Contributor

negz commented Feb 6, 2018

@bskiba Sure! This week is a little busy, but I will try find some time.

@bskiba
Copy link
Member Author

bskiba commented Mar 28, 2018

This is actually a duplicate of #516

@bskiba bskiba closed this as completed Mar 28, 2018
yaroslava-serdiuk pushed a commit to yaroslava-serdiuk/autoscaler that referenced this issue Feb 22, 2024
…modules/k8s.io/component-base-0.26.2

Bump k8s.io/component-base from 0.26.1 to 0.26.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cluster-autoscaler area/provider/aws Issues or PRs related to aws provider
Projects
None yet
Development

No branches or pull requests

2 participants