Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Karpenter adds a node from incorrent zone #1011

Closed
Noksa opened this issue Dec 16, 2021 · 3 comments
Closed

Karpenter adds a node from incorrent zone #1011

Noksa opened this issue Dec 16, 2021 · 3 comments
Labels
feature New feature or request

Comments

@Noksa
Copy link

Noksa commented Dec 16, 2021

Tell us about your request

I have three pods in stateful set, each of them has volume in different AZ.

When I scale STS from 0 to 1, Karpenter creates a node for a pod from different zone than volume.
And the pod can't start.

│   Warning  FailedAttachVolume  6s                   attachdetach-controller  AttachVolume.Attach failed for volume "pvc-221eed59-ffab-4986-9d13-1c408474597a │
│ " : error attaching EBS volume "vol-0c01c1ec9ec633405" to instance "i-0370927d1b2294b18": "InvalidVolume.ZoneMismatch: The volume 'vol-0c01c1ec9ec633405' is │
│  not in the same availability zone as instance 'i-0370927d1b2294b18'\n\tstatus code: 400, request id: 456911a7-40e6-48bf-8fdd-0b0fe8033df8"

How can I solve it? I have the following provisioner:

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: storage-db-smb-cluster1
spec:
  ttlSecondsAfterEmpty: 180
  labels:
    only-db: "true"
  taints:
    - effect: NoSchedule
      key: only-db
      value: "true"
  provider:
    instanceProfile: ad-eks-c-NodeInstanceProfile
    subnetSelector:
      kubernetes.io/cluster/ad-eks-c: '*'
    launchTemplate: ad-eks-default-lt

Does Karpenter consider persistent storage/volume?
For example I can have all volumes in only one zone but I don't want to specify only one zone in Provisoner because of future scaling.
In this case I expected that Karpenter would create all nodes from zones where I have related pvc/pv first.

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment
@Noksa Noksa added the feature New feature or request label Dec 16, 2021
@Noksa
Copy link
Author

Noksa commented Dec 16, 2021

Hm, looks like it is the same as #622

@ellistarn
Copy link
Contributor

Do you mind commenting on #622 and we can close here? This is absolutely on our roadmap.

@Noksa
Copy link
Author

Noksa commented Dec 16, 2021

@ellistarn
Sure. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants