Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not working with karpenter-provider-aws 1.0.6 #189

Open
caleblloyd opened this issue Oct 14, 2024 · 10 comments
Open

Not working with karpenter-provider-aws 1.0.6 #189

caleblloyd opened this issue Oct 14, 2024 · 10 comments

Comments

@caleblloyd
Copy link

I am installing using the same code as integ.karpenter.ts with the following changes:

// cluster
      version: KubernetesVersion.V1_31,
      kubectlLayer: new KubectlV31Layer(this, 'KubectlLayer'),

// karpenter chart
    version: '1.0.6'

The node pool is failing to install with the following error:

5:04:15 PM | CREATE_FAILED        | Custom::AWSCDK-EKS-KubernetesResource | Clustermanifestnodepool9668C774
Received response status [FAILED] from custom resource. Message returned: Error: b'Error from server: error when creating "/tmp/manifest.yaml": conversion webhook for karpenter.sh/v1beta1,
Kind=NodePool failed: Post "https://karpenter.kube-system.svc:8443/conversion/karpenter.sh?timeout=30s": service "karpenter" not found\n'

Might have to do with the resource API groups changing from karpenter.k8s.aws/v1beta1 to karpenter.k8s.aws/v1?

@andskli
Copy link
Contributor

andskli commented Nov 5, 2024

@caleblloyd can you let us know if #190 solved this issue?

@caleblloyd
Copy link
Author

I don't think so, the issue is that API Versions have changed from karpenter.k8s.aws/v1beta1 and karpenter.sh/v1beta1 to v1 instead of v1beta1

@markusl
Copy link

markusl commented Nov 17, 2024

@andskli could you please publish a new version with the changes from #190?

@ClayLu
Copy link

ClayLu commented Nov 29, 2024

Hey guys,

I successfully installed Karpenter with these changes. I believe we are talking about the same issue.

// cluster
      version: KubernetesVersion.V1_30,
      kubectlLayer: new KubectlV31Layer(this, 'KubectlLayer'),

// karpenter chart
    version: '1.0.7'

In addition, I downloaded the v1.0.7 source code locally and modified the namespace setting. This was necessary due to a bug where the Karpenter service account is created in the kube-system namespace by default.

this.namespace = props.namespace ?? 'kube-system';

    /**
     * For the Karpenter controller to be able to talk to the AWS APIs, we need to set up a few
     * resources which will allow the Karpenter controller to use IAM Roles for Service Accounts
     */
    /*
    const namespace = this.cluster.addManifest('karpenter-namespace', {
      apiVersion: 'v1',
      kind: 'Namespace',
      metadata: {
        name: this.namespace,
      },
    });
    */

// this.serviceAccount.node.addDependency(namespace);

// this.chart.node.addDependency(namespace);

@knuurr
Copy link

knuurr commented Nov 29, 2024

Hey guys,

I successfully installed Karpenter with these changes. I believe we are talking about the same issue.

// cluster
      version: KubernetesVersion.V1_30,
      kubectlLayer: new KubectlV31Layer(this, 'KubectlLayer'),

// karpenter chart
    version: '1.0.7'

In addition, I downloaded the v1.0.7 source code locally and modified the namespace setting. This was necessary due to a bug where the Karpenter service account is created in the kube-system namespace by default.

this.namespace = props.namespace ?? 'kube-system';

    /**
     * For the Karpenter controller to be able to talk to the AWS APIs, we need to set up a few
     * resources which will allow the Karpenter controller to use IAM Roles for Service Accounts
     */
    /*
    const namespace = this.cluster.addManifest('karpenter-namespace', {
      apiVersion: 'v1',
      kind: 'Namespace',
      metadata: {
        name: this.namespace,
      },
    });
    */

// this.serviceAccount.node.addDependency(namespace);

// this.chart.node.addDependency(namespace);

Hi @ClayLu that's great to hear as I am working on the same issue as I wish to upgrade it too.

Just to clarify, You downloaded and modified the source code of this library, or Karpenter itself?

Also, what about IAM settings, as there are also subtle differences between one hardcoded in library vs CF file for 1.0.0?

And do we have a way to override namespace for ServiceAccount using this library? If so could you point to exact changes in library code?

@ClayLu
Copy link

ClayLu commented Nov 30, 2024

@knuurr I only download the source code and modify it. I suggest you try it first. Thx.

@knuurr
Copy link

knuurr commented Dec 2, 2024

@knuurr I only download the source code and modify it. I suggest you try it first. Thx.

How did you manage to build it locally?

I get lots of errors for https.

node_modules/@types/node/https.d.ts:306:25 - error TS1109: Expression expected.

306         requestListener?: http.RequestListener<Request, Response>,
                            ~
node_modules/@types/node/https.d.ts:306:66 - error TS1109: Expression expected.

306         requestListener?: http.RequestListener<Request, Response>,
                                                                     ~
node_modules/@types/node/https.d.ts:307:5 - error TS1109: Expression expected.

307     ): Server<Request, Response>;
        ~
node_modules/@types/node/https.d.ts:307:6 - error TS1128: Declaration or statement expected.

307     ): Server<Request, Response>;
         ~
node_modules/@types/node/https.d.ts:307:33 - error TS1005: '(' expected.

307     ): Server<Request, Response>;
👾 Task "build » compile" failed when executing "jsii --silence-warnings=reserved-word" (cwd: /Users/qbus/Documents/GitHub/cdk-eks-karpenter)
node --version
v18.20.5

@ClayLu
Copy link

ClayLu commented Dec 2, 2024

  1. Download the https://github.com/aws-samples/cdk-eks-karpenter/tree/main/src/index.ts to local.

  2. Move the code to your project and import it.
    import { Karpenter } from "./{path}/index.ts";

  3. Modify your code.

  4. When you are ready, you can build it.

@knuurr
Copy link

knuurr commented Dec 4, 2024

Ok thanks @ClayLu . Looks like it worked in my case too.

@andskli
Copy link
Contributor

andskli commented Dec 10, 2024

@andskli could you please publish a new version with the changes from #190?

Built and released to PyPI and NPM. 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants