Can not run apply example

[bayerlse]

Describe the bug
Hey there 👋 When running your apply example, Im always getting an error with:
Failed to fetch resource metadata for apps/v1/Deployment: HTTP request failed
I think it's not really related to the actual example, but more due to setup before. But I couldnt
find anything related to my issue here.

** Client Version **
0.14.0

** Server Version **
1.17.15-gke.800

To Reproduce
I loaded my cluster Info via getCluster via @google-cloud/container client lib
And filled the necessary props for loadFromClusterAndUser

After that I run your example of typescript apply but Always getting the (not very meaningful error) Failed to fetch resource metadata for apps/v1/Deployment: HTTP request failed

Expected behavior
Would love to have this running without problems :)

** Example Code**

// Load Cluster Info:
const containerClient = new ClusterManagerClient();
const [clusterInfo] = await container.getCluster({name: "my-cluster-name"});

const masterAuth = clusterInfo.masterAuth

const cluster: Cluster = {
  name: clusterName,
  caData:masterAuth?.clusterCaCertificate!,
  server: `https://${clusterInfo.endpoint}`
  skipTLSVerify: true
};

const user: User = {
  certData: masterAuth?.clientCertificate!,
  username: masterAuth?.username!,
  password: masterAuth?.password!,
  name: masterAuth?.username!
};

const kc = new k8s.KubeConfig();
kc.loadFromClusterAndUser(cluster, user);
const k8sAPIClient = k8s.KubernetesObjectApi.makeApiClient(kc);

try {
  for (const spec of this.k8sSpecs) {
    spec.metadata = spec.metadata || {};
    spec.metadata.annotations = spec.metadata.annotations || {};
    delete spec.metadata.annotations['kubectl.kubernetes.io/last-applied-configuration'];
    spec.metadata.annotations['kubectl.kubernetes.io/last-applied-configuration'] = JSON.stringify(spec);
    try {
      await k8sAPIClient.read(spec);
      await k8sAPIClient.patch(spec);
    } catch (e) {
      core.error(`Could not patch spec: ${e.message}`);
      await k8sAPIClient.create(spec);
    }
  }
} catch (e) {
  throw new Error(`Could not deploy service: ${e.message}`);
}

Environment (please complete the following information):

• OS: Github Action
• NodeJS Version 12
• Cloud runtime: Google Cloud (GKE)

Additional context
I also tried to set up user for loadFromClusterAndUser via documentation for GKE API Authentication but then (despite the GOOGLE_APPLICATION_CREDENTIALS env is set correctly, as the getCluster runs smoothly with same keyfile) I get Failed to fetch resource metadata for apps/v1/Deployment: Cannot read property 'access-token' of undefined

For this my user object (which will be used in loadFromClusterAndUser) would look like:

const user: User = {
  name: 'ci-cd-pipeline-gsa',
  authProvider: {
    name: 'gcp'
  }
};

Can you provide any help here please?

Thanks in advance!

[brendandburns]

The current GCP authentication doesn't support that environment variable (GOOGLE_APPLICATION_CREDENTIALS). We'd take a PR to add support if you wanted to send it.

For the first one, try removing the username and password. You are using certificate authentication, so they shouldn't be necessary.

Also, please validate that Kubernetes RBAC is working correctly (easiest way to do this is to use those parameters with kubectl)

[bayerlse]

Hey @brendandburns

Thanks for your reply and suggestions.

I managed to do it via first getting the access-token from google container client lib and append it to all requests.

The new script (for all others who may run into this issue) is:

// Load Cluster Info:
const containerClient = new ClusterManagerClient();
const accessToken = await containerClient.auth.getAccessToken();
const [clusterInfo] = await container.getCluster({name: `projects/${<PROJECT_ID>}/locations/${<LOCATION>}/clusters/${<CLUSTER_NAME>}`});

const masterAuth = clusterInfo.masterAuth;
const clusterName = clusterInfo.name!;


const cluster: Cluster = {
  name: clusterName,
  caData:masterAuth?.clusterCaCertificate!,
  server: `https://${clusterInfo.endpoint}`
  skipTLSVerify: true
};

const user: User = {
  name: '<YOUR_SA_USER_NAME>-gsa',
  authProvider: {
    name: 'gcp'
  }
};

const kc = new k8s.KubeConfig();
kc.loadFromClusterAndUser(cluster, user);
const k8sAPIClient = k8s.KubernetesObjectApi.makeApiClient(kc);
k8sAPIClient.setDefaultAuthentication({
  applyToRequest: (opts) => {
     opts.ca = Buffer.from(masterAuth.clusterCaCertificate!, 'base64');
     opts.headers!.Authorization = 'Bearer ' + accessToken;
  }
});

try {
  for (const spec of this.k8sSpecs) {
    spec.metadata = spec.metadata || {};
    spec.metadata.annotations = spec.metadata.annotations || {};
    delete spec.metadata.annotations['kubectl.kubernetes.io/last-applied-configuration'];
    spec.metadata.annotations['kubectl.kubernetes.io/last-applied-configuration'] = JSON.stringify(spec);
    try {
      await k8sAPIClient.read(spec);
      await k8sAPIClient.patch(spec);
    } catch (e) {
      core.error(`Could not patch spec: ${e.message}`);
      await k8sAPIClient.create(spec);
    }
  }
} catch (e) {
  throw new Error(`Could not deploy service: ${e.message}`);
}

[mansona]

I'm trying to figure out your workaround and I can't find any documentation on the first 3 lines of your solution:

const containerClient = new ClusterManagerClient();
const accessToken = await containerClient.auth.getAccessToken();
const [clusterInfo] = await container.getCluster({name: `projects/${<PROJECT_ID>}/locations/${<LOCATION>}/clusters/${<CLUSTER_NAME>}`});

where do these come from?

[dimip1606]

They come from the @google-cloud/container package.

The import is missing in the script:

import { ClusterManagerClient } from '@google-cloud/container'

For me, this is quite useful, as I only have a keyFile.json for our service account.

You can use it like this:

const client = new ClusterManagerClient({
  keyFilename: 'keyFile.json',
})

Does anyone know if there is a more elegant way to provide the information in this json file to KubeConfig, in order to let it handle the authentication and retrieve the access token automatically?