Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add providerId, maxTokens and topk to ai spec #545

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

samirtahir91
Copy link

Closes # #535

📑 Description

We need to set maxTokens, providerId and topk for backends like google gemini and googlevertexai, else they fail with related errors.

This PR adds new environment variables for K8SGPT_MAX_TOKENS (default 2048) and K8SGPT_TOP_K (default 50) into the k8sgpt deployment spec.

It optionally adds the env var K8SGPT_PROVIDER_ID to the deployment spec

The new K8SGPT CRD specs introduced for this are:

ai:
  providerId
  maxTokens
  topk

I have tested with vertex ai in GKE with a custom image of k8sgpt based off this PR k8sgpt-ai/k8sgpt#1280 with the example:

apiVersion: core.k8sgpt.ai/v1alpha1
kind: K8sGPT
metadata:
  name: k8sgpt-sample-vertexai
  namespace: k8sgpt-operator-system
spec:
  ai:
    model: gemini-1.5-pro-002
    backend: googlevertexai
    providerId: <my gcp project ID>
    enabled: true
    anonymized: true
    maxTokens: "2048"
    topk: "40"
  version: latest
  repository: samirtahir91076/k8sgpt
  noCache: false

✅ Checks

  • My pull request adheres to the code style of this project
  • My code requires changes to the documentation
  • I have updated the documentation as required
  • All the tests have passed

ℹ Additional Information

This PR depends on k8sgpt-ai/k8sgpt#1280 for K8SGPT_MAX_TOKENS env var to configure k8sgpt server config.

@samirtahir91 samirtahir91 force-pushed the fix/add-provider-and-max-tokens branch from 9610901 to 8811f2b Compare October 23, 2024 20:01
@samirtahir91
Copy link
Author

@AlexsJones Could you review this please?

@samirtahir91
Copy link
Author

@AlexsJones - Can this be reviewed pls?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant