Skip to content

Commit

Permalink
[Feature] [ML] Container Envs (#1550)
Browse files Browse the repository at this point in the history
  • Loading branch information
ajanikow authored Dec 22, 2023
1 parent de4def4 commit a496add
Show file tree
Hide file tree
Showing 10 changed files with 993 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
- (Feature) (ML) Port adjustments
- (Feature) (ML) Support for deployments with JWT auth enabled
- (Feature) (ML) GPU Jobs
- (Feature) (ML) Container Envs

## [1.2.35](https://github.com/arangodb/kube-arangodb/tree/1.2.35) (2023-11-06)
- (Maintenance) Update go-driver to v1.6.0, update IsNotFound() checks
Expand Down
176 changes: 176 additions & 0 deletions docs/api/ArangoMLExtension.V1Alpha1.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,28 @@ Links:

***

### .spec.deployment.prediction.env

Type: `core.EnvVar` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L33)</sup>

Env keeps the information about environment variables provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core)

***

### .spec.deployment.prediction.envFrom

Type: `core.EnvFromSource` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L38)</sup>

EnvFrom keeps the information about environment variable sources provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core)

***

### .spec.deployment.prediction.gpu

Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/extension_spec_deployment_component.go#L31)</sup>
Expand Down Expand Up @@ -132,6 +154,28 @@ Links:

***

### .spec.deployment.project.env

Type: `core.EnvVar` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L33)</sup>

Env keeps the information about environment variables provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core)

***

### .spec.deployment.project.envFrom

Type: `core.EnvFromSource` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L38)</sup>

EnvFrom keeps the information about environment variable sources provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core)

***

### .spec.deployment.project.gpu

Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/extension_spec_deployment_component.go#L31)</sup>
Expand Down Expand Up @@ -260,6 +304,28 @@ Links:

***

### .spec.deployment.training.env

Type: `core.EnvVar` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L33)</sup>

Env keeps the information about environment variables provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core)

***

### .spec.deployment.training.envFrom

Type: `core.EnvFromSource` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L38)</sup>

EnvFrom keeps the information about environment variable sources provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core)

***

### .spec.deployment.training.gpu

Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/extension_spec_deployment_component.go#L31)</sup>
Expand Down Expand Up @@ -345,6 +411,28 @@ Links:

***

### .spec.init.env

Type: `core.EnvVar` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L33)</sup>

Env keeps the information about environment variables provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core)

***

### .spec.init.envFrom

Type: `core.EnvFromSource` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L38)</sup>

EnvFrom keeps the information about environment variable sources provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core)

***

### .spec.init.hostIPC

Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/container_namespace.go#L33)</sup>
Expand Down Expand Up @@ -492,6 +580,28 @@ Links:

***

### .spec.jobsTemplates.prediction.cpu.env

Type: `core.EnvVar` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L33)</sup>

Env keeps the information about environment variables provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core)

***

### .spec.jobsTemplates.prediction.cpu.envFrom

Type: `core.EnvFromSource` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L38)</sup>

EnvFrom keeps the information about environment variable sources provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core)

***

### .spec.jobsTemplates.prediction.cpu.hostIPC

Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/container_namespace.go#L33)</sup>
Expand Down Expand Up @@ -639,6 +749,28 @@ Links:

***

### .spec.jobsTemplates.prediction.gpu.env

Type: `core.EnvVar` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L33)</sup>

Env keeps the information about environment variables provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core)

***

### .spec.jobsTemplates.prediction.gpu.envFrom

Type: `core.EnvFromSource` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L38)</sup>

EnvFrom keeps the information about environment variable sources provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core)

***

### .spec.jobsTemplates.prediction.gpu.hostIPC

Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/container_namespace.go#L33)</sup>
Expand Down Expand Up @@ -786,6 +918,28 @@ Links:

***

### .spec.jobsTemplates.training.cpu.env

Type: `core.EnvVar` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L33)</sup>

Env keeps the information about environment variables provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core)

***

### .spec.jobsTemplates.training.cpu.envFrom

Type: `core.EnvFromSource` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L38)</sup>

EnvFrom keeps the information about environment variable sources provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core)

***

### .spec.jobsTemplates.training.cpu.hostIPC

Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/container_namespace.go#L33)</sup>
Expand Down Expand Up @@ -933,6 +1087,28 @@ Links:

***

### .spec.jobsTemplates.training.gpu.env

Type: `core.EnvVar` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L33)</sup>

Env keeps the information about environment variables provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core)

***

### .spec.jobsTemplates.training.gpu.envFrom

Type: `core.EnvFromSource` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L38)</sup>

EnvFrom keeps the information about environment variable sources provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core)

***

### .spec.jobsTemplates.training.gpu.hostIPC

Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/container_namespace.go#L33)</sup>
Expand Down
22 changes: 22 additions & 0 deletions docs/api/ArangoMLStorage.V1Alpha1.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,28 @@ Default Value: `/`

***

### .spec.mode.sidecar.env

Type: `core.EnvVar` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L33)</sup>

Env keeps the information about environment variables provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core)

***

### .spec.mode.sidecar.envFrom

Type: `core.EnvFromSource` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/envs.go#L38)</sup>

EnvFrom keeps the information about environment variable sources provided to the container

Links:
* [Kubernetes Docs](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core)

***

### .spec.mode.sidecar.image

Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/shared/v1/image.go#L31)</sup>
Expand Down
13 changes: 13 additions & 0 deletions pkg/apis/shared/v1/core_container_spec.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ type ContainerTemplate struct {

// SecurityContainer keeps the security settings for Container
*SecurityContainer `json:",inline"`

// Environments keeps the environment variables for Container
*Environments `json:",inline"`
}

func (a *ContainerTemplate) With(other *ContainerTemplate) *ContainerTemplate {
Expand All @@ -52,6 +55,7 @@ func (a *ContainerTemplate) With(other *ContainerTemplate) *ContainerTemplate {
Image: a.GetImage().With(other.GetImage()),
Resources: a.GetResources().With(other.GetResources()),
SecurityContainer: a.GetSecurityContainer().With(other.GetSecurityContainer()),
Environments: a.GetEnvironments().With(other.GetEnvironments()),
}
}

Expand All @@ -71,6 +75,14 @@ func (a *ContainerTemplate) GetSecurityContainer() *SecurityContainer {
return a.SecurityContainer
}

func (a *ContainerTemplate) GetEnvironments() *Environments {
if a == nil || a.Environments == nil {
return nil
}

return a.Environments
}

func (a *ContainerTemplate) GetResources() *Resources {
if a == nil || a.Resources == nil {
return nil
Expand All @@ -87,5 +99,6 @@ func (a *ContainerTemplate) Validate() error {
a.GetImage().Validate(),
a.GetResources().Validate(),
a.GetSecurityContainer().Validate(),
a.GetEnvironments().Validate(),
)
}
62 changes: 62 additions & 0 deletions pkg/apis/shared/v1/envs.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
//
// DISCLAIMER
//
// Copyright 2023 ArangoDB GmbH, Cologne, Germany
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// Copyright holder is ArangoDB GmbH, Cologne, Germany
//

package v1

import (
core "k8s.io/api/core/v1"

"github.com/arangodb/kube-arangodb/pkg/util/k8sutil/envs"
)

type Environments struct {
// Env keeps the information about environment variables provided to the container
// +doc/type: core.EnvVar
// +doc/link: Kubernetes Docs|https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envvar-v1-core
Env []core.EnvVar `json:"env,omitempty"`

// EnvFrom keeps the information about environment variable sources provided to the container
// +doc/type: core.EnvFromSource
// +doc/link: Kubernetes Docs|https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#envfromsource-v1-core
EnvFrom []core.EnvFromSource `json:"envFrom,omitempty"`
}

func (e *Environments) With(other *Environments) *Environments {
if e == nil && other == nil {
return nil
}

if e == nil {
return other.DeepCopy()
}

if other == nil {
return e.DeepCopy()
}

return &Environments{
Env: envs.MergeEnvs(e.Env, other.Env...),
EnvFrom: envs.MergeEnvFrom(e.EnvFrom, other.EnvFrom...),
}
}

func (e *Environments) Validate() error {
return nil
}
Loading

0 comments on commit a496add

Please sign in to comment.