Skip to content

Commit

Permalink
Introduce shared.Object
Browse files Browse the repository at this point in the history
  • Loading branch information
nikita-vanyasin committed Nov 27, 2023
1 parent 458ea80 commit 6cab685
Show file tree
Hide file tree
Showing 9 changed files with 197 additions and 60 deletions.
46 changes: 28 additions & 18 deletions docs/api/ArangoMLStorage.V1Alpha1.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

### .spec.backend.s3.allowInsecure

Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L42)</sup>
Type: `boolean` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L43)</sup>

AllowInsecure if set to true, the Endpoint certificates won't be checked

Expand All @@ -14,38 +14,48 @@ Default Value: `false`

### .spec.backend.s3.bucketName

Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L36)</sup>
Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L37)</sup>

BucketName specifies the name of the bucket
Required

***

### .spec.backend.s3.caSecretName
### .spec.backend.s3.caSecret.name

Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L48)</sup>
Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/batchjob_status.go#L23)</sup>

CASecretName if not empty, the given secret will be used to check the authenticity of Endpoint
The specified `Secret`, must contain the following data fields:
- `ca.crt` PEM encoded public key of the CA certificate
- `ca.key` PEM encoded private key of the CA certificate
Name of the object

Default Value: `""`
***

### .spec.backend.s3.caSecret.namespace

Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/cronjob.go#L6)</sup>

Namespace of the object. Should default to the namespace of the parent object

***

### .spec.backend.s3.credentialsSecretName
### .spec.backend.s3.credentialsSecret.name

Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L39)</sup>
Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/batchjob_status.go#L23)</sup>

CredentialsSecretName specifies the name of the secret containing AccessKey and SecretKey for S3 API authorization
Required
Name of the object

***

### .spec.backend.s3.credentialsSecret.namespace

Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/cronjob.go#L6)</sup>

Namespace of the object. Should default to the namespace of the parent object

***

### .spec.backend.s3.endpoint

Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L33)</sup>
Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L34)</sup>

Endpoint specifies the S3 API-compatible endpoint which implements storage
Required
Expand All @@ -54,17 +64,17 @@ Required

### .spec.backend.s3.region

Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L51)</sup>
Type: `string` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go#L52)</sup>

Region defines the availability zone name. If empty, defaults to 'us-east-1'
Region defines the availability zone name.

Default Value: `""`

***

### .spec.mode.sidecar.listenPort

Type: `integer` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_mode_sidecar.go#L40)</sup>
Type: `integer` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_mode_sidecar.go#L41)</sup>

ListenPort defines on which port the sidecar container will be listening for connections

Expand All @@ -74,7 +84,7 @@ Default Value: `9201`

### .spec.mode.sidecar.resources

Type: `core.ResourceRequirements` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_mode_sidecar.go#L45)</sup>
Type: `core.ResourceRequirements` <sup>[\[ref\]](https://github.com/arangodb/kube-arangodb/blob/1.2.35/pkg/apis/ml/v1alpha1/storage_spec_mode_sidecar.go#L46)</sup>

Resources holds resource requests & limits for container running the S3 proxy

Expand Down
15 changes: 13 additions & 2 deletions internal/docs_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,8 @@ func Test_GenerateAPIDocs(t *testing.T) {
root := os.Getenv("ROOT")
require.NotEmpty(t, root)

sharedFields, sharedFilesSet := parseSourceFiles(t, fmt.Sprintf("%s/pkg/apis/shared/v1", root))

// package path -> result doc file name -> name of the top-level field to be described -> field instance for reflection
input := map[string]map[string]map[string]interface{}{
fmt.Sprintf("%s/pkg/apis/deployment/v1", root): {
Expand Down Expand Up @@ -180,8 +182,17 @@ func Test_GenerateAPIDocs(t *testing.T) {

resultPaths := make(map[string]string)
for apiDir, docs := range input {
fields, fileSets := parseSourceFiles(t, apiDir)
util.CopyMap(resultPaths, generateDocs(t, docs, fields, fileSets))
fields, fileSet := parseSourceFiles(t, apiDir)

for n, f := range sharedFields {
fields[n] = f
}
sharedFilesSet.Iterate(func(file *token.File) bool {
fileSet.AddFile(file.Name(), fileSet.Base()+file.Base(), file.Size())
return true
})

util.CopyMap(resultPaths, generateDocs(t, docs, fields, fileSet))
}
generateIndex(t, resultPaths)
}
Expand Down
44 changes: 24 additions & 20 deletions pkg/apis/ml/v1alpha1/storage_spec_backend_s3.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
"net/url"

"github.com/arangodb/kube-arangodb/pkg/apis/shared"
sharedApi "github.com/arangodb/kube-arangodb/pkg/apis/shared/v1"
"github.com/arangodb/kube-arangodb/pkg/util/errors"
)

Expand All @@ -34,19 +35,19 @@ type ArangoMLStorageSpecBackendS3 struct {
// BucketName specifies the name of the bucket
// Required
BucketName *string `json:"bucketName"`
// CredentialsSecretName specifies the name of the secret containing AccessKey and SecretKey for S3 API authorization
// CredentialsSecret specifies the Kubernetes Secret containing AccessKey and SecretKey for S3 API authorization
// Required
CredentialsSecretName *string `json:"credentialsSecretName"`
CredentialsSecret *sharedApi.Object `json:"credentialsSecret"`
// AllowInsecure if set to true, the Endpoint certificates won't be checked
// +doc/default: false
AllowInsecure *bool `json:"allowInsecure,omitempty"`
// CASecretName if not empty, the given secret will be used to check the authenticity of Endpoint
// The specified `Secret`, must contain the following data fields:
// CASecret if not empty, the given Kubernetes Secret will be used to check the authenticity of Endpoint
// The specified Secret, must contain the following data fields:
// - `ca.crt` PEM encoded public key of the CA certificate
// - `ca.key` PEM encoded private key of the CA certificate
// +doc/default: ""
CASecretName *string `json:"caSecretName,omitempty"`
// Region defines the availability zone name. If empty, defaults to 'us-east-1'
// +doc/default: nil
CASecret *sharedApi.Object `json:"caSecret,omitempty"`
// Region defines the availability zone name.
// +doc/default: ""
Region *string `json:"region,omitempty"`
}
Expand All @@ -59,20 +60,23 @@ func (s *ArangoMLStorageSpecBackendS3) Validate() error {
var errs []error

if s.GetBucketName() == "" {
errs = append(errs, errors.New("bucketName must be not empty"))
errs = append(errs, shared.PrefixResourceErrors("bucketName", errors.New("must be not empty")))
}

if s.GetEndpoint() == "" {
errs = append(errs, errors.New("endpoint must be not empty"))
errs = append(errs, shared.PrefixResourceErrors("endpoint", errors.New("must be not empty")))
}

if _, err := url.Parse(s.GetEndpoint()); err != nil {
errs = append(errs, errors.Newf("invalid endpoint URL was provided: %s", err.Error()))
errs = append(errs, shared.PrefixResourceErrors("endpoint", errors.Newf("invalid URL: %s", err.Error())))
}

if s.GetCredentialsSecretName() == "" {
errs = append(errs, errors.New("credentialsSecretName must be not empty"))
errs = append(errs, shared.PrefixResourceErrors("credentialsSecret", s.GetCredentialsSecret().Validate()))

if caSecret := s.GetCASecret(); !caSecret.IsEmpty() {
errs = append(errs, shared.PrefixResourceErrors("caSecret", caSecret.Validate()))
}

return shared.WithErrors(errs...)
}

Expand All @@ -90,11 +94,11 @@ func (s *ArangoMLStorageSpecBackendS3) GetBucketName() string {
return *s.BucketName
}

func (s *ArangoMLStorageSpecBackendS3) GetCredentialsSecretName() string {
if s == nil || s.CredentialsSecretName == nil {
return ""
func (s *ArangoMLStorageSpecBackendS3) GetCredentialsSecret() *sharedApi.Object {
if s == nil || s.CredentialsSecret == nil {
return &sharedApi.Object{}
}
return *s.CredentialsSecretName
return s.CredentialsSecret
}

func (s *ArangoMLStorageSpecBackendS3) GetAllowInsecure() bool {
Expand All @@ -104,11 +108,11 @@ func (s *ArangoMLStorageSpecBackendS3) GetAllowInsecure() bool {
return *s.AllowInsecure
}

func (s *ArangoMLStorageSpecBackendS3) GetCASecretName() string {
if s == nil || s.CASecretName == nil {
return ""
func (s *ArangoMLStorageSpecBackendS3) GetCASecret() *sharedApi.Object {
if s == nil || s.CASecret == nil {
return &sharedApi.Object{}
}
return *s.CASecretName
return s.CASecret
}

func (s *ArangoMLStorageSpecBackendS3) GetRegion() string {
Expand Down
3 changes: 2 additions & 1 deletion pkg/apis/ml/v1alpha1/storage_spec_mode_sidecar.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
core "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"

"github.com/arangodb/kube-arangodb/pkg/apis/shared"
"github.com/arangodb/kube-arangodb/pkg/util/errors"
)

Expand All @@ -50,7 +51,7 @@ func (s *ArangoMLStorageSpecModeSidecar) Validate() error {
s = &ArangoMLStorageSpecModeSidecar{}
}
if s.GetListenPort() < 1 {
return errors.Newf("invalid listenPort value: must be positive")
return shared.PrefixResourceErrors("database", errors.Newf("must be positive"))
}
return nil
}
Expand Down
10 changes: 7 additions & 3 deletions pkg/apis/ml/v1alpha1/storage_spec_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import (
core "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"

sharedApi "github.com/arangodb/kube-arangodb/pkg/apis/shared/v1"
"github.com/arangodb/kube-arangodb/pkg/util"
)

Expand All @@ -49,9 +50,12 @@ func Test_ArangoMLStorageSpec(t *testing.T) {

require.Error(t, s.Backend.S3.Validate())
s.Backend.S3 = &ArangoMLStorageSpecBackendS3{
Endpoint: util.NewType("http://test.s3.example.com"),
BucketName: util.NewType("bucket"),
CredentialsSecretName: util.NewType("a-secret"),
Endpoint: util.NewType("http://test.s3.example.com"),
BucketName: util.NewType("bucket"),
CredentialsSecret: &sharedApi.Object{
Name: "a-secret",
Namespace: nil,
},
}
require.NoError(t, s.Validate())

Expand Down
17 changes: 9 additions & 8 deletions pkg/apis/ml/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

75 changes: 75 additions & 0 deletions pkg/apis/shared/v1/object.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
//
// DISCLAIMER
//
// Copyright 2023 ArangoDB GmbH, Cologne, Germany
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// Copyright holder is ArangoDB GmbH, Cologne, Germany
//

package v1

import (
meta "k8s.io/apimachinery/pkg/apis/meta/v1"

"github.com/arangodb/kube-arangodb/pkg/apis/shared"
"github.com/arangodb/kube-arangodb/pkg/util/errors"
)

type Object struct {
// Name of the object
Name string `json:"name"`

// Namespace of the object. Should default to the namespace of the parent object
Namespace *string `json:"namespace,omitempty"`
}

func (o *Object) IsEmpty() bool {
return o == nil ||
(o.Name == "" && o.Namespace != nil)
}

func (o *Object) GetName() string {
if o == nil {
return ""
}

return o.Name
}

func (o *Object) GetNamespace(obj meta.Object) string {
if o != nil {
if n := o.Namespace; n != nil {
return *n
}
}

return obj.GetNamespace()
}

func (o *Object) Validate() error {
if o == nil {
o = &Object{}
}

var errs []error
if o.Name == "" {
errs = append(errs, shared.PrefixResourceErrors("name", errors.New("must be not empty")))
}
if o.Namespace != nil && *o.Namespace == "" {
errs = append(errs, shared.PrefixResourceErrors("namespace", errors.New("must be nil or non-empty string")))
}

return shared.WithErrors(errs...)
}
Loading

0 comments on commit 6cab685

Please sign in to comment.