Skip to content

Commit

Permalink
Ready For Review: in response to Automate Onboarding to Cluster via CLI
Browse files Browse the repository at this point in the history
#24 (#44)

* Kustomize model changes, kustomize util changes, and related other model, api, api/testdata and go mod/sum changes

* Onboard related model, constant, cmd, api, and api/testdata changes or additions

* Documentation and Administrative changes (README.md and OWNERS)
  • Loading branch information
Gregory-Pereira authored Mar 24, 2022
1 parent a789dbd commit 460b3e2
Show file tree
Hide file tree
Showing 84 changed files with 1,903 additions and 206 deletions.
1 change: 1 addition & 0 deletions OWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ approvers:
- durandom
- tumido
- HumairAK
- Gregory-Pereira

reviewers:
- hemajv
Expand Down
116 changes: 111 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Available Commands:
enable-monitoring Enable monitoring for a Kubernetes namespace
grant-access Grant a group access to a namespace
help Help about any command
onboard Creates Groups, Namespaces, and Roles for a new Operate First project
Flags:
-a, --app-name string application name (default "cluster-scope")
Expand All @@ -36,7 +37,7 @@ Flags:
-R, --repodir string path to opf repository
```

### create-group
## create-group

```
Create a group.
Expand All @@ -47,10 +48,13 @@ Usage:
opfcli create-group group [flags]
Flags:
-h, --help help for create-project
-h, --help help for create-project
-d, --display-name short team description for easy identification of project
-n do not set a limitrange on this project
-u, --users comma seperated list of users to add to the group
```

### create-project
## create-project

```
Onboard a new project into Operate First.
Expand All @@ -76,7 +80,7 @@ Global Flags:
## enable-monitoring

```
Enable monitoring fora Kubernetes namespace.
Enable monitoring for a Kubernetes namespace.
This will add a RoleBinding to the target namespace that permits
Prometheus to access certain metrics about pods, services, etc.
Expand All @@ -88,7 +92,7 @@ Flags:
-h, --help help for enable-monitoring
```

### grant-access
## grant-access

```
Grant a group access to a namespace.
Expand All @@ -103,6 +107,108 @@ Flags:
-h, --help help for grant-access
```

## Onboard

Onboard everything necessary for a new application into the Operate-First environment.

The onboard command expects an onboarding configuration file. These are the necessary and optional paramaters that make up the onboarding configuration file.

Necessary parameters:
- `team_name` = name of the group looking to be onboarded
- `<namespaces[i]>.name` = the name of the namespace(s) for you project (defined per namespace)
- `project_description` = short summary of your project
- `target_cluster` = name of the cluster to onboard to
- `env` = name of the environment in which the `target_cluster` lives

Optional parameters:
- `<namespaces[i]>.quota` = Namespace Quota
- `<namespaces[i]>.custom_quota` = a customizeable resource quota to be applied to its namespace
- `users` = users to be given access to the project when onboarding


This is an example of a valid configuration file using every option. Below each component will be discussed.

```
env: MOC
namespaces:
- enable_monitoring: false
name: testproject
quota: testquota
disable_limit_range: false
project_display_name: testprojectdisplayname
custom_quota:
limits.cpu: '28'
requests.cpu: '28'
limits.memory: 32Gi
requests.memory: 32Gi
requests.storage: 100Gi
count/objectbucketclaims.objectbucket.io: 1
project_description: This is the configuration for a sample project / app to onboard
target_cluster: Smaug
team_name: testgroup
users:
- testing_gh_handle_1
- testing_gh_handle_2
```

### `target_cluster` and `env`

Values for `target_cluster` can be obtained by running:
```
kustomize build https://github.com/operate-first/apps/acm/overlays/moc/infra/managedclusters?ref=master | yq e -N '.metadata.name' -
```
- Requires: [yq](https://github.com/mikefarah/yq#install) and [kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/)

Similarly, values for `env` can be obtained by running:

```
curl -sX GET https://api.github.com/repos/operate-first/apps/contents/argocd/overlays/moc-infra/applications/envs | yq e '.[].name' -
```
- Requires: [yq](https://github.com/mikefarah/yq#install)

NOTE: the `opfcli` is made for use in `operate-first/apps` which is why these urls are either `curl`ed or used in kustomize build.

If you cannot install `kustomize` and or `yq`, our last known options for `env` and `target_cluster` are as follows:

- env: `moc`
- target_cluster: `smaug`
- target_cluster: `infra`
- env: `osc`
- target_cluster: `osc-cl1`
- env: `emea`:
- target_cluster: `rick`
- target_cluster: `morty`

### Namespace-Scoped Configurations

1. `quota`s: Quota's are optional values that refer to names of common [ResourceQuotas in `operate-first/apps`](https://github.com/operate-first/apps/tree/master/cluster-scope/components/resourcequotas). They limit the resources able to be used in a specific namespace.

2. `custom_quota`s: Custom quotas are optional configurations for a custom `ResourceQuota` to be used in a namespace. The values of a custom quota reflect those of our resourcequotas defined per namespace in operate-first/apps [see example](https://github.com/operate-first/apps/blob/master/cluster-scope/base/core/namespaces/sandbox/resourcequota.yaml).
- If a valid `quota` is selected but a `custom_quota` is also defined in your configuration file, it will default to creating the namespace using the `custom_quota`.
- All values are optional in a `custom_quota`.
- `custom_quota` options: `[limits.cpu, requests.cpu, limits.memory, requests.memroy, requests.storage, count/objectbucketclaims.objectbucket.io]`

3. `disable_limit_range`: All requests to create and modify resources in Openshift are evaluated against each `LimitRange` object in the project. If the resource violates any of the enumerated constraints, the resource is rejected. Setting `disable_limit_range` to `true`, creates a namespace without the [operate-first default limit range](https://github.com/operate-first/apps/blob/master/cluster-scope/components/limitranges/default/limitrange.yaml).

4. `display_project_name`: display-names in openshift are user-defined strings which provide an alternative, more human-readable way to refer to a namespace. For more information on this checkout the [openshift docs](https://docs.openshift.com/online/pro/architecture/core_concepts/projects_and_users.html#projects) on projects.

### Users

GitHub handles that will be used to authenticate OCP clusters via GitHub. These GitHub usernames are converted to OCP users and are also used to extend permissions via Openshift RBAC.

### Usage

```
Usage:
opfcli onboard <file-path>
Flags:
-d, --display-name provide a shorter name to use to refer to the project in openshift
-n, --no-limitrange Do not set a resource limitrange on this project
```

Use "opfcli [command] --help" for more information about a command.

## Configuration
Expand Down
2 changes: 1 addition & 1 deletion api/AddComponent.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ func (api *API) AddComponent(projectName, componentName string) error {
log.Printf("adding component %s to project %s", componentName, projectName)
err = utils.AddKustomizeComponent(
nsPath,
filepath.Join(constants.ComponentRelPath, componentName),
[]string{filepath.Join(constants.ComponentRelPath, componentName)},
)
if err != nil {
return err
Expand Down
2 changes: 1 addition & 1 deletion api/AddGroupRBAC.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ func (api *API) AddGroupRBAC(projectName, groupName, roleName string) error {
log.Printf("granting %s role %s on %s", groupName, roleName, projectName)
err = utils.AddKustomizeComponent(
nsPath,
filepath.Join(constants.ComponentRelPath, bindingName, groupName),
[]string{filepath.Join(constants.ComponentRelPath, bindingName, groupName)},
)
if err != nil {
return err
Expand Down
1 change: 1 addition & 0 deletions api/AddGroupRBAC_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ func (suite *apiTestSuite) TestAddGroupRBAC() {
// Should work if both project and group exist
err = suite.api.CreateGroup(
"testgroup2",
[]string{},
false,
)
assert.Nil(err)
Expand Down
60 changes: 60 additions & 0 deletions api/CreateCustomResourceQuota.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
package api

import (
// "fmt"

"fmt"
"io/ioutil"
"path/filepath"

"github.com/operate-first/opfcli/constants"
"github.com/operate-first/opfcli/models"
"github.com/operate-first/opfcli/utils"
log "github.com/sirupsen/logrus"
)

func (api *API) CreateCustomResourceQuota(namespace string, customResourceQuota models.CustomResourceQuota, existsOk bool) error {
path := filepath.Join(
api.RepoDirectory, api.AppName,
constants.NamespacePath, namespace, "resourcequota.yaml",
)

exists, err := utils.PathExists(path)
if err != nil {
return err
}

if exists {
if existsOk {
log.Warnf("custom resource for namespace %s already exists (continuing)", namespace)
return nil
}
return fmt.Errorf("resource quota already exists in namespace %s", namespace)
}
customResource := models.NewCustomResourceQuota(namespace, customResourceQuota)
customResourceOut, err := models.ToYAML(customResource)
if err != nil {
return err
}

log.Printf("writing ResourceQuota definition to %s", filepath.Dir(path))
if exist, err := utils.PathExists(filepath.Dir(path)); err != nil || !exist {
if exist {
return fmt.Errorf("error writing resource quota. error: %s", err)
}
return fmt.Errorf("directory should already have been created for the namespace. directory %s does not exist", namespace)
}

err = ioutil.WriteFile(path, customResourceOut, 0644)
if err != nil {
return fmt.Errorf("failed to write resource quota file: %w", err)
}

err = utils.AddKustomizeResources(filepath.Dir(path), []string{"resourcequota.yaml"})
if err != nil {
return fmt.Errorf("failed to append ResourceQuota resource to kustomization file")
}

return nil

}
36 changes: 36 additions & 0 deletions api/CreateCustomResourceQuota_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
package api

import (
"os"
"path/filepath"

"github.com/operate-first/opfcli/constants"
"github.com/operate-first/opfcli/models"
"github.com/operate-first/opfcli/utils"
"github.com/stretchr/testify/require"
)

func (suite *apiTestSuite) TestCreateCustomResourceQuota() {
assert := require.New(suite.T())
path := filepath.Join(
suite.api.RepoDirectory, suite.api.AppName,
constants.NamespacePath, "testproject", "kustomization.yaml",
)
// should fail if no namespace is present when attempting to append the kustomization
err := suite.api.CreateCustomResourceQuota("test", models.CustomResourceQuota{}, true)
assert.EqualError(err, "directory should already have been created for the namespace. directory test does not exist")

// creates namespace directory
err = os.MkdirAll(filepath.Dir(path), 0755)
assert.Nil(err)

// should fail because kustomization file in namespace directory does not exist, and so cannot append resource quota value to it
err = suite.api.CreateCustomResourceQuota("testproject", models.CustomResourceQuota{}, true)
assert.EqualError(err, "failed to append ResourceQuota resource to kustomization file")

// should succeed
kustom := models.NewKustomization([]string{"namespace.yaml"}, []string{"../../../../components/limitranges/default", "../../../../components/project-admin-rolebindings/testgroup"}, "testgroup")
err = utils.WriteKustomization(filepath.Dir(path), kustom)
assert.Nil(err)

}
15 changes: 9 additions & 6 deletions api/CreateGroup.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,13 @@ import (
"os"
"path/filepath"

log "github.com/sirupsen/logrus"

"github.com/operate-first/opfcli/constants"
"github.com/operate-first/opfcli/models"
"github.com/operate-first/opfcli/utils"
log "github.com/sirupsen/logrus"
)

func (api *API) CreateGroup(groupName string, existsOk bool) error {
func (api *API) CreateGroup(groupName string, users []string, existsOk bool) error {
path := filepath.Join(
api.RepoDirectory, api.AppName,
constants.GroupPath, groupName, "group.yaml")
Expand All @@ -31,8 +30,11 @@ func (api *API) CreateGroup(groupName string, existsOk bool) error {
return fmt.Errorf("group %s already exists", groupName)
}

group := models.NewGroup(groupName)
groupOut := models.ToYAML(group)
group := models.NewGroup(groupName, users)
groupOut, err := models.ToYAML(group)
if err != nil {
return err
}

log.Printf("writing group definition to %s", filepath.Dir(path))
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
Expand All @@ -49,7 +51,8 @@ func (api *API) CreateGroup(groupName string, existsOk bool) error {
nil,
"",
)
err = kustom.Write(filepath.Dir(path))

err = utils.WriteKustomization(filepath.Dir(path), kustom)
if err != nil {
return err
}
Expand Down
Loading

0 comments on commit 460b3e2

Please sign in to comment.