Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Namespaces #1459

Merged
merged 48 commits into from
Apr 5, 2023
Merged

Namespaces #1459

merged 48 commits into from
Apr 5, 2023

Conversation

markphelps
Copy link
Collaborator

This PR is the culmination of many smaller PRs which ships the backend/server portion of supporting namespaces. This PR is purely for transparency and to give us the opportunity to give it one final 👀 as the code in it has been reviewed separately in the smaller PRs.

markphelps and others added 30 commits March 1, 2023 12:35
…1368)

* feat: (sqlite) wip support namespace at storage level, mostly for flags

* chore: rename down namespace file correctly

* chore: fix server/flag_test.go

* chore: redo migration to use temp_tables for composite keys

* chore: just use TODO for now for down migration

* chore: comments

* chore: wip namespace protos

* chore: better err messages

* Update internal/storage/sql/common/flag.go

Co-authored-by: George <[email protected]>

* chore: fix tests

* chore: split into seperate migrations

* chore: more error messages

* chore: rm println statment

---------

Co-authored-by: George <[email protected]>
* main:
  chore(deps): bump github.com/grpc-ecosystem/grpc-gateway/v2 (#1379)
  chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc (#1376)
  chore(deps): bump github.com/grpc-ecosystem/grpc-gateway/v2 in /_tools (#1375)
  chore(deps): bump google.golang.org/grpc/cmd/protoc-gen-go-grpc (#1374)
  chore: add Prose logo to README (#1373)
* feat: wip namespace storage

* chore: add tests; storage impl

* chore: appease the linter

* chore: less sleepy
* feat: wip namespace storage

* chore: add tests; storage impl

* chore: appease the linter

* feat: (wip) add namespace mysql migrations

* chore: rename migrations

* chore: rm empty migrations for the moment

* chore: less sleepy

* chore: 6 bytes for timestamp

* chore: add rules migration

* chore: add migration comments, revert local config

* chore: reset mysql example
* feat: (wip) postgres ddl

* chore: fix tests

* chore: disable container logging unless verbose env var passed
* feat: cockroachdb migrations

* chore: fix migrations for cockroach
* chore: add segment namespace storage tests

* chore: regen protos

* chore: ignore sdk dir when fmting
* chore: add remaining storage layer tests

* feat: add namespaces server mappings

* chore: add back default namespace to otel metrics

* chore: fix middleware test
…#1422)

* feat: add namespaces server impl

* chore: spacing

* chore: rm ability to set protected

* feat: check for protected or flags existing when deleting a namespace

* chore: fix test

* chore: add test for non-existing delete
…1436)

* chore: regenerate protobuf

* feat(rpc/flipt): configure gateway routes for namespaces

* feat(hack/build): add cases for namespace scoped integration tests

* chore(rpc/flipt): remove TODO

* chore(hack/build): use random string for namespace key

* test(hack/build): ensure default namespace cannot be deleted

* test(hack/build): ensure default namespace is protected but updateable

* test(hack/build): ensure namespace with flags cannot be deleted
* main:
  chore(deps): bump github.com/grpc-ecosystem/go-grpc-middleware (#1441)
  chore(deps): bump github.com/testcontainers/testcontainers-go (#1442)
  chore(deps): bump github.com/Masterminds/squirrel from 1.5.3 to 1.5.4 (#1443)
  chore(deps): bump go.opentelemetry.io/otel/exporters/jaeger (#1439)
#1446)

* fix(migrations/sqlite): reorder to ensure we copy all data before drop

* fix(migrations/sqlite): copy distributions into temp table before dropping everything
yquansah and others added 13 commits March 29, 2023 09:53
feat: add checks and tests for potential `namespace`d related failures
* chore(deps): bump go.opentelemetry.io/otel/exporters/zipkin (#1440)

Bumps [go.opentelemetry.io/otel/exporters/zipkin](https://github.com/open-telemetry/opentelemetry-go) from 1.13.0 to 1.14.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@v1.13.0...v1.14.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/zipkin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat(github): dagger based integrations tests github workflow (#1428)

* feat(github): install dagger based ITs as action workflow

* fix(github): add workdir to mage action

* chore(hack/build): bump dagger to 0.5.1

* chore: enable daggers experimental gha cache

* chore(hack/build): reword test log lines

* fix(github): move env var from legacy to experimental tests

* chore(hack/build): arbitrary change to trigger rebuild

* chore(github): remove experimental dagger cache env var

* feat(hack/build): direct integration test flipt logs into directory

* chore: empty commit to trigger CI

* chore(hack/build): log error when failing to copy flipt logs

* chore(hack/build): adding log line to integration test start

* fix(github): stop caching entire hack build directory

* refactor(hack/build): organize integration cases into test case struct

* refactor(github): remove legacy API workflows and replace with dagger mage task

* chore(hack/build): use protocolPorts map for test case iteration

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore: wip rm down migrations

* chore: fix tests

* chore: fix tests; cleanup

* chore: use new migrator on import if drop specified

* chore: fix drop before import for sqlite

* chore: move mysql fk disable into migrator down method

* chore: codecov's last chance
* refactor(export): promote namespace to NewExporter argument

* refactor(export): use GRPC service interface as exporter dependency

* feat(export): support --export-from-address remote flipt address flag

* refactor(cmd): share config as argument not global state

* refactor(cmd/export): move command boostrapping from main.go to export.go

* refactor(cmd): move signal trapping into main

* refactor(cmd/import): move command boostrapping from main.go to import.go

* refactor(ext): expose namespace as argument to NewImporter

* feat(cmd/import): support remote import via --import-to-address

* refactor(hack/build): restructure testing packages

* refactor(hack/build): create import and export integration tests

* feat(hack/build): more import integration test cases

* feat(cmd/export): support --namespace flag

* chore: go mod tidy

* fix(cmd): do not close db early in import/export

* refactor(cmd): rename import/export flags to --address and --token

* refactor(cmd/import): move server config into server.go
* chore: validation logic for namespace requests

* chore: fix import command to use the namespace key as the name as well
fix(cmd/import): pass namespace key as name on create
@markphelps markphelps requested a review from a team as a code owner April 5, 2023 13:45
@codecov
Copy link

codecov bot commented Apr 5, 2023

Codecov Report

Merging #1459 (5218a4b) into main (f04ddb1) will decrease coverage by 0.40%.
The diff coverage is 77.69%.

@@            Coverage Diff             @@
##             main    #1459      +/-   ##
==========================================
- Coverage   78.17%   77.77%   -0.40%     
==========================================
  Files          43       44       +1     
  Lines        3239     3402     +163     
==========================================
+ Hits         2532     2646     +114     
- Misses        565      603      +38     
- Partials      142      153      +11     
Impacted Files Coverage Δ
internal/storage/sql/migrator.go 20.00% <0.00%> (-3.46%) ⬇️
internal/server/middleware/grpc/middleware.go 57.06% <55.55%> (-0.52%) ⬇️
internal/ext/importer.go 69.62% <73.21%> (-5.94%) ⬇️
internal/server/namespace.go 75.34% <75.34%> (ø)
internal/ext/exporter.go 88.37% <100.00%> (+2.26%) ⬆️
internal/server/evaluator.go 94.73% <100.00%> (+0.34%) ⬆️
internal/server/flag.go 82.95% <100.00%> (+1.02%) ⬆️
internal/server/rule.go 76.92% <100.00%> (ø)
internal/server/segment.go 79.16% <100.00%> (ø)
internal/storage/sql/db.go 91.36% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@GeorgeMac GeorgeMac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

speed read

LGTM

@markphelps markphelps merged commit 1eb2750 into main Apr 5, 2023
@markphelps markphelps deleted the namespaces branch April 5, 2023 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants