From ede34d2da8e11c056fd904ca836f9bbcc9ea75ed Mon Sep 17 00:00:00 2001 From: Paul Latzelsperger <43503240+paullatzelsperger@users.noreply.github.com> Date: Tue, 21 Mar 2023 16:16:23 +0100 Subject: [PATCH] docs: create decision record about splitting repos (#2597) * docs: create decision record about splitting repos * pr remarks --- .../2023-03-15_repository_split/README.md | 79 +++++++++++++++++++ docs/developer/decision-records/README.md | 1 + 2 files changed, 80 insertions(+) create mode 100644 docs/developer/decision-records/2023-03-15_repository_split/README.md diff --git a/docs/developer/decision-records/2023-03-15_repository_split/README.md b/docs/developer/decision-records/2023-03-15_repository_split/README.md new file mode 100644 index 00000000000..ab6266d5ce5 --- /dev/null +++ b/docs/developer/decision-records/2023-03-15_repository_split/README.md @@ -0,0 +1,79 @@ +# Splitting the Connector repository + +## Decision + +The Connector repository will be split up into a "core" repository and technology repositories. Extensions that are +specific to a particular cloud environment will get moved out into new repositories to which we will refer as +"technology repos". +All technology-specific extensions of all the other repos (FederatedCatalog, IdentityHub, RegistrationService) will be +moved out as well. + +The Connector repository will _not_ get renamed, but it can henceforth be referred to as "core" or "connector-core". + +## Rationale + +Many third-party technology extensions are very specific to a particular cloud provider, and cannot reasonably be used +across multiple clouds, thus they depend on specific libraries/SDKs and testing them may even require an account with +that cloud provider. + +The reasoning for moving these kinds of extensions out to separate repositories is as follows: + +- keeping the core small and agile: building and running it should be fast and efficient, no third-party account should + be required. +- simplifying the core's CI builds: the build should be self-contained using GitHub services which are docker + containers. Checking for the presence of account keys or other service credentials in CI builds is thus obsolete. +- reducing build time: not having to build and test many extensions everytime will significantly improve the developer + experience, because build times will be shorter. +- decoupling responsibilities: contributing in one repository requires only specific knowledge about that + particular tech, as opposed to: understanding of the _entire_ code base. Maintainer teams with specific knowledge can + be established. +- different contribution criteria: while the criteria for adoption in the core should be kept very strict, the same + might + not necessarily be true for technology repos. There, features could get adopted simply for the sake of having them, as + long as they are properly maintained. While in the core there would have to be a specific reason to _adopt_ a feature, + in the technology repos there would have to be a specific reason to _deny_ them. +- decoupling of lifecycles: archiving/abandoning a particular technology repo would not influence the core at all, + should we ever need to do that. + +## Approach + +At the time of this writing three technology repositories can be identified: + +- `Technology-Azure` +- `Technology-Aws` +- `Technology-Gcp` + +Once these repositories are created, all extensions, that contain code for services, that are cloud-specific, will be +moved out accordingly. Initially, this will just be a code dump in the technology repos, but all further development +will then happen there. +To maintain consistency, all specific cloud-provider-extensions will go in the technology repo, e.g. Azure: KeyVault, +BlobStore, CosmosDB etc. + +### Continuous Integration + +Technology repos should maintain all access keys, service credentials etc. that are needed to run integration tests +against a live testing environment as repo secrets. For example, running i-tests against an actual CosmosDB instance +requires a secret key. These secrets are kept per repository, as opposed to: per GitHub org. + +CI workflows must take that into account, and also consider a situation where they are run from a fork. + +### Version Catalogs + +Technology repositories should publish their own version catalogs. All technology-specific entries will be removed by +the core's version catalog. + +## Further Consequences + +While the contribution standards for the Connector core will remain high (if anything, they will get raised even more), +technology repositories may have different contribution guidelines. In technology repos, "feature completeness" could +be a sufficient justification to adopt a feature. Cloud providers may want to create a way to run a connector +exclusively using their technology, and for that, they may want a wide variety of services. + +Every technology repository should define a team of contributors, who primarily take care of maintaining the code in the +repo. + +If a technology repo is not properly maintained, and is not ready for release at a predetermined time, the project +committee may elect to omit it from the release. + +When a repository loses its maintainers, or development becomes otherwise stale, the project committee can elect to +archive the repository after an appropriate notice period. \ No newline at end of file diff --git a/docs/developer/decision-records/README.md b/docs/developer/decision-records/README.md index f1ac8fae00f..615ef063153 100644 --- a/docs/developer/decision-records/README.md +++ b/docs/developer/decision-records/README.md @@ -54,4 +54,5 @@ - [2023-02-27 Dataspace Protocol TransferProcess state transitions](2023-02-27-dataspace-protocol-transferprocess-state-transitions) - [2023-03-02 Entity Store Refactoring](2023-03-02_entity_store_refactoring) - [2023-03-09-Event Framework Refactoring](2023-03-09-event-framework-refactoring) +- [2023-03-15-Repository_Split](2023-03-15_repository_split/)