Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLOUDP-238527: Atlas streams instance controller #1529

Merged
merged 29 commits into from
Apr 29, 2024
Merged

Conversation

helderjs
Copy link
Collaborator

All Submissions:

  • Have you signed our CLA?
  • Put closes #XXXX in your comment to auto-close the issue that your PR fixes (if there is one).
  • Update docs/release-notes/release-notes.md if your changes should be included in the release notes for the next release.

An integration test will be added in an upcoming PR, the connection controller needs to be finished first as the controllers communicate with each other.

Copy link
Contributor

github-actions bot commented Apr 22, 2024

Copy link
Collaborator

@s-urbaniak s-urbaniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking great 🎉

Generally: We're missing a streams connection controller handling the finalizer of AtlasStreamConnection custom resources.

pkg/api/v1/atlascustomresource.go Outdated Show resolved Hide resolved
pkg/api/v1/atlasstreamconnection_types.go Outdated Show resolved Hide resolved
pkg/api/v1/atlasstreaminstance_types.go Show resolved Hide resolved
pkg/controller/atlasstream/atlasstream_connection.go Outdated Show resolved Hide resolved
pkg/controller/atlasstream/atlasstream_connection.go Outdated Show resolved Hide resolved
pkg/controller/atlasstream/atlasstream_connection.go Outdated Show resolved Hide resolved
cmd/manager/main.go Outdated Show resolved Hide resolved
Comment on lines +92 to +103
for i := range ops.NoOp {
akoStreamConnection := ops.NoOp[i]
ctx.EnsureStatusOption(
status.AtlasStreamInstanceAddConnection(
akoStreamConnection.Spec.Name,
common.ResourceRefNamespaced{
Name: akoStreamConnection.Name,
Namespace: akoStreamConnection.Namespace,
},
),
)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not understand no ops, we are doing something so, are they really no ops?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are connections that didn't change anything in a reconcile loop, we still need to push them in the status.

Comment on lines +242 to +245
func hasStreamConnectionChanged(
streamConnection *akov2.AtlasStreamConnection,
atlasStreamConnection *admin.StreamsConnection,
mapper streamConnectionMapper,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side comment:

With a translation layer this always compares pears to pears, because the admin type would have been already translated before getting here. Eg:

func userMatchesSpec(log *zap.SugaredLogger, atlasUsername, operatorUser *akov2.AtlasDatabaseUserSpec) (bool, error) {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still haven't checked your PR, but yes, if we had a middle layer that wouldn't be needed. However, I'm wondering if the transformation back and forth from both sides worth it.

return r.terminate(workflowCtx, workflow.AtlasAPIAccessNotConfigured, err)
}
workflowCtx.SdkClient = atlasClient
workflowCtx.OrgID = orgID
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure we use this?
In the PoC I could not see usages, so I removed it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this resource reconciliation, we don't need to be aware of the org ID, but workflow context is built in multiple steps and I prefer not to leave it incomplete because the org id is not required by streams.

Copy link
Collaborator

@josvazg josvazg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM when CI agrees

@s-urbaniak
Copy link
Collaborator

we need to do some more logistics around helm

Copy link
Collaborator

@roothorp roothorp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - good unit test coverage 😄

@s-urbaniak s-urbaniak merged commit e4e8217 into main Apr 29, 2024
49 checks passed
@s-urbaniak s-urbaniak deleted the atlas-streams branch April 29, 2024 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants