Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(kinesisfirehose-alpha): refactor sourceStream property to support multiple types of sources #31723

Merged
merged 13 commits into from
Oct 17, 2024

Conversation

paulhcsun
Copy link
Contributor

@paulhcsun paulhcsun commented Oct 11, 2024

Reason for this change

The previous API for source was designed under the assumption that a Source would either be a Stream or Direct Put if not. Since the alpha module was written, support on the service side for MSK as a Source has been added so we should update the source property to accept an ISource which can then be implemented by different types of Sources.

Description of changes

Replaced the sourceStream property with source.

Changed the source property from IStream to ISource.

Added an ISource interface which is implemented by classes which represent the different Source types. Currently implemented by the KinesisStreamSource class. The MSKSource class can be added in a separate PR.

Added a SourceConfig which contains the property configs for each respective source (as the fields within these property configs are different across each source). In delivery-stream.ts we call the _bind method which will populate and return the correct property config for the Source and that gets directly injected where the L1 CFNDeliveryStream is created. This pattern is also used for Destinations:

    const destinationConfig = props.destination.bind(this, {});
    const sourceConfig = props.source?._bind(this, this._role?.roleArn);

    const resource = new CfnDeliveryStream(this, 'Resource', {
      deliveryStreamEncryptionConfigurationInput: encryptionConfig,
      deliveryStreamName: props.deliveryStreamName,
      deliveryStreamType: props.source ? 'KinesisStreamAsSource' : 'DirectPut',
      ...sourceConfig,
      ...destinationConfig,
    });

Description of how you validated changes

no behavioural changes. the updated integ tests and unit tests still pass existing tests.

exempting integ tests because we don't want the generated CFN to change.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license


BREAKING CHANGE: Replaced the sourceStream property with source. Changed the source property type from IStream to ISource. Instead of passing in the source Stream directly, it will be passed in by calling the appropriate class like so: source: new source.KinesisStreamSource(sourceStream).

@github-actions github-actions bot added the p2 label Oct 11, 2024
@aws-cdk-automation aws-cdk-automation requested a review from a team October 11, 2024 00:14
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Oct 11, 2024
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@paulhcsun paulhcsun changed the title WIP - feat(kinesisfirehose-alpha): refactor sourceStream property to support multiple types of sources feat(kinesisfirehose-alpha): refactor sourceStream property to support multiple types of sources Oct 15, 2024
@paulhcsun paulhcsun marked this pull request as ready for review October 15, 2024 16:56
@paulhcsun paulhcsun added the pr-linter/exempt-integ-test The PR linter will not require integ test changes label Oct 15, 2024
@aws-cdk-automation aws-cdk-automation dismissed their stale review October 15, 2024 22:23

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Oct 15, 2024
Copy link
Member

@Leo10Gama Leo10Gama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, just a few nits and a few clarifications on some of the new implementation

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Oct 16, 2024
Copy link
Member

@Leo10Gama Leo10Gama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

mergify bot commented Oct 16, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@Leo10Gama
Copy link
Member

@Mergifyio update

Copy link
Contributor

mergify bot commented Oct 16, 2024

update

❌ Mergify doesn't have permission to update

For security reasons, Mergify can't update this pull request. Try updating locally.
GitHub response: refusing to allow a GitHub App to create or update workflow .github/workflows/request-cli-integ-test.yml without workflows permission

Copy link
Contributor

mergify bot commented Oct 16, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: e0576e0
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 0260046 into aws:main Oct 17, 2024
12 checks passed
Copy link
Contributor

mergify bot commented Oct 17, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 17, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
contribution/core This is a PR that came from AWS. p2 pr-linter/exempt-integ-test The PR linter will not require integ test changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants