Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix example to work with KCL #123

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jnewbigin
Copy link

*Issue #122 and #11
Description of changes:

  • Remove explicit_hash_key
  • Keep the same partition_key for every user_record in a kinesis_record

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* Remove explicit_hash_key
* Keep the same partition_key for every user_record in a kinesis_record

See awslabs#11
@IanMeyers
Copy link
Contributor

IanMeyers commented Nov 19, 2020

Hi,

Thanks for this, but I don't think that the method signature matches your changes. The interface is to send a random record, which implies that it will have a random partition key. If you set the same partition key on every record, then you will only target one shard and a single KCL process thread. By sending differing values within the User Records, you allow the Aggregation Code to generate the correct protocol buffer encoded Map of messages, which should then spread across many consumers.

Thx,

Ian

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants