Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Task]: Add a default watermark emit interval to FlinkUnboundedSourceReader #30125

Closed
1 of 16 tasks
je-ik opened this issue Jan 26, 2024 · 3 comments
Closed
1 of 16 tasks

[Task]: Add a default watermark emit interval to FlinkUnboundedSourceReader #30125

je-ik opened this issue Jan 26, 2024 · 3 comments
Assignees
@hardshah
Labels
flink good first issue P3 task
Milestone
2.55.0 Release

Comments

@je-ik
Copy link
Contributor

je-ik commented Jan 26, 2024

What needs to happen?

Currently, when autoWatermarkInterval is not set, watermarks are not emitted in FlinkUnboundedSourceReader. This does not work wth Beam's event-time default behavior. Use a sensible default for watermark emission when not explicitly set.

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
@hardshah
Copy link
Contributor

.take-issue

@hardshah hardshah mentioned this issue Jan 29, 2024
Merged
3 tasks
@hardshah
Copy link
Contributor

Hi @je-ik I have submitted a PR for your review whenever you have time.

@je-ik je-ik changed the title [Task]: Add a default watermark emit interval to FlinkUnboundedSourceReaeder [Task]: Add a default watermark emit interval to FlinkUnboundedSourceReader Feb 1, 2024
@je-ik
Copy link
Contributor Author

je-ik commented Feb 6, 2024

Closed via #30141

@je-ik je-ik closed this as completed Feb 6, 2024
@github-actions github-actions bot added this to the 2.55.0 Release milestone Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hardshah hardshah

Labels
flink good first issue P3 task
Projects
None yet
Milestone
2.55.0 Release
Development

No branches or pull requests
2 participants
@je-ik @hardshah