Startup metric solves feature request #179 #180
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…f selected rows in a query
|
Thanks for the PR! I just ran the CI and I could see that tests are failing, are the tests passing locally on your machine? |
|
Thanks, rerunning CI |
|
Okay so CI looks good |
|
This feature looks interesting for us too. Do you know when it will be usable @mdedetrich ? |
Claudenw
requested changes
Feb 24, 2023
| import org.slf4j.LoggerFactory; | ||
|
|
||
| /** | ||
| * Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et |
There was a problem hiding this comment.
Please add proper documentation here.
|
|
||
| import static io.aiven.connect.jdbc.source.JdbcSourceTaskConfig.TABLES_CONFIG; | ||
|
|
||
| public class StartupMetricUpdater { |
There was a problem hiding this comment.
Please add proper documentation here. What does this class do. Are there any design constraints that future developers should know about, etc.
| properties.put(JdbcSourceTaskConfig.TABLES_CONFIG, ""); | ||
| properties.put(JdbcSourceTaskConfig.QUERY_CONFIG, | ||
| "SELECT * FROM \"" + SINGLE_TABLE_NAME + "\" WHERE \"id\" > 0"); | ||
| properties.put("name", "soneTaskName"); |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces the feature discussed in #179.
It allows enabling a custom metric to determine the number of messages/records that will be published into the Kafka topic.
To reduce database load, the corresponding query is executed at startup only. The metric is published via JMX under
io.aiven.connect.jdbc.initialImportCount<task=\"{connector/task_name}\", topic=\"{topic_name}\", [table=\"{table_name}\",The attribute ‘table’ is optional, depending on the table or query mode.
The feature is disabled by default and can be enabled with the config
JdbcSourceTaskConfig.INITIAL_MESSAGE_COUNT_METRIC_ENABLED_CONFIG