Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug][Connector-v2][PulsarSource]Fix pulsar option topic-pattern bug. #3989

Merged
merged 3 commits into from
Feb 4, 2023
Merged

[Bug][Connector-v2][PulsarSource]Fix pulsar option topic-pattern bug. #3989

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
73264a8
Fix pulsar option topic-pattern bug.
zhaoliang01 Jan 23, 2023
f72d2d6
delete redundancy judgment.
zhaoliang01 Feb 3, 2023
deb6e80
add topic config judge.
zhaoliang01 Feb 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions ...r/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/PulsarSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,12 +213,11 @@ private void setPartitionDiscoverer(Config config) {
if (StringUtils.isNotBlank(topic)) {
this.partitionDiscoverer = new TopicListDiscoverer(Arrays.asList(StringUtils.split(topic, ",")));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

topic should use config.hasPath(TOPIC_PATTERN.key() judge whether there is

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

}
String topicPattern = config.getString(TOPIC_PATTERN.key());
if (StringUtils.isNotBlank(topicPattern)) {
if (this.partitionDiscoverer != null) {
throw new PulsarConnectorException(SeaTunnelAPIErrorCode.OPTION_VALIDATION_FAILED, String.format("The properties '%s' and '%s' is exclusive.", TOPIC.key(), TOPIC_PATTERN.key()));
if (config.hasPath(TOPIC_PATTERN.key())) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If TOPIC_PATTERN and TOPIC are exclusive, you need add exclusive(TOPIC_PATTERN, TOPIC) in OptionRule. And you can reference https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/sink/KafkaSinkFactory.java

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exclusive configuration of these two parameters has been added.
image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

back to this question. #3989 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, redundancy judgment deleted.

String topicPattern = config.getString(TOPIC_PATTERN.key());
if (StringUtils.isNotBlank(topicPattern)) {
this.partitionDiscoverer = new TopicPatternDiscoverer(Pattern.compile(topicPattern));
}
this.partitionDiscoverer = new TopicPatternDiscoverer(Pattern.compile(topicPattern));
}
if (this.partitionDiscoverer == null) {
throw new PulsarConnectorException(SeaTunnelAPIErrorCode.OPTION_VALIDATION_FAILED, String.format("The properties '%s' or '%s' is required.", TOPIC.key(), TOPIC_PATTERN.key()));
Expand Down