Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removed unnessary checks in S3 sink #341

Merged

Conversation

Claudenw
Copy link
Contributor

cleaned up code and added javadocs.

From a Slack conversation.

Q: I was looking at S3SinkConfig and came across this.

 @Override
    public List<OutputField> getOutputFields() {
        if (Objects.nonNull(getList(FORMAT_OUTPUT_FIELDS_CONFIG))
                && !get(FORMAT_OUTPUT_FIELDS_CONFIG).equals(ConfigDef.NO_DEFAULT_VALUE)) {
            return getOutputFields(FORMAT_OUTPUT_FIELDS_CONFIG);
        }
        if (Objects.nonNull(getList(OUTPUT_FIELDS)) && !get(OUTPUT_FIELDS).equals(ConfigDef.NO_DEFAULT_VALUE)) {
            return getOutputFields(OUTPUT_FIELDS);
        }
        return List.of(new OutputField(OutputFieldType.VALUE, OutputFieldEncodingType.BASE64));
    }

Now FORMAT_OUTPUT_FIELDS_CONFIG is defined as a List with the default of null.

so checking if Objects.nonNull(getList(FORMAT_OUTPUT_FIELDS_CONFIG) makes sense.

But this part I don't get && !get(FORMAT_OUTPUT_FIELDS_CONFIG).equals(ConfigDef.NO_DEFAULT_VALUE)) If NO_DEFAULT_VALUE is specified as the default when the ConfigDef is created, then when the map of key/value pairs is passed in if there is no value in the map the NO_DEFAULT_VALUE causes a ConfigException to be thrown. So I don't see any way that the value of the property can be a NO_DEFAULT_VALUE , it is either null, or a list.

I missing something?

A: Yeah that stuff doesn't make sense. NO_DEFAULT_VALUE is a sentinel for the validation logic, it's not a real default.

@Claudenw Claudenw requested review from a team as code owners November 12, 2024 09:15
@Claudenw Claudenw self-assigned this Nov 12, 2024
Copy link
Contributor

@aindriu-aiven aindriu-aiven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Claudenw Claudenw requested a review from RyanSkraba November 14, 2024 09:08
@@ -298,13 +298,13 @@ private static void addAwsStsConfigGroup(final ConfigDef configDef) {

configDef.define(AWS_STS_ROLE_SESSION_NAME, ConfigDef.Type.STRING, null, new ConfigDef.NonEmptyString(),
Importance.MEDIUM, "AWS STS Session name", GROUP_AWS_STS, awsStsGroupCounter++, // NOPMD
// UnusedAssignment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This formatting change is a bit irritating, because these // NOPMD comments shouldn't be split across lines in the first place. 🙄

Technically only the last one in this list is actually necessary anyway, and we're ignoring checks that wouldn't fail.

* {@link #OUTPUT_FIELDS}. If neither is set will create an output field of {@link OutputFieldType#VALUE} and
* {@link OutputFieldEncodingType#BASE64}.
*
* @return The list of output fields. WIll not be {@code null}.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* @return The list of output fields. WIll not be {@code null}.
* @return The list of output fields. Will not be {@code null}.

Copy link
Contributor

@RyanSkraba RyanSkraba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM even as it is -- those little nitpicks are extremely minor, do you want to fix them here?

@aindriu-aiven aindriu-aiven merged commit 9070bca into Aiven-Open:main Nov 20, 2024
8 checks passed
@aindriu-aiven
Copy link
Contributor

@RyanSkraba merged as these were nits and I want to get the number of open PRs down.

@Claudenw Claudenw deleted the KCON-54_Unnecessary_checks_in_sink branch February 19, 2025 09:41
aindriu-aiven pushed a commit that referenced this pull request Feb 20, 2025
Proposed changelog:
##   What's changed:

### Common changes
* Various dependency upgrades
* All tests migrated AssertJ assertions
* Update README.md with information about semantic versioning. by
@Claudenw in
#320
* Update to CONTRIBUTING by @aindriu-aiven in
#352
* Common source task assignment by @aindriu-aiven in
#345
* Common config with fragments by @aindriu-aiven in
#344
* Utility to extract variables from a rendered template by @RyanSkraba
in
#353
* Don't upload zip file from tar location by @afreysenacor in
#371
* ensure timestamp is stable for a record by @mkeskells in
#321
* Clarify javadocs for OjectDistributionStrategy by @RyanSkraba in
#355

### Aiven's S3 sink connector for Apache Kafka
* Removed unnessary checks in S3 sink by @Claudenw in
#341
* Adding information for handling S3 Sink multipart upload aborted parts
by @aindriu-aiven in
#385

### Aiven's GCS sink connector for Apache Kafka
* Fixed GcsSinkTask logger by @AnatolyPopov in
#348
* gcs: Migration to java.time.Duration by @AnatolyPopov in
#347

### Aiven's Azure Blob Storage sink connector for Apache Kafka
* chore: fix Azure Blob sink connector artifacts upload by @AnatolyPopov
in
#310
* Implement KIP-898: Add Service Loader to Azure Blob Sink Connector by
@aindriu-aiven in
#377

### Aiven's S3 source connector for Apache Kafka
* This release introduces new [**Aiven's S3 source connector for Apache
Kafka**](https://github.com/Aiven-Open/cloud-storage-connectors-for-apache-kafka/tree/main/s3-source-connector#readme)

## New Contributors
* @mkeskells made their first contribution in
#322
* @afreysenacor made their first contribution in
#371

**Full Changelog**:
v3.1.0...v3.2.0

---------

Co-authored-by: GitHub Action <[email protected]>
muralibasani added a commit that referenced this pull request Feb 20, 2025
Proposed changelog:
##   What's changed:

### Common changes
* Various dependency upgrades
* All tests migrated AssertJ assertions
* Update README.md with information about semantic versioning. by
@Claudenw in
#320
* Update to CONTRIBUTING by @aindriu-aiven in
#352
* Common source task assignment by @aindriu-aiven in
#345
* Common config with fragments by @aindriu-aiven in
#344
* Utility to extract variables from a rendered template by @RyanSkraba
in
#353
* Don't upload zip file from tar location by @afreysenacor in
#371
* ensure timestamp is stable for a record by @mkeskells in
#321
* Clarify javadocs for OjectDistributionStrategy by @RyanSkraba in
#355

### Aiven's S3 sink connector for Apache Kafka
* Removed unnessary checks in S3 sink by @Claudenw in
#341
* Adding information for handling S3 Sink multipart upload aborted parts
by @aindriu-aiven in
#385

### Aiven's GCS sink connector for Apache Kafka
* Fixed GcsSinkTask logger by @AnatolyPopov in
#348
* gcs: Migration to java.time.Duration by @AnatolyPopov in
#347

### Aiven's Azure Blob Storage sink connector for Apache Kafka
* chore: fix Azure Blob sink connector artifacts upload by @AnatolyPopov
in
#310
* Implement KIP-898: Add Service Loader to Azure Blob Sink Connector by
@aindriu-aiven in
#377

### Aiven's S3 source connector for Apache Kafka
* This release introduces new [**Aiven's S3 source connector for Apache
Kafka**](https://github.com/Aiven-Open/cloud-storage-connectors-for-apache-kafka/tree/main/s3-source-connector#readme)

## New Contributors
* @mkeskells made their first contribution in
#322
* @afreysenacor made their first contribution in
#371

**Full Changelog**:
v3.1.0...v3.2.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants