Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][broker][PIP-195] Add topicName and cursorName for ledger metadata of bucket snapshot #19802

Merged
merged 2 commits into from
Mar 15, 2023

Conversation

coderzc
Copy link
Member

@coderzc coderzc commented Mar 14, 2023

PIP: #16763

Motivation & Modifications

add topicName and cursorName for BK metadata of bucket snapshot, so we know who is using the ledger.

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Mar 14, 2023
@coderzc coderzc changed the title [improve][broker][PIP-195] Add topicName and cursorName for storage metadata of bucket snapshot [improve][broker][PIP-195] Add topicName and cursorName for ledger metadata of bucket snapshot Mar 14, 2023
@coderzc coderzc force-pushed the improve/bucket_delayed_storage branch from a47a28a to d4611d1 Compare March 14, 2023 04:58
@coderzc coderzc self-assigned this Mar 14, 2023
@coderzc coderzc added this to the 3.0.0 milestone Mar 14, 2023
@coderzc coderzc added area/broker type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages labels Mar 14, 2023
Copy link
Contributor

@codelipenghui codelipenghui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

I think it should be one that we missed in the proposal
Could you please also update the proposal according
to the changes of the Ledger's metadata? It can help
users to troubleshoot issues like why the ledger has not
been deleted.

@codelipenghui
Copy link
Contributor

/pulsarbot run-failure-checks

@coderzc
Copy link
Member Author

coderzc commented Mar 14, 2023

Good catch!

I think it should be one that we missed in the proposal Could you please also update the proposal according to the changes of the Ledger's metadata? It can help users to troubleshoot issues like why the ledger has not been deleted.

Yes, I will update this part of the proposal

@@ -48,7 +48,9 @@ public final class LedgerMetadataUtils {
private static final String METADATA_PROPERTY_COMPACTEDTO = "pulsar/compactedTo";
private static final String METADATA_PROPERTY_SCHEMAID = "pulsar/schemaId";

private static final String METADATA_PROPERTY_DELAYED_INDEX_BUCKETID = "pulsar/delayedIndexBucketId";
private static final String METADATA_PROPERTY_DELAYED_INDEX_BUCKET_KEY = "pulsar/delayedIndexBucketKey";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this change affect getting older values?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we don't read metadata directly, it only used to troubleshoot issues.

@coderzc coderzc merged commit 160a864 into apache:master Mar 15, 2023
coderzc added a commit to coderzc/pulsar that referenced this pull request Mar 17, 2023
coderzc added a commit to coderzc/pulsar that referenced this pull request Mar 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/broker doc-not-needed Your PR changes do not impact docs type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants