Skip to content

Commit

Permalink
add iceberg and tomstone properties
Browse files Browse the repository at this point in the history
  • Loading branch information
Deflaimun committed Nov 11, 2024
1 parent 488c9f6 commit b379b52
Showing 1 changed file with 152 additions and 0 deletions.
152 changes: 152 additions & 0 deletions modules/reference/pages/properties/cluster-properties.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -1504,6 +1504,20 @@ Maximum age of the metadata cached in the health monitor of a non-controller bro

---

=== iceberg_catalog_base_location

Base path for the cloud object storage-backed Iceberg catalog. After Iceberg is enabled, do not change this value.

*Requires restart:* Yes

*Visibility:* `user`

*Type:* string

*Default:* `redpanda-iceberg-catalog`

---

=== http_authentication

A list of supported HTTP authentication mechanisms.
Expand All @@ -1520,6 +1534,36 @@ A list of supported HTTP authentication mechanisms.

---

=== iceberg_catalog_commit_interval_ms

The frequency at which the Iceberg coordinator commits topic files to the catalog. This is the interval between commit transactions across all topics monitored by the coordinator, not the interval between individual commits.

*Unit:* milliseconds

*Requires restart:* No

*Visibility:* `tunable`

*Type:* integer

*Accepted values:* [`-17592186044416`, `17592186044415`]

*Default:* `10000`

=== iceberg_catalog_type

Iceberg catalog type that Redpanda will use to commit table metadata updates.

*Accepted Values:* `rest`, `filesystem`

*Requires restart:* Yes

*Visibility:* `user`

*Default:* `filesystem`

---

=== iceberg_enabled

Enables the translation of topic data into Iceberg tables. Setting `iceberg_enabled` to `true` activates the feature at the cluster level, but each topic must also set the `redpanda.iceberg.enabled` topic-level property to `true` to use it. If `iceberg_enabled` is set to `false`, then the feature is disabled for all topics in the cluster, overriding any topic-level settings.
Expand All @@ -1534,6 +1578,96 @@ Enables the translation of topic data into Iceberg tables. Setting `iceberg_enab

---

=== iceberg_rest_catalog_endpoint

URL of Iceberg REST catalog endpoint.

*Requires restart:* Yes

*Visibility:* `user`

*Type:* string

*Default:* `null`

---

=== iceberg_rest_catalog_request_timeout_ms

Maximum length of time that Redpanda waits for a response from the REST catalog before aborting the request.

*Unit:* milliseconds

*Requires restart:* No

*Visibility:* `tunable`

*Type:* integer

*Accepted values:* [`-17592186044416`, `17592186044415`]

*Default:* `10000`

---

=== iceberg_rest_catalog_secret

Secret to authenticate against Iceberg REST catalog. Required if catalog type is set to `rest`.

*Requires restart:* Yes

*Visibility:* `user`

*Type:* string

*Default:* `null`

---

=== iceberg_rest_catalog_token

Token used to access the REST Iceberg catalog. If the token is present, Redpanda ignores credentials stored in the properties iceberg_rest_catalog_user_id and iceberg_rest_catalog_secret.

*Requires restart:* Yes

*Visibility:* `user`

*Type:* string

*Default:* `null`

---

=== iceberg_rest_catalog_user_id

Iceberg REST catalog user ID. This ID is used to query the catalog API for the OAuth token. Required if catalog type is set to `rest`.

*Requires restart:* Yes

*Visibility:* `user`

*Type:* string

*Default:* `null`

---

=== iceberg_translation_interval_ms_default

How often an Iceberg enabled topic is checked for new data to translate. You can override this value at topic level using redpanda.iceberg.translation.interval.ms.

*Requires restart:* No

*Visibility:* `tunable`

*Type:* integer

*Accepted values:* [`-17592186044416`, `17592186044415`]

*Default:* `std::chrono::milliseconds(1min)`

---

=== id_allocator_batch_size

The ID allocator allocates messages in batches (each batch is a one log record) and then serves requests from memory without touching the log until the batch is exhausted.
Expand Down Expand Up @@ -4856,6 +4990,24 @@ Transaction manager's synchronization timeout. Maximum time to wait for internal

---

=== tombstone_retention_ms

The retention time for tombstone records in a compacted topic. Cannot be enabled at the same time as any of `cloud_storage_enabled`, `cloud_storage_enable_remote_read`, or `cloud_storage_enable_remote_write`.

*Unit:* milliseconds

*Requires restart:* No

*Visibility:* `user`

*Type:* integer

*Accepted values:* [`-17592186044416`, `17592186044415`]

*Default:* `null`

---

=== topic_fds_per_partition

Required file handles per partition when creating topics.
Expand Down

0 comments on commit b379b52

Please sign in to comment.