diff --git a/docs/src/main/sphinx/admin/event-listeners-http.md b/docs/src/main/sphinx/admin/event-listeners-http.md index 0707713ceff..6eeea6d5185 100644 --- a/docs/src/main/sphinx/admin/event-listeners-http.md +++ b/docs/src/main/sphinx/admin/event-listeners-http.md @@ -14,6 +14,7 @@ and metadata about the query processing. Running the capture system separate from Trino reduces the performance impact and avoids downtime for non-client-facing changes. +(http-event-listener-requirements)= ## Requirements You need to perform the following steps: @@ -46,62 +47,61 @@ event-listener.config-files=etc/http-event-listener.properties,... ### Configuration properties -```{eval-rst} -.. list-table:: - :widths: 40, 40, 20 - :header-rows: 1 - - * - Property name - - Description - - Default - - * - http-event-listener.log-created - - Enable the plugin to log ``QueryCreatedEvent`` events - - ``false`` - - * - http-event-listener.log-completed - - Enable the plugin to log ``QueryCompletedEvent`` events - - ``false`` - - * - http-event-listener.log-split - - Enable the plugin to log ``SplitCompletedEvent`` events - - ``false`` - - * - http-event-listener.connect-ingest-uri - - The URI that the plugin will POST events to - - None. See the `requirements <#requirements>`_ section. - - * - http-event-listener.connect-http-headers - - List of custom HTTP headers to be sent along with the events. See - :ref:`http-event-listener-custom-headers` for more details - - Empty - - * - http-event-listener.connect-retry-count - - The number of retries on server error. A server is considered to be - in an error state when the response code is 500 or higher - - ``0`` - - * - http-event-listener.connect-retry-delay - - Duration for which to delay between attempts to send a request - - ``1s`` - - * - http-event-listener.connect-backoff-base - - The base used for exponential backoff when retrying on server error. - The formula used to calculate the delay is - :math:`attemptDelay = retryDelay * backoffBase^{attemptCount}`. - Attempt count starts from 0. Leave this empty or set to 1 to disable - exponential backoff and keep constant delays - - ``2`` - - * - http-event-listener.connect-max-delay - - The upper bound of a delay between 2 retries. This should be - used with exponential backoff. - - ``1m`` - - * - http-event-listener.* - - Pass configuration onto the HTTP client - - -``` +:::{list-table} +:widths: 40, 40, 20 +:header-rows: 1 + +* - Property name + - Description + - Default + +* - http-event-listener.log-created + - Enable the plugin to log `QueryCreatedEvent` events + - `false` + +* - http-event-listener.log-completed + - Enable the plugin to log `QueryCompletedEvent` events + - `false` + +* - http-event-listener.log-split + - Enable the plugin to log `SplitCompletedEvent` events + - `false` + +* - http-event-listener.connect-ingest-uri + - The URI that the plugin will POST events to + - None. See the [requirements](http-event-listener-requirements) section. + +* - http-event-listener.connect-http-headers + - List of custom HTTP headers to be sent along with the events. See + [](http-event-listener-custom-headers) for more details + - Empty + +* - http-event-listener.connect-retry-count + - The number of retries on server error. A server is considered to be + in an error state when the response code is 500 or higher + - `0` + +* - http-event-listener.connect-retry-delay + - Duration for which to delay between attempts to send a request + - `1s` + +* - http-event-listener.connect-backoff-base + - The base used for exponential backoff when retrying on server error. + The formula used to calculate the delay is + `attemptDelay = retryDelay * backoffBase^{attemptCount}`. + Attempt count starts from 0. Leave this empty or set to 1 to disable + exponential backoff and keep constant delays + - `2` + +* - http-event-listener.connect-max-delay + - The upper bound of a delay between 2 retries. This should be + used with exponential backoff. + - `1m` + +* - http-event-listener.* + - Pass configuration onto the HTTP client + - +::: (http-event-listener-custom-headers)= diff --git a/docs/src/main/sphinx/admin/fault-tolerant-execution.md b/docs/src/main/sphinx/admin/fault-tolerant-execution.md index 3361f27b676..54185a71f3b 100644 --- a/docs/src/main/sphinx/admin/fault-tolerant-execution.md +++ b/docs/src/main/sphinx/admin/fault-tolerant-execution.md @@ -55,33 +55,32 @@ connector. The following connectors support fault-tolerant execution: The following configuration properties control the behavior of fault-tolerant execution on a Trino cluster: -```{eval-rst} -.. list-table:: Fault-tolerant execution configuration properties - :widths: 30, 50, 20 - :header-rows: 1 - - * - Property name - - Description - - Default value - * - ``retry-policy`` - - Configures what is retried in the event of failure, either - ``QUERY`` to retry the whole query, or ``TASK`` to retry tasks - individually if they fail. See :ref:`retry policy ` for - more information. - - ``NONE`` - * - ``exchange.deduplication-buffer-size`` - - :ref:`Data size ` of the coordinator's in-memory - buffer used by fault-tolerant execution to store output of query - :ref:`stages `. If this buffer is filled during - query execution, the query fails with a "Task descriptor storage capacity - has been exceeded" error message unless an :ref:`exchange manager - ` is configured. - - ``32MB`` - * - ``exchange.compression-enabled`` - - Enable compression of spooling data. Setting to ``true`` is recommended - when using an :ref:`exchange manager `. - - ``false`` -``` + +:::{list-table} Fault-tolerant execution configuration properties +:widths: 30, 50, 20 +:header-rows: 1 + +* - Property name + - Description + - Default value +* - `retry-policy` + - Configures what is retried in the event of failure, either `QUERY` to retry + the whole query, or `TASK` to retry tasks individually if they fail. See + [retry policy](fte-retry-policy) for more information. + - `NONE` +* - `exchange.deduplication-buffer-size` + - [Data size](prop-type-data-size) of the coordinator's in-memory buffer used + by fault-tolerant execution to store output of query + [stages](trino-concept-stage). If this buffer is filled during query + execution, the query fails with a "Task descriptor storage capacity has been + exceeded" error message unless an [exchange manager](fte-exchange-manager) + is configured. + - `32MB` +* - `exchange.compression-enabled` + - Enable compression of spooling data. Setting to `true` is recommended + when using an [exchange manager](fte-exchange-manager). + - ``false`` +::: (fte-retry-policy)= @@ -158,46 +157,44 @@ troubleshooting purposes. The following configuration properties control the thresholds at which queries/tasks are no longer retried in the event of repeated failures: -```{eval-rst} -.. list-table:: Fault tolerance retry limit configuration properties - :widths: 30, 50, 20, 30 - :header-rows: 1 - - * - Property name - - Description - - Default value - - Retry policy - * - ``query-retry-attempts`` - - Maximum number of times Trino may attempt to retry a query before - declaring the query as failed. - - ``4`` - - Only ``QUERY`` - * - ``task-retry-attempts-per-task`` - - Maximum number of times Trino may attempt to retry a single task before - declaring the query as failed. - - ``4`` - - Only ``TASK`` - * - ``retry-initial-delay`` - - Minimum :ref:`time ` that a failed query or task must - wait before it is retried. May be overridden with the - ``retry_initial_delay`` :ref:`session property - `. - - ``10s`` - - ``QUERY`` and ``TASK`` - * - ``retry-max-delay`` - - Maximum :ref:`time ` that a failed query or task must - wait before it is retried. Wait time is increased on each subsequent - failure. May be overridden with the ``retry_max_delay`` :ref:`session - property `. - - ``1m`` - - ``QUERY`` and ``TASK`` - * - ``retry-delay-scale-factor`` - - Factor by which retry delay is increased on each query or task failure. - May be overridden with the ``retry_delay_scale_factor`` :ref:`session - property `. - - ``2.0`` - - ``QUERY`` and ``TASK`` -``` +:::{list-table} Fault tolerance retry limit configuration properties +:widths: 30, 50, 20, 30 +:header-rows: 1 + +* - Property name + - Description + - Default value + - Retry policy +* - `query-retry-attempts` + - Maximum number of times Trino may attempt to retry a query before declaring + the query as failed. + - `4` + - Only `QUERY` +* - `task-retry-attempts-per-task` + - Maximum number of times Trino may attempt to retry a single task before + declaring the query as failed. + - `4` + - Only `TASK` +* - `retry-initial-delay` + - Minimum [time](prop-type-duration) that a failed query or task must wait + before it is retried. May be overridden with the `retry_initial_delay` + [session property](session-properties-definition). + - `10s` + - `QUERY` and `TASK` +* - `retry-max-delay` + - Maximum :ref:`time ` that a failed query or task must + wait before it is retried. Wait time is increased on each subsequent + failure. May be overridden with the ``retry_max_delay`` [session + property](session-properties-definition). + - `1m` + - `QUERY` and `TASK` +* - `retry-delay-scale-factor` + - Factor by which retry delay is increased on each query or task failure. May + be overridden with the `retry_delay_scale_factor` [session + property](session-properties-definition). + - `2.0` + - `QUERY` and `TASK` +::: ### Task sizing @@ -213,85 +210,82 @@ during fault-tolerant task execution, you can configure the following configuration properties to manually control task sizing. These configuration properties only apply to a `TASK` retry policy. -```{eval-rst} -.. list-table:: Task sizing configuration properties - :widths: 30, 50, 20 - :header-rows: 1 - - * - Property name - - Description - - Default value - * - ``fault-tolerant-execution-standard-split-size`` - - Standard :ref:`split ` :ref:`data size - ` processed by tasks that read data from source - tables. Value is interpreted with split weight taken into account. If the - weight of splits produced by a catalog denotes that they are lighter or - heavier than "standard" split, then the number of splits processed by a - single task is adjusted accordingly. - - May be overridden for the current session with the - ``fault_tolerant_execution_standard_split_size`` - :ref:`session property `. - - ``64MB`` - * - ``fault-tolerant-execution-max-task-split-count`` - - Maximum number of :ref:`splits ` processed by a - single task. This value is not split weight-adjusted and serves as - protection against situations where catalogs report an incorrect split - weight. - - May be overridden for the current session with the - ``fault_tolerant_execution_max_task_split_count`` - :ref:`session property `. - - ``256`` - * - ``fault-tolerant-execution-arbitrary-distribution-compute-task-target-size-growth-period`` - - The number of tasks created for any given non-writer stage of arbitrary - distribution before task size is increased. - - ``64`` - * - ``fault-tolerant-execution-arbitrary-distribution-compute-task-target-size-growth-factor`` - - Growth factor for adaptive sizing of non-writer tasks of arbitrary - distribution for fault-tolerant execution. Lower bound is 1.0. For every - task size increase, new task target size is old task target size - multiplied by this growth factor. - - ``1.26`` - * - ``fault-tolerant-execution-arbitrary-distribution-compute-task-target-size-min`` - - Initial/minimum target input :ref:`data size ` for - non-writer tasks of arbitrary distribution of fault-tolerant execution. - - ``512MB`` - * - ``fault-tolerant-execution-arbitrary-distribution-compute-task-target-size-max`` - - Maximum target input :ref:`data size ` for each - non-writer task of arbitrary distribution of fault-tolerant execution. - - ``50GB`` - * - ``fault-tolerant-execution-arbitrary-distribution-write-task-target-size-growth-period`` - - The number of tasks created for any given writer stage of arbitrary - distribution before task size is increased. - - ``64`` - * - ``fault-tolerant-execution-arbitrary-distribution-write-task-target-size-growth-factor`` - - Growth factor for adaptive sizing of writer tasks of arbitrary - distribution for fault-tolerant execution. Lower bound is 1.0. For every - task size increase, new task target size is old task target size - multiplied by this growth factor. - - ``1.26`` - * - ``fault-tolerant-execution-arbitrary-distribution-write-task-target-size-min`` - - Initial/minimum target input :ref:`data size ` for - writer tasks of arbitrary distribution of fault-tolerant execution. - - ``4GB`` - * - ``fault-tolerant-execution-arbitrary-distribution-write-task-target-size-max`` - - Maximum target input :ref:`data size ` for writer - tasks of arbitrary distribution of fault-tolerant execution. - - ``50GB`` - * - ``fault-tolerant-execution-hash-distribution-compute-task-target-size`` - - Target input :ref:`data size ` for non-writer tasks - of hash distribution of fault-tolerant execution. - - ``512MB`` - * - ``fault-tolerant-execution-hash-distribution-write-task-target-size`` - - Target input :ref:`data size ` of writer tasks of - hash distribution of fault-tolerant execution. - - ``4GB`` - * - ``fault-tolerant-execution-hash-distribution-write-task-target-max-count`` - - Soft upper bound on number of writer tasks in a stage of hash - distribution of fault-tolerant execution. - - ``2000`` -``` +:::{list-table} Task sizing configuration properties +:widths: 30, 50, 20 +:header-rows: 1 + +* - Property name + - Description + - Default value +* - `fault-tolerant-execution-standard-split-size` + - Standard [split](trino-concept-splits) [data size]( prop-type-data-size) + processed by tasks that read data from source tables. Value is interpreted + with split weight taken into account. If the weight of splits produced by a + catalog denotes that they are lighter or heavier than "standard" split, then + the number of splits processed by a single task is adjusted accordingly. + + May be overridden for the current session with the + `fault_tolerant_execution_standard_split_size` [session + property](session-properties-definition). + - `64MB` +* - `fault-tolerant-execution-max-task-split-count` + - Maximum number of [splits](trino-concept-splits) processed by a single task. + This value is not split weight-adjusted and serves as protection against + situations where catalogs report an incorrect split weight. + + May be overridden for the current session with the + `fault_tolerant_execution_max_task_split_count` [session + property](session-properties-definition). + - `256` +* - `fault-tolerant-execution-arbitrary-distribution-compute-task-target-size-growth-period` + - The number of tasks created for any given non-writer stage of arbitrary + distribution before task size is increased. + - `64` +* - `fault-tolerant-execution-arbitrary-distribution-compute-task-target-size-growth-factor` + - Growth factor for adaptive sizing of non-writer tasks of arbitrary + distribution for fault-tolerant execution. Lower bound is 1.0. For every + task size increase, new task target size is old task target size multiplied + by this growth factor. + - `1.26` +* - `fault-tolerant-execution-arbitrary-distribution-compute-task-target-size-min` + - Initial/minimum target input [data size](prop-type-data-size) for non-writer + tasks of arbitrary distribution of fault-tolerant execution. + - `512MB` +* - `fault-tolerant-execution-arbitrary-distribution-compute-task-target-size-max` + - Maximum target input [data size](prop-type-data-size) for each non-writer + task of arbitrary distribution of fault-tolerant execution. + - `50GB` +* - `fault-tolerant-execution-arbitrary-distribution-write-task-target-size-growth-period` + - The number of tasks created for any given writer stage of arbitrary + distribution before task size is increased. + - `64` +* - `fault-tolerant-execution-arbitrary-distribution-write-task-target-size-growth-factor` + - Growth factor for adaptive sizing of writer tasks of arbitrary distribution + for fault-tolerant execution. Lower bound is 1.0. For every task size + increase, new task target size is old task target size multiplied by this + growth factor. + - `1.26` +* - `fault-tolerant-execution-arbitrary-distribution-write-task-target-size-min` + - Initial/minimum target input [data size](prop-type-data-size) for writer + tasks of arbitrary distribution of fault-tolerant execution. + - `4GB` +* - `fault-tolerant-execution-arbitrary-distribution-write-task-target-size-max` + - Maximum target input [data size](prop-type-data-size) for writer tasks of + arbitrary distribution of fault-tolerant execution. + - `50GB` +* - `fault-tolerant-execution-hash-distribution-compute-task-target-size` + - Target input [data size](prop-type-data-size) for non-writer tasks of hash + distribution of fault-tolerant execution. + - `512MB` +* - `fault-tolerant-execution-hash-distribution-write-task-target-size` + - Target input [data size](prop-type-data-size) of writer tasks of hash + distribution of fault-tolerant execution. + - ``4GB`` +* - `fault-tolerant-execution-hash-distribution-write-task-target-max-count` + - Soft upper bound on number of writer tasks in a stage of hash distribution + of fault-tolerant execution. + - `2000` +::: ### Node allocation @@ -304,79 +298,76 @@ The initial task memory-requirements estimation is static and configured with the `fault-tolerant-task-memory` configuration property. This property only applies to a `TASK` retry policy. -```{eval-rst} -.. list-table:: Node allocation configuration properties - :widths: 30, 50, 20 - :header-rows: 1 - - * - Property name - - Description - - Default value - * - ``fault-tolerant-execution-task-memory`` - - Initial task memory :ref:`data size ` estimation - used for bin-packing when allocating nodes for tasks. May be overridden - for the current session with the - ``fault_tolerant_execution_task_memory`` - :ref:`session property `. - - ``5GB`` -``` +:::{list-table} Node allocation configuration properties +:widths: 30, 50, 20 +:header-rows: 1 + +* - Property name + - Description + - Default value +* - `fault-tolerant-execution-task-memory` + - Initial task memory [data size](prop-type-data-size) estimation + used for bin-packing when allocating nodes for tasks. May be overridden + for the current session with the + `fault_tolerant_execution_task_memory` + [session property](session-properties-definition). + - `5GB` +::: ### Other tuning The following additional configuration property can be used to manage fault-tolerant execution: -```{eval-rst} -.. list-table:: Other fault-tolerant execution configuration properties - :widths: 30, 50, 20, 30 - :header-rows: 1 - - * - Property name - - Description - - Default value - - Retry policy - * - ``fault-tolerant-execution-task-descriptor-storage-max-memory`` - - Maximum :ref:`data size ` of memory to be used to - store task descriptors for fault tolerant queries on coordinator. Extra - memory is needed to be able to reschedule tasks in case of a failure. - - (JVM heap size * 0.15) - - Only ``TASK`` - * - ``fault-tolerant-execution-max-partition-count`` - - Maximum number of partitions to use for distributed joins and - aggregations, similar in function to the - ``query.max-hash-partition-count`` :doc:`query management property - `. It is not recommended to increase - this property value above the default of ``50``, which may result in - instability and poor performance. May be overridden for the current - session with the ``fault_tolerant_execution_max_partition_count`` - :ref:`session property `. - - ``50`` - - Only ``TASK`` - * - ``fault-tolerant-execution-min-partition-count`` - - Minimum number of partitions to use for distributed joins and - aggregations, similar in function to the - ``query.min-hash-partition-count`` :doc:`query management property - `. May be overridden for the current - session with the ``fault_tolerant_execution_min_partition_count`` - :ref:`session property `. - - ``4`` - - Only ``TASK`` - * - ``fault-tolerant-execution-min-partition-count-for-write`` - - Minimum number of partitions to use for distributed joins and - aggregations in write queries, similar in function to the - ``query.min-hash-partition-count-for-write`` :doc:`query management - property `. May be overridden for - the current session with the - ``fault_tolerant_execution_min_partition_count_for_write`` - :ref:`session property `. - - ``50`` - - Only ``TASK`` - * - ``max-tasks-waiting-for-node-per-stage`` - - Allow for up to configured number of tasks to wait for node allocation - per stage, before pausing scheduling for other tasks from this stage. - - 5 - - Only ``TASK`` -``` +:::{list-table} Other fault-tolerant execution configuration properties +:widths: 30, 50, 20, 30 +:header-rows: 1 + +* - Property name + - Description + - Default value + - Retry policy +* - `fault-tolerant-execution-task-descriptor-storage-max-memory` + - Maximum [data size](prop-type-data-size) of memory to be used to + store task descriptors for fault tolerant queries on coordinator. Extra + memory is needed to be able to reschedule tasks in case of a failure. + - (JVM heap size * 0.15) + - Only `TASK` +* - `fault-tolerant-execution-max-partition-count` + - Maximum number of partitions to use for distributed joins and aggregations, + similar in function to the ``query.max-hash-partition-count`` [query + management property](/admin/properties-query-management). It is not + recommended to increase this property value above the default of `50`, which + may result in instability and poor performance. May be overridden for the + current session with the `fault_tolerant_execution_max_partition_count` + [session property](session-properties-definition). + - `50` + - Only `TASK` +* - `fault-tolerant-execution-min-partition-count` + - Minimum number of partitions to use for distributed joins and aggregations, + similar in function to the `query.min-hash-partition-count` [query + management property](/admin/properties-query-management). May be overridden + for the current session with the + `fault_tolerant_execution_min_partition_count` [session + property](session-properties-definition). + - `4` + - Only `TASK` +* - `fault-tolerant-execution-min-partition-count-for-write` + - Minimum number of partitions to use for distributed joins and aggregations + in write queries, similar in function to the + `query.min-hash-partition-count-for-write` [query management + property](/admin/properties-query-management). May be overridden for the + current session with the + `fault_tolerant_execution_min_partition_count_for_write` [session + property](session-properties-definition). + - `50` + - Only `TASK` +* - `max-tasks-waiting-for-node-per-stage` + - Allow for up to configured number of tasks to wait for node allocation + per stage, before pausing scheduling for other tasks from this stage. + - 5 + - Only `TASK` +::: (fte-exchange-manager)= @@ -401,118 +392,115 @@ The following table lists the available configuration properties for `exchange-manager.properties`, their default values, and which filesystem(s) the property may be configured for: -```{eval-rst} -.. list-table:: Exchange manager configuration properties - :widths: 30, 50, 20, 30 - :header-rows: 1 - - * - Property name - - Description - - Default value - - Supported filesystem - * - ``exchange.base-directories`` - - Comma-separated list of URI locations that the exchange manager uses to - store spooling data. - - - - Any - * - ``exchange.sink-buffer-pool-min-size`` - - The minimum buffer pool size for an exchange sink. The larger the buffer - pool size, the larger the write parallelism and memory usage. - - ``10`` - - Any - * - ``exchange.sink-buffers-per-partition`` - - The number of buffers per partition in the buffer pool. The larger the - buffer pool size, the larger the write parallelism and memory usage. - - ``2`` - - Any - * - ``exchange.sink-max-file-size`` - - Max :ref:`data size ` of files written by exchange - sinks. - - ``1GB`` - - Any - * - ``exchange.source-concurrent-readers`` - - Number of concurrent readers to read from spooling storage. The - larger the number of concurrent readers, the larger the read parallelism - and memory usage. - - ``4`` - - Any - * - ``exchange.s3.aws-access-key`` - - AWS access key to use. Required for a connection to AWS S3 and GCS, can - be ignored for other S3 storage systems. - - - - AWS S3, GCS - * - ``exchange.s3.aws-secret-key`` - - AWS secret key to use. Required for a connection to AWS S3 and GCS, can - be ignored for other S3 storage systems. - - - - AWS S3, GCS - * - ``exchange.s3.iam-role`` - - IAM role to assume. - - - - AWS S3, GCS - * - ``exchange.s3.external-id`` - - External ID for the IAM role trust policy. - - - - AWS S3, GCS - * - ``exchange.s3.region`` - - Region of the S3 bucket. - - - - AWS S3, GCS - * - ``exchange.s3.endpoint`` - - S3 storage endpoint server if using an S3-compatible storage system that - is not AWS. If using AWS S3, this can be ignored. If using GCS, set it - to ``https://storage.googleapis.com``. - - - - Any S3-compatible storage - * - ``exchange.s3.max-error-retries`` - - Maximum number of times the exchange manager's S3 client should retry - a request. - - ``10`` - - Any S3-compatible storage - * - ``exchange.s3.path-style-access`` - - Enables using `path-style access `_ - for all requests to S3. - - ``false`` - - Any S3-compatible storage - * - ``exchange.s3.upload.part-size`` - - Part :ref:`data size ` for S3 multi-part upload. - - ``5MB`` - - Any S3-compatible storage - * - ``exchange.gcs.json-key-file-path`` - - Path to the JSON file that contains your Google Cloud Platform - service account key. Not to be set together with - ``exchange.gcs.json-key`` - - - - GCS - * - ``exchange.gcs.json-key`` - - Your Google Cloud Platform service account key in JSON format. - Not to be set together with ``exchange.gcs.json-key-file-path`` - - - - GCS - * - ``exchange.azure.connection-string`` - - Connection string used to access the spooling container. - - - - Azure Blob Storage - * - ``exchange.azure.block-size`` - - Block :ref:`data size ` for Azure block blob - parallel upload. - - ``4MB`` - - Azure Blob Storage - * - ``exchange.azure.max-error-retries`` - - Maximum number of times the exchange manager's Azure client should - retry a request. - - ``10`` - - Azure Blob Storage - * - ``exchange.hdfs.block-size`` - - Block :ref:`data size ` for HDFS storage. - - ``4MB`` - - HDFS - * - ``hdfs.config.resources`` - - Comma-separated list of paths to HDFS configuration files, for example ``/etc/hdfs-site.xml``. - The files must exist on all nodes in the Trino cluster. - - - - HDFS -``` +:::{list-table} Exchange manager configuration properties +:widths: 30, 50, 20, 30 +:header-rows: 1 + +* - Property name + - Description + - Default value + - Supported filesystem +* - `exchange.base-directories` + - Comma-separated list of URI locations that the exchange manager uses to + store spooling data. + - + - Any +* - `exchange.sink-buffer-pool-min-size` + - The minimum buffer pool size for an exchange sink. The larger the buffer + pool size, the larger the write parallelism and memory usage. + - `10` + - Any +* - `exchange.sink-buffers-per-partition` + - The number of buffers per partition in the buffer pool. The larger the + buffer pool size, the larger the write parallelism and memory usage. + - `2` + - Any +* - `exchange.sink-max-file-size` + - Max [data size](prop-type-data-size) of files written by exchange sinks. + - ``1GB`` + - Any +* - `exchange.source-concurrent-readers` + - Number of concurrent readers to read from spooling storage. The larger the + number of concurrent readers, the larger the read parallelism and memory + usage. + - `4` + - Any +* - `exchange.s3.aws-access-key` + - AWS access key to use. Required for a connection to AWS S3 and GCS, can be + ignored for other S3 storage systems. + - + - AWS S3, GCS +* - `exchange.s3.aws-secret-key` + - AWS secret key to use. Required for a connection to AWS S3 and GCS, can be + ignored for other S3 storage systems. + - + - AWS S3, GCS +* - `exchange.s3.iam-role` + - IAM role to assume. + - + - AWS S3, GCS +* - `exchange.s3.external-id` + - External ID for the IAM role trust policy. + - + - AWS S3, GCS +* - `exchange.s3.region` + - Region of the S3 bucket. + - + - AWS S3, GCS +* - `exchange.s3.endpoint` + - S3 storage endpoint server if using an S3-compatible storage system that + is not AWS. If using AWS S3, this can be ignored. If using GCS, set it + to `https://storage.googleapis.com`. + - + - Any S3-compatible storage +* - `exchange.s3.max-error-retries` + - Maximum number of times the exchange manager's S3 client should retry + a request. + - `10` + - Any S3-compatible storage +* - `exchange.s3.path-style-access` + - Enables using [path-style access](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access) + for all requests to S3. + - `false` + - Any S3-compatible storage +* - `exchange.s3.upload.part-size` + - Part [data size](prop-type-data-size) for S3 multi-part upload. + - `5MB` + - Any S3-compatible storage +* - `exchange.gcs.json-key-file-path` + - Path to the JSON file that contains your Google Cloud Platform service + account key. Not to be set together with `exchange.gcs.json-key` + - + - GCS +* - `exchange.gcs.json-key` + - Your Google Cloud Platform service account key in JSON format. Not to be set + together with `exchange.gcs.json-key-file-path` + - + - GCS +* - `exchange.azure.connection-string` + - Connection string used to access the spooling container. + - + - Azure Blob Storage +* - `exchange.azure.block-size` + - Block [data size](prop-type-data-size) for Azure block blob parallel upload. + - `4MB` + - Azure Blob Storage +* - `exchange.azure.max-error-retries` + - Maximum number of times the exchange manager's Azure client should + retry a request. + - `10` + - Azure Blob Storage +* - `exchange.hdfs.block-size` + - Block [data size](prop-type-data-size) for HDFS storage. + - `4MB` + - HDFS +* - `hdfs.config.resources` + - Comma-separated list of paths to HDFS configuration files, for example + `/etc/hdfs-site.xml`. The files must exist on all nodes in the Trino + cluster. + - + - HDFS +::: It is recommended to set the `exchange.compression-enabled` property to `true` in the cluster's `config.properties` file, to reduce the exchange diff --git a/docs/src/main/sphinx/admin/resource-groups.md b/docs/src/main/sphinx/admin/resource-groups.md index ff6e1e497d0..241b5b552c1 100644 --- a/docs/src/main/sphinx/admin/resource-groups.md +++ b/docs/src/main/sphinx/admin/resource-groups.md @@ -56,39 +56,37 @@ Trino clusters to be stored in the same database if required. The configuration is reloaded from the database every second, and the changes are reflected automatically for incoming queries. -```{eval-rst} -.. list-table:: Database resource group manager properties - :widths: 40, 50, 10 - :header-rows: 1 - - * - Property name - - Description - - Default value - * - ``resource-groups.config-db-url`` - - Database URL to load configuration from. - - ``none`` - * - ``resource-groups.config-db-user`` - - Database user to connect with. - - ``none`` - * - ``resource-groups.config-db-password`` - - Password for database user to connect with. - - ``none`` - * - ``resource-groups.max-refresh-interval`` - - The maximum time period for which the cluster will continue to accept - queries after refresh failures, causing configuration to become stale. - - ``1h`` - * - ``resource-groups.refresh-interval`` - - How often the cluster reloads from the database - - ``1s`` - * - ``resource-groups.exact-match-selector-enabled`` - - Setting this flag enables usage of an additional - ``exact_match_source_selectors`` table to configure resource group - selection rules defined exact name based matches for source, environment - and query type. By default, the rules are only loaded from the - ``selectors`` table, with a regex-based filter for ``source``, among - other filters. - - ``false`` -``` +:::{list-table} Database resource group manager properties +:widths: 40, 50, 10 +:header-rows: 1 + +* - Property name + - Description + - Default value +* - `resource-groups.config-db-url` + - Database URL to load configuration from. + - `none` +* - `resource-groups.config-db-user` + - Database user to connect with. + - `none` +* - `resource-groups.config-db-password` + - Password for database user to connect with. + - `none` +* - `resource-groups.max-refresh-interval` + - The maximum time period for which the cluster will continue to accept + queries after refresh failures, causing configuration to become stale. + - `1h` +* - `resource-groups.refresh-interval` + - How often the cluster reloads from the database + - `1s` +* - `resource-groups.exact-match-selector-enabled` + - Setting this flag enables usage of an additional + `exact_match_source_selectors` table to configure resource group selection + rules defined exact name based matches for source, environment and query + type. By default, the rules are only loaded from the `selectors` table, with + a regex-based filter for `source`, among other filters. + - `false` +::: ## Resource group properties diff --git a/docs/src/main/sphinx/client/cli.md b/docs/src/main/sphinx/client/cli.md index dd066f5af81..2d09a0eabaa 100644 --- a/docs/src/main/sphinx/client/cli.md +++ b/docs/src/main/sphinx/client/cli.md @@ -125,84 +125,83 @@ trino:tiny> Many other options are available to further configure the CLI in interactive mode: -```{eval-rst} -.. list-table:: - :widths: 40, 60 - :header-rows: 1 - - * - Option - - Description - * - ``--catalog`` - - Sets the default catalog. You can change the default catalog and schema - with :doc:`/sql/use`. - * - ``--client-info`` - - Adds arbitrary text as extra information about the client. - * - ``--client-request-timeout`` - - Sets the duration for query processing, after which, the client request is - terminated. Defaults to ``2m``. - * - ``--client-tags`` - - Adds extra tags information about the client and the CLI user. Separate - multiple tags with commas. The tags can be used as input for - :doc:`/admin/resource-groups`. - * - ``--debug`` - - Enables display of debug information during CLI usage for - :ref:`cli-troubleshooting`. Displays more information about query - processing statistics. - * - ``--disable-auto-suggestion`` - - Disables autocomplete suggestions. - * - ``--disable-compression`` - - Disables compression of query results. - * - ``--editing-mode`` - - Sets key bindings in the CLI to be compatible with VI or - EMACS editors. Defaults to ``EMACS``. - * - ``--http-proxy`` - - Configures the URL of the HTTP proxy to connect to Trino. - * - ``--history-file`` - - Path to the :ref:`history file `. Defaults to ``~/.trino_history``. - * - ``--network-logging`` - - Configures the level of detail provided for network logging of the CLI. - Defaults to ``NONE``, other options are ``BASIC``, ``HEADERS``, or - ``BODY``. - * - ``--output-format-interactive=`` - - Specify the :ref:`format ` to use - for printing query results. Defaults to ``ALIGNED``. - * - ``--pager=`` - - Path to the pager program used to display the query results. Set to - an empty value to completely disable pagination. Defaults to ``less`` - with a carefully selected set of options. - * - ``--no-progress`` - - Do not show query processing progress. - * - ``--password`` - - Prompts for a password. Use if your Trino server requires password - authentication. You can set the ``TRINO_PASSWORD`` environment variable - with the password value to avoid the prompt. For more information, see :ref:`cli-username-password-auth`. - * - ``--schema`` - - Sets the default schema. You can change the default catalog and schema - with :doc:`/sql/use`. - * - ``--server`` - - The HTTP/HTTPS address and port of the Trino coordinator. The port must be - set to the port the Trino coordinator is listening for connections on. - Trino server location defaults to ``http://localhost:8080``. - Can only be set if URL is not specified. - * - ``--session`` - - Sets one or more :ref:`session properties - `. Property can be used multiple times with - the format ``session_property_name=value``. - * - ``--socks-proxy`` - - Configures the URL of the SOCKS proxy to connect to Trino. - * - ``--source`` - - Specifies the name of the application or source connecting to Trino. - Defaults to ``trino-cli``. The value can be used as input for - :doc:`/admin/resource-groups`. - * - ``--timezone`` - - Sets the time zone for the session using the `time zone name - `_. Defaults - to the timezone set on your workstation. - * - ``--user`` - - Sets the username for :ref:`cli-username-password-auth`. Defaults to your - operating system username. You can override the default username, - if your cluster uses a different username or authentication mechanism. -``` +:::{list-table} +:widths: 40, 60 +:header-rows: 1 + +* - Option + - Description +* - `--catalog` + - Sets the default catalog. You can change the default catalog and schema with + [](/sql/use). +* - `--client-info` + - Adds arbitrary text as extra information about the client. +* - `--client-request-timeout` + - Sets the duration for query processing, after which, the client request is + terminated. Defaults to `2m`. +* - `--client-tags` + - Adds extra tags information about the client and the CLI user. Separate + multiple tags with commas. The tags can be used as input for + [](/admin/resource-groups). +* - `--debug` + - Enables display of debug information during CLI usage for + [](cli-troubleshooting). Displays more information about query + processing statistics. +* - `--disable-auto-suggestion` + - Disables autocomplete suggestions. +* - `--disable-compression` + - Disables compression of query results. +* - `--editing-mode` + - Sets key bindings in the CLI to be compatible with VI or + EMACS editors. Defaults to `EMACS`. +* - `--http-proxy` + - Configures the URL of the HTTP proxy to connect to Trino. +* - `--history-file` + - Path to the [history file](cli-history). Defaults to `~/.trino_history`. +* - `--network-logging` + - Configures the level of detail provided for network logging of the CLI. + Defaults to `NONE`, other options are `BASIC`, `HEADERS`, or `BODY`. +* - `--output-format-interactive=` + - Specify the [format](cli-output-format) to use for printing query results. + Defaults to `ALIGNED`. +* - `--pager=` + - Path to the pager program used to display the query results. Set to an empty + value to completely disable pagination. Defaults to `less` with a carefully + selected set of options. +* - `--no-progress` + - Do not show query processing progress. +* - `--password` + - Prompts for a password. Use if your Trino server requires password + authentication. You can set the `TRINO_PASSWORD` environment variable with + the password value to avoid the prompt. For more information, see + [](cli-username-password-auth). +* - `--schema` + - Sets the default schema. You can change the default catalog and schema + with [](/sql/use). +* - `--server` + - The HTTP/HTTPS address and port of the Trino coordinator. The port must be + set to the port the Trino coordinator is listening for connections on. Trino + server location defaults to `http://localhost:8080`. Can only be set if URL + is not specified. +* - `--session` + - Sets one or more [session properties](session-properties-definition). + Property can be used multiple times with the format + `session_property_name=value`. +* - `--socks-proxy` + - Configures the URL of the SOCKS proxy to connect to Trino. +* - `--source` + - Specifies the name of the application or source connecting to Trino. + Defaults to `trino-cli`. The value can be used as input for + [](/admin/resource-groups). +* - `--timezone` + - Sets the time zone for the session using the [time zone name]( + ). Defaults to + the timezone set on your workstation. +* - `--user` + - Sets the username for [](cli-username-password-auth). Defaults to your + operating system username. You can override the default username, if your + cluster uses a different username or authentication mechanism. +::: Most of the options can also be set as parameters in the URL. This means a JDBC URL can be used in the CLI after removing the `jdbc:` prefix. @@ -236,41 +235,40 @@ recognizes these certificates. Use the options from the following table to further configure TLS and certificate usage: -```{eval-rst} -.. list-table:: - :widths: 40, 60 - :header-rows: 1 - - * - Option - - Description - * - ``--insecure`` - - Skip certificate validation when connecting with TLS/HTTPS (should only be - used for debugging). - * - ``--keystore-path`` - - The location of the Java Keystore file that contains the certificate of - the server to connect with TLS. - * - ``--keystore-password`` - - The password for the keystore. This must match the password you specified - when creating the keystore. - * - ``--keystore-type`` - - Determined by the keystore file format. The default keystore type is JKS. - This advanced option is only necessary if you use a custom Java - Cryptography Architecture (JCA) provider implementation. - * - ``--truststore-password`` - - The password for the truststore. This must match the password you - specified when creating the truststore. - * - ``--truststore-path`` - - The location of the Java truststore file that will be used to secure TLS. - * - ``--truststore-type`` - - Determined by the truststore file format. The default keystore type is - JKS. This advanced option is only necessary if you use a custom Java - Cryptography Architecture (JCA) provider implementation. - * - ``--use-system-truststore`` - - Verify the server certificate using the system truststore of the - operating system. Windows and macOS are supported. For other operating - systems, the default Java truststore is used. The truststore type can - be overridden using ``--truststore-type``. -``` +:::{list-table} +:widths: 40, 60 +:header-rows: 1 + +* - Option + - Description +* - `--insecure` + - Skip certificate validation when connecting with TLS/HTTPS (should only be + used for debugging). +* - `--keystore-path` + - The location of the Java Keystore file that contains the certificate of the + server to connect with TLS. +* - `--keystore-password` + - The password for the keystore. This must match the password you specified + when creating the keystore. +* - `--keystore-type` + - Determined by the keystore file format. The default keystore type is JKS. + This advanced option is only necessary if you use a custom Java Cryptography + Architecture (JCA) provider implementation. +* - `--truststore-password` + - The password for the truststore. This must match the password you specified + when creating the truststore. +* - `--truststore-path` + - The location of the Java truststore file that will be used to secure TLS. +* - `--truststore-type` + - Determined by the truststore file format. The default keystore type is JKS. + This advanced option is only necessary if you use a custom Java Cryptography + Architecture (JCA) provider implementation. +* - `--use-system-truststore` + - Verify the server certificate using the system truststore of the operating + system. Windows and macOS are supported. For other operating systems, the + default Java truststore is used. The truststore type can be overridden using + `--truststore-type`. +::: (cli-authentication)= @@ -340,20 +338,19 @@ The detailed behavior is as follows: Use the following CLI arguments to connect to a cluster that uses {doc}`certificate authentication `. -```{eval-rst} -.. list-table:: CLI options for certificate authentication - :widths: 35 65 - :header-rows: 1 - - * - Option - - Description - * - ``--keystore-path=`` - - Absolute or relative path to a :doc:`PEM ` or - :doc:`JKS ` file, which must contain a certificate - that is trusted by the Trino cluster you are connecting to. - * - ``--keystore-password=`` - - Only required if the keystore has a password. -``` +:::{list-table} CLI options for certificate authentication +:widths: 35 65 +:header-rows: 1 + +* - Option + - Description +* - `--keystore-path=` + - Absolute or relative path to a [PEM](/security/inspect-pem) or + [JKS](/security/inspect-jks) file, which must contain a certificate + that is trusted by the Trino cluster you are connecting to. +* - `--keystore-password=` + - Only required if the keystore has a password. +::: The truststore related options are independent of client certificate authentication with the CLI; instead, they control the client's trust of the @@ -394,30 +391,28 @@ through {doc}`TLS and HTTPS `. The following table lists the available options for Kerberos authentication: -```{eval-rst} -.. list-table:: CLI options for Kerberos authentication - :widths: 40, 60 - :header-rows: 1 - - * - Option - - Description - * - ``--krb5-config-path`` - - Path to Kerberos configuration files. - * - ``--krb5-credential-cache-path`` - - Kerberos credential cache path. - * - ``--krb5-disable-remote-service-hostname-canonicalization`` - - Disable service hostname canonicalization using the DNS reverse lookup. - * - ``--krb5-keytab-path`` - - The location of the keytab that can be used to authenticate the principal - specified by ``--krb5-principal``. - * - ``--krb5-principal`` - - The principal to use when authenticating to the coordinator. - * - ``--krb5-remote-service-name`` - - Trino coordinator Kerberos service name. - * - ``--krb5-service-principal-pattern`` - - Remote kerberos service principal pattern. Defaults to - ``${SERVICE}@${HOST}``. -``` +:::{list-table} CLI options for Kerberos authentication +:widths: 40, 60 +:header-rows: 1 + +* - Option + - Description +* - `--krb5-config-path` + - Path to Kerberos configuration files. +* - `--krb5-credential-cache-path` + - Kerberos credential cache path. +* - `--krb5-disable-remote-service-hostname-canonicalization` + - Disable service hostname canonicalization using the DNS reverse lookup. +* - `--krb5-keytab-path` + - The location of the keytab that can be used to authenticate the principal + specified by `--krb5-principal`. +* - `--krb5-principal` + - The principal to use when authenticating to the coordinator. +* - `--krb5-remote-service-name` + - Trino coordinator Kerberos service name. +* - `--krb5-service-principal-pattern` + - Remote kerberos service principal pattern. Defaults to `${SERVICE}@${HOST}`. +::: (cli-kerberos-debug)= @@ -520,27 +515,26 @@ other formats and redirect the output to a file. The following options are available to further configure the CLI in batch mode: -```{eval-rst} -.. list-table:: - :widths: 40, 60 - :header-rows: 1 - - * - Option - - Description - * - ``--execute=`` - - Execute specified statements and exit. - * - ``-f``, ``--file=`` - - Execute statements from file and exit. - * - ``--ignore-errors`` - - Continue processing in batch mode when an error occurs. Default is to - exit immediately. - * - ``--output-format=`` - - Specify the :ref:`format ` to use - for printing query results. Defaults to ``CSV``. - * - ``--progress`` - - Show query progress in batch mode. It does not affect the output, - which, for example can be safely redirected to a file. -``` +:::{list-table} +:widths: 40, 60 +:header-rows: 1 + +* - Option + - Description +* - `--execute=` + - Execute specified statements and exit. +* - `-f`, `--file=` + - Execute statements from file and exit. +* - `--ignore-errors` + - Continue processing in batch mode when an error occurs. Default is to exit + immediately. +* - `--output-format=` + - Specify the [format](cli-output-format) to use for printing query results. + Defaults to `CSV`. +* - `--progress` + - Show query progress in batch mode. It does not affect the output, which, for + example can be safely redirected to a file. +::: ### Examples @@ -615,41 +609,40 @@ The available options shown in the following table must be entered in uppercase. The default value is `ALIGNED` in interactive mode, and `CSV` in non-interactive mode. -```{eval-rst} -.. list-table:: Output format options - :widths: 25, 75 - :header-rows: 1 - - * - Option - - Description - * - ``CSV`` - - Comma-separated values, each value quoted. No header row. - * - ``CSV_HEADER`` - - Comma-separated values, quoted with header row. - * - ``CSV_UNQUOTED`` - - Comma-separated values without quotes. - * - ``CSV_HEADER_UNQUOTED`` - - Comma-separated values with header row but no quotes. - * - ``TSV`` - - Tab-separated values. - * - ``TSV_HEADER`` - - Tab-separated values with header row. - * - ``JSON`` - - Output rows emitted as JSON objects with name-value pairs. - * - ``ALIGNED`` - - Output emitted as an ASCII character table with values. - * - ``VERTICAL`` - - Output emitted as record-oriented top-down lines, one per value. - * - ``AUTO`` - - Same as ``ALIGNED`` if output would fit the current terminal width, - and ``VERTICAL`` otherwise. - * - ``MARKDOWN`` - - Output emitted as a Markdown table. - * - ``NULL`` - - Suppresses normal query results. This can be useful during development - to test a query's shell return code or to see whether it results in - error messages. -``` +:::{list-table} Output format options +:widths: 25, 75 +:header-rows: 1 + +* - Option + - Description +* - `CSV` + - Comma-separated values, each value quoted. No header row. +* - `CSV_HEADER` + - Comma-separated values, quoted with header row. +* - `CSV_UNQUOTED` + - Comma-separated values without quotes. +* - `CSV_HEADER_UNQUOTED` + - Comma-separated values with header row but no quotes. +* - `TSV` + - Tab-separated values. +* - `TSV_HEADER` + - Tab-separated values with header row. +* - `JSON` + - Output rows emitted as JSON objects with name-value pairs. +* - `ALIGNED` + - Output emitted as an ASCII character table with values. +* - `VERTICAL` + - Output emitted as record-oriented top-down lines, one per value. +* - `AUTO` + - Same as `ALIGNED` if output would fit the current terminal width, + and `VERTICAL` otherwise. +* - `MARKDOWN` + - Output emitted as a Markdown table. +* - `NULL` + - Suppresses normal query results. This can be useful during development to + test a query's shell return code or to see whether it results in error + messages. +::: (cli-troubleshooting)= diff --git a/docs/src/main/sphinx/client/jdbc.md b/docs/src/main/sphinx/client/jdbc.md index 76d4710d34a..ab0535ad703 100644 --- a/docs/src/main/sphinx/client/jdbc.md +++ b/docs/src/main/sphinx/client/jdbc.md @@ -117,139 +117,132 @@ may not be specified using both methods. ## Parameter reference -```{eval-rst} -.. list-table:: - :widths: 35, 65 - :header-rows: 1 - - * - Name - - Description - * - ``user`` - - Username to use for authentication and authorization. - * - ``password`` - - Password to use for LDAP authentication. - * - ``sessionUser`` - - Session username override, used for impersonation. - * - ``socksProxy`` - - SOCKS proxy host and port. Example: ``localhost:1080`` - * - ``httpProxy`` - - HTTP proxy host and port. Example: ``localhost:8888`` - * - ``clientInfo`` - - Extra information about the client. - * - ``clientTags`` - - Client tags for selecting resource groups. Example: ``abc,xyz`` - * - ``traceToken`` - - Trace token for correlating requests across systems. - * - ``source`` - - Source name for the Trino query. This parameter should be used in - preference to ``ApplicationName``. Thus, it takes precedence over - ``ApplicationName`` and/or ``applicationNamePrefix``. - * - ``applicationNamePrefix`` - - Prefix to append to any specified ``ApplicationName`` client info - property, which is used to set the source name for the Trino query if the - ``source`` parameter has not been set. If neither this property nor - ``ApplicationName`` or ``source`` are set, the source name for the query - is ``trino-jdbc``. - * - ``accessToken`` - - :doc:`JWT ` access token for token based authentication. - * - ``SSL`` - - Set ``true`` to specify using TLS/HTTPS for connections. - * - ``SSLVerification`` - - The method of TLS verification. There are three modes: ``FULL`` - (default), ``CA`` and ``NONE``. For ``FULL``, the normal TLS verification - is performed. For ``CA``, only the CA is verified but hostname mismatch - is allowed. For ``NONE``, there is no verification. - * - ``SSLKeyStorePath`` - - Use only when connecting to a Trino cluster that has :doc:`certificate - authentication ` enabled. Specifies the path to a - :doc:`PEM ` or :doc:`JKS ` - file, which must contain a certificate that is trusted by the Trino - cluster you connect to. - * - ``SSLKeyStorePassword`` - - The password for the KeyStore, if any. - * - ``SSLKeyStoreType`` - - The type of the KeyStore. The default type is provided by the Java - ``keystore.type`` security property or ``jks`` if none exists. - * - ``SSLTrustStorePath`` - - The location of the Java TrustStore file to use to validate HTTPS server - certificates. - * - ``SSLTrustStorePassword`` - - The password for the TrustStore. - * - ``SSLTrustStoreType`` - - The type of the TrustStore. The default type is provided by the Java - ``keystore.type`` security property or ``jks`` if none exists. - * - ``SSLUseSystemTrustStore`` - - Set ``true`` to automatically use the system TrustStore based on the - operating system. The supported OSes are Windows and macOS. For Windows, - the ``Windows-ROOT`` TrustStore is selected. For macOS, the - ``KeychainStore`` TrustStore is selected. For other OSes, the default - Java TrustStore is loaded. The TrustStore specification can be overridden - using ``SSLTrustStoreType``. - * - ``hostnameInCertificate`` - - Expected hostname in the certificate presented by the Trino server. Only - applicable with full SSL verification enabled. - * - ``KerberosRemoteServiceName`` - - Trino coordinator Kerberos service name. This parameter is required for - Kerberos authentication. - * - ``KerberosPrincipal`` - - The principal to use when authenticating to the Trino coordinator. - * - ``KerberosUseCanonicalHostname`` - - Use the canonical hostname of the Trino coordinator for the Kerberos - service principal by first resolving the hostname to an IP address and - then doing a reverse DNS lookup for that IP address. This is enabled by - default. - * - ``KerberosServicePrincipalPattern`` - - Trino coordinator Kerberos service principal pattern. The default is - ``${SERVICE}@${HOST}``. ``${SERVICE}`` is replaced with the value of - ``KerberosRemoteServiceName`` and ``${HOST}`` is replaced with the - hostname of the coordinator (after canonicalization if enabled). - * - ``KerberosConfigPath`` - - Kerberos configuration file. - * - ``KerberosKeytabPath`` - - Kerberos keytab file. - * - ``KerberosCredentialCachePath`` - - Kerberos credential cache. - * - ``KerberosDelegation`` - - Set to ``true`` to use the token from an existing Kerberos context. This - allows client to use Kerberos authentication without passing the Keytab - or credential cache. Defaults to ``false``. - * - ``extraCredentials`` - - Extra credentials for connecting to external services, specified as a - list of key-value pairs. For example, ``foo:bar;abc:xyz`` creates the - credential named ``abc`` with value ``xyz`` and the credential named - ``foo`` with value ``bar``. - * - ``roles`` - - Authorization roles to use for catalogs, specified as a list of key-value - pairs for the catalog and role. For example, - ``catalog1:roleA;catalog2:roleB`` sets ``roleA`` for ``catalog1`` and - ``roleB`` for ``catalog2``. - * - ``sessionProperties`` - - Session properties to set for the system and for catalogs, specified as a - list of key-value pairs. For example, ``abc:xyz;example.foo:bar`` sets - the system property ``abc`` to the value ``xyz`` and the ``foo`` property - for catalog ``example`` to the value ``bar``. - * - ``externalAuthentication`` - - Set to true if you want to use external authentication via - :doc:`/security/oauth2`. Use a local web browser to authenticate with an - identity provider (IdP) that has been configured for the Trino - coordinator. - * - ``externalAuthenticationTokenCache`` - - Allows the sharing of external authentication tokens between different - connections for the same authenticated user until the cache is - invalidated, such as when a client is restarted or when the classloader - reloads the JDBC driver. This is disabled by default, with a value of - ``NONE``. To enable, set the value to ``MEMORY``. If the JDBC driver is - used in a shared mode by different users, the first registered token is - stored and authenticates all users. - * - ``disableCompression`` - - Whether compression should be enabled. - * - ``assumeLiteralUnderscoreInMetadataCallsForNonConformingClients`` - - When enabled, the name patterns passed to ``DatabaseMetaData`` methods are - treated as underscores. You can use this as a workaround for - applications that do not escape schema or table names when passing them - to ``DatabaseMetaData`` methods as schema or table name patterns. - * - ``timezone`` - - Sets the time zone for the session using the `time zone passed - `_. Defaults - to the timezone of the JVM running the JDBC driver. -``` +:::{list-table} +:widths: 35, 65 +:header-rows: 1 + +* - Name + - Description +* - `user` + - Username to use for authentication and authorization. +* - `password` + - Password to use for LDAP authentication. +* - `sessionUser` + - Session username override, used for impersonation. +* - `socksProxy` + - SOCKS proxy host and port. Example: `localhost:1080` +* - `httpProxy` + - HTTP proxy host and port. Example: `localhost:8888` +* - `clientInfo` + - Extra information about the client. +* - `clientTags` + - Client tags for selecting resource groups. Example: `abc,xyz` +* - `traceToken` + - Trace token for correlating requests across systems. +* - `source` + - Source name for the Trino query. This parameter should be used in preference + to `ApplicationName`. Thus, it takes precedence over `ApplicationName` + and/or `applicationNamePrefix`. +* - `applicationNamePrefix` + - Prefix to append to any specified `ApplicationName` client info property, + which is used to set the source name for the Trino query if the `source` + parameter has not been set. If neither this property nor `ApplicationName` + or `source` are set, the source name for the query is `trino-jdbc`. +* - `accessToken` + - [JWT](/security/jwt) access token for token based authentication. +* - `SSL` + - Set `true` to specify using TLS/HTTPS for connections. +* - `SSLVerification` + - The method of TLS verification. There are three modes: `FULL` + (default), `CA` and `NONE`. For `FULL`, the normal TLS verification + is performed. For `CA`, only the CA is verified but hostname mismatch + is allowed. For `NONE`, there is no verification. +* - `SSLKeyStorePath` + - Use only when connecting to a Trino cluster that has [certificate + authentication](/security/certificate) enabled. Specifies the path to a + [PEM](/security/inspect-pem) or [JKS](/security/inspect-jks) file, which must + contain a certificate that is trusted by the Trino cluster you connect to. +* - `SSLKeyStorePassword` + - The password for the KeyStore, if any. +* - `SSLKeyStoreType` + - The type of the KeyStore. The default type is provided by the Java + `keystore.type` security property or `jks` if none exists. +* - `SSLTrustStorePath` + - The location of the Java TrustStore file to use to validate HTTPS server + certificates. +* - `SSLTrustStorePassword` + - The password for the TrustStore. +* - `SSLTrustStoreType` + - The type of the TrustStore. The default type is provided by the Java + `keystore.type` security property or `jks` if none exists. +* - `SSLUseSystemTrustStore` + - Set `true` to automatically use the system TrustStore based on the operating + system. The supported OSes are Windows and macOS. For Windows, the + `Windows-ROOT` TrustStore is selected. For macOS, the `KeychainStore` + TrustStore is selected. For other OSes, the default Java TrustStore is + loaded. The TrustStore specification can be overridden using + `SSLTrustStoreType`. +* - `hostnameInCertificate` + - Expected hostname in the certificate presented by the Trino server. Only + applicable with full SSL verification enabled. +* - `KerberosRemoteServiceName` + - Trino coordinator Kerberos service name. This parameter is required for + Kerberos authentication. +* - `KerberosPrincipal` + - The principal to use when authenticating to the Trino coordinator. +* - `KerberosUseCanonicalHostname` + - Use the canonical hostname of the Trino coordinator for the Kerberos service + principal by first resolving the hostname to an IP address and then doing a + reverse DNS lookup for that IP address. This is enabled by default. +* - `KerberosServicePrincipalPattern` + - Trino coordinator Kerberos service principal pattern. The default is + `${SERVICE}@${HOST}`. `${SERVICE}` is replaced with the value of + `KerberosRemoteServiceName` and `${HOST}` is replaced with the hostname of + the coordinator (after canonicalization if enabled). +* - `KerberosConfigPath` + - Kerberos configuration file. +* - `KerberosKeytabPath` + - Kerberos keytab file. +* - `KerberosCredentialCachePath` + - Kerberos credential cache. +* - `KerberosDelegation` + - Set to `true` to use the token from an existing Kerberos context. This + allows client to use Kerberos authentication without passing the Keytab or + credential cache. Defaults to `false`. +* - `extraCredentials` + - Extra credentials for connecting to external services, specified as a list + of key-value pairs. For example, `foo:bar;abc:xyz` creates the credential + named `abc` with value `xyz` and the credential named `foo` with value + `bar`. +* - `roles` + - Authorization roles to use for catalogs, specified as a list of key-value + pairs for the catalog and role. For example, `catalog1:roleA;catalog2:roleB` + sets `roleA` for `catalog1` and `roleB` for `catalog2`. +* - `sessionProperties` + - Session properties to set for the system and for catalogs, specified as a + list of key-value pairs. For example, `abc:xyz;example.foo:bar` sets the + system property `abc` to the value `xyz` and the `foo` property for catalog + `example` to the value `bar`. +* - `externalAuthentication` + - Set to true if you want to use external authentication via + [](/security/oauth2). Use a local web browser to authenticate with an + identity provider (IdP) that has been configured for the Trino coordinator. +* - `externalAuthenticationTokenCache` + - Allows the sharing of external authentication tokens between different + connections for the same authenticated user until the cache is invalidated, + such as when a client is restarted or when the classloader reloads the JDBC + driver. This is disabled by default, with a value of `NONE`. To enable, set + the value to `MEMORY`. If the JDBC driver is used in a shared mode by + different users, the first registered token is stored and authenticates all + users. +* - `disableCompression` + - Whether compression should be enabled. +* - `assumeLiteralUnderscoreInMetadataCallsForNonConformingClients` + - When enabled, the name patterns passed to `DatabaseMetaData` methods are + treated as underscores. You can use this as a workaround for applications + that do not escape schema or table names when passing them to + `DatabaseMetaData` methods as schema or table name patterns. ::: +* - `timezone` + - Sets the time zone for the session using the [time zone + passed](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/time/ZoneId.html#of(java.lang.String)). + Defaults to the timezone of the JVM running the JDBC driver. diff --git a/docs/src/main/sphinx/functions/comparison.md b/docs/src/main/sphinx/functions/comparison.md index 47f2585cd20..bb137c267aa 100644 --- a/docs/src/main/sphinx/functions/comparison.md +++ b/docs/src/main/sphinx/functions/comparison.md @@ -4,28 +4,27 @@ ## Comparison operators -```{eval-rst} -.. list-table:: - :widths: 30, 70 - :header-rows: 1 - - * - Operator - - Description - * - ``<`` - - Less than - * - ``>`` - - Greater than - * - ``<=`` - - Less than or equal to - * - ``>=`` - - Greater than or equal to - * - ``=`` - - Equal - * - ``<>`` - - Not equal - * - ``!=`` - - Not equal (non-standard but popular syntax) -``` +:::{list-table} +:widths: 30, 70 +:header-rows: 1 + +* - Operator + - Description +* - `<` + - Less than +* - `>` + - Greater than +* - `<=` + - Less than or equal to +* - `>=` + - Greater than or equal to +* - `=` + - Equal +* - `<>` + - Not equal +* - `!=` + - Not equal (non-standard but popular syntax) +::: (range-operator)= @@ -178,27 +177,26 @@ SELECT 42 >= SOME (SELECT 41 UNION ALL SELECT 42 UNION ALL SELECT 43); -- true Here are the meanings of some quantifier and comparison operator combinations: -```{eval-rst} -.. list-table:: - :widths: 40, 60 - :header-rows: 1 - - * - Expression - - Meaning - * - ``A = ALL (...)`` - - Evaluates to ``true`` when ``A`` is equal to all values. - * - ``A <> ALL (...)`` - - Evaluates to ``true`` when ``A`` doesn't match any value. - * - ``A < ALL (...)`` - - Evaluates to ``true`` when ``A`` is smaller than the smallest value. - * - ``A = ANY (...)`` - - Evaluates to ``true`` when ``A`` is equal to any of the values. This form - is equivalent to ``A IN (...)``. - * - ``A <> ANY (...)`` - - Evaluates to ``true`` when ``A`` doesn't match one or more values. - * - ``A < ANY (...)`` - - Evaluates to ``true`` when ``A`` is smaller than the biggest value. -``` +:::{list-table} +:widths: 40, 60 +:header-rows: 1 + +* - Expression + - Meaning +* - `A = ALL (...)` + - Evaluates to `true` when `A` is equal to all values. +* - `A <> ALL (...)` + - Evaluates to `true` when `A` doesn't match any value. +* - `A < ALL (...)` + - Evaluates to `true` when `A` is smaller than the smallest value. +* - `A = ANY (...)` + - Evaluates to `true` when `A` is equal to any of the values. This form + is equivalent to `A IN (...)`. +* - `A <> ANY (...)` + - Evaluates to `true` when `A` doesn't match one or more values. +* - `A < ANY (...)` + - Evaluates to `true` when `A` is smaller than the biggest value. +::: `ANY` and `SOME` have the same meaning and can be used interchangeably. diff --git a/docs/src/main/sphinx/functions/conversion.md b/docs/src/main/sphinx/functions/conversion.md index e12b1a18e91..7de546ab7ed 100644 --- a/docs/src/main/sphinx/functions/conversion.md +++ b/docs/src/main/sphinx/functions/conversion.md @@ -62,42 +62,41 @@ SELECT format_number(1000000); -- '1M' The `parse_data_size` function supports the following units: -```{eval-rst} -.. list-table:: - :widths: 30, 40, 30 - :header-rows: 1 - - * - Unit - - Description - - Value - * - ``B`` - - Bytes - - 1 - * - ``kB`` - - Kilobytes - - 1024 - * - ``MB`` - - Megabytes - - 1024\ :sup:`2` - * - ``GB`` - - Gigabytes - - 1024\ :sup:`3` - * - ``TB`` - - Terabytes - - 1024\ :sup:`4` - * - ``PB`` - - Petabytes - - 1024\ :sup:`5` - * - ``EB`` - - Exabytes - - 1024\ :sup:`6` - * - ``ZB`` - - Zettabytes - - 1024\ :sup:`7` - * - ``YB`` - - Yottabytes - - 1024\ :sup:`8` -``` +:::{list-table} +:widths: 30, 40, 30 +:header-rows: 1 + +* - Unit + - Description + - Value +* - ``B`` + - Bytes + - 1 +* - ``kB`` + - Kilobytes + - 1024 +* - ``MB`` + - Megabytes + - 1024{sup}`2` +* - ``GB`` + - Gigabytes + - 1024{sup}`3` +* - ``TB`` + - Terabytes + - 1024{sup}`4` +* - ``PB`` + - Petabytes + - 1024{sup}`5` +* - ``EB`` + - Exabytes + - 1024{sup}`6` +* - ``ZB`` + - Zettabytes + - 1024{sup}`7` +* - ``YB`` + - Yottabytes + - 1024{sup}`8` +::: :::{function} parse_data_size(string) -> decimal(38) Parses `string` of format `value unit` into a number, where diff --git a/docs/src/main/sphinx/functions/decimal.md b/docs/src/main/sphinx/functions/decimal.md index f9b6bfb2c05..d6371f30926 100644 --- a/docs/src/main/sphinx/functions/decimal.md +++ b/docs/src/main/sphinx/functions/decimal.md @@ -10,20 +10,19 @@ The precision of a decimal type for a literal will be equal to the number of dig in the literal (including trailing and leading zeros). The scale will be equal to the number of digits in the fractional part (including trailing zeros). -```{eval-rst} -.. list-table:: - :widths: 50, 50 - :header-rows: 1 - - * - Example literal - - Data type - * - ``DECIMAL '0'`` - - ``DECIMAL(1)`` - * - ``DECIMAL '12345'`` - - ``DECIMAL(5)`` - * - ``DECIMAL '0000012345.1234500000'`` - - ``DECIMAL(20, 10)`` -``` +:::{list-table} +:widths: 50, 50 +:header-rows: 1 + +* - Example literal + - Data type +* - `DECIMAL '0'` + - `DECIMAL(1)` +* - `DECIMAL '12345'` + - `DECIMAL(5)` +* - `DECIMAL '0000012345.1234500000'` + - `DECIMAL(20, 10)` +::: ## Binary arithmetic decimal operators @@ -31,41 +30,44 @@ Standard mathematical operators are supported. The table below explains precision and scale calculation rules for result. Assuming `x` is of type `DECIMAL(xp, xs)` and `y` is of type `DECIMAL(yp, ys)`. -```{eval-rst} -.. list-table:: - :widths: 30, 40, 30 - :header-rows: 1 - - * - Operation - - Result type precision - - Result type scale - * - ``x + y`` and ``x - y`` - - .. code-block:: text - - min(38, - 1 + - max(xs, ys) + - max(xp - xs, yp - ys) - ) - - ``max(xs, ys)`` - * - ``x * y`` - - ``min(38, xp + yp)`` - - ``xs + ys`` - * - ``x / y`` - - .. code-block:: text - - min(38, - xp + ys-xs - + max(0, ys-xs) - ) - - ``max(xs, ys)`` - * - ``x % y`` - - .. code-block:: text - - min(xp - xs, yp - ys) + - max(xs, bs) - - ``max(xs, ys)`` -``` +:::{list-table} +:widths: 30, 40, 30 +:header-rows: 1 + +* - Operation + - Result type precision + - Result type scale +* - `x + y` and `x - y` + - + ``` + min(38, + 1 + + max(xs, ys) + + max(xp - xs, yp - ys) + ) + ``` + - `max(xs, ys)` +* - `x * y` + - ``` + min(38, xp + yp) + ``` + - `xs + ys` +* - `x / y` + - + ``` + min(38, + xp + ys-xs + + max(0, ys-xs) + ) + ``` + - `max(xs, ys)` +* - `x % y` + - ``` + min(xp - xs, yp - ys) + + max(xs, bs) + ``` + - `max(xs, ys)` +::: If the mathematical result of the operation is not exactly representable with the precision and scale of the result data type, diff --git a/docs/src/main/sphinx/functions/json.md b/docs/src/main/sphinx/functions/json.md index f05323fc927..249db0af144 100644 --- a/docs/src/main/sphinx/functions/json.md +++ b/docs/src/main/sphinx/functions/json.md @@ -590,37 +590,36 @@ diverge from the expected schema. The following table shows the differences between the two modes. -```{eval-rst} -.. list-table:: - :widths: 40 20 40 - :header-rows: 1 - - * - Condition - - strict mode - - lax mode - * - Performing an operation which requires a non-array on an array, e.g.: - - ``$.key`` requires a JSON object - - ``$.floor()`` requires a numeric value - - ERROR - - The array is automatically unnested, and the operation is performed on - each array element. - * - Performing an operation which requires an array on an non-array, e.g.: - - ``$[0]``, ``$[*]``, ``$.size()`` - - ERROR - - The non-array item is automatically wrapped in a singleton array, and - the operation is performed on the array. - * - A structural error: accessing a non-existent element of an array or a - non-existent member of a JSON object, e.g.: - - ``$[-1]`` (array index out of bounds) - - ``$.key``, where the input JSON object does not have a member ``key`` - - ERROR - - The error is suppressed, and the operation results in an empty sequence. -``` +:::{list-table} +:widths: 40 20 40 +:header-rows: 1 + +* - Condition + - strict mode + - lax mode +* - Performing an operation which requires a non-array on an array, e.g.: + + `$.key` requires a JSON object + + `$.floor()` requires a numeric value + - ERROR + - The array is automatically unnested, and the operation is performed on + each array element. +* - Performing an operation which requires an array on an non-array, e.g.: + + `$[0]`, `$[*]`, `$.size()` + - ERROR + - The non-array item is automatically wrapped in a singleton array, and + the operation is performed on the array. +* - A structural error: accessing a non-existent element of an array or a + non-existent member of a JSON object, e.g.: + + `$[-1]` (array index out of bounds) + + `$.key`, where the input JSON object does not have a member `key` + - ERROR + - The error is suppressed, and the operation results in an empty sequence. +::: #### Examples of the lax mode behavior diff --git a/docs/src/main/sphinx/installation/deployment.md b/docs/src/main/sphinx/installation/deployment.md index 5c6eb3019cb..c82c6b42348 100644 --- a/docs/src/main/sphinx/installation/deployment.md +++ b/docs/src/main/sphinx/installation/deployment.md @@ -302,30 +302,29 @@ The installation provides a `bin/launcher` script, which requires Python in the `PATH`. The script can be used manually or as a daemon startup script. It accepts the following commands: -```{eval-rst} -.. list-table:: ``launcher`` commands - :widths: 15, 85 - :header-rows: 1 - - * - Command - - Action - * - ``run`` - - Starts the server in the foreground and leaves it running. To shut down - the server, use Ctrl+C in this terminal or the ``stop`` command from - another terminal. - * - ``start`` - - Starts the server as a daemon and returns its process ID. - * - ``stop`` - - Shuts down a server started with either ``start`` or ``run``. Sends the - SIGTERM signal. - * - ``restart`` - - Stops then restarts a running server, or starts a stopped server, - assigning a new process ID. - * - ``kill`` - - Shuts down a possibly hung server by sending the SIGKILL signal. - * - ``status`` - - Prints a status line, either *Stopped pid* or *Running as pid*. -``` +:::{list-table} `launcher` commands +:widths: 15, 85 +:header-rows: 1 + +* - Command + - Action +* - `run` + - Starts the server in the foreground and leaves it running. To shut down + the server, use Ctrl+C in this terminal or the `stop` command from + another terminal. +* - `start` + - Starts the server as a daemon and returns its process ID. +* - `stop` + - Shuts down a server started with either `start` or `run`. Sends the + SIGTERM signal. +* - `restart` + - Stops then restarts a running server, or starts a stopped server, + assigning a new process ID. +* - `kill` + - Shuts down a possibly hung server by sending the SIGKILL signal. +* - `status` + - Prints a status line, either *Stopped pid* or *Running as pid*. +::: A number of additional options allow you to specify configuration file and directory locations, as well as Java options. Run the launcher with `--help` diff --git a/docs/src/main/sphinx/installation/rpm.md b/docs/src/main/sphinx/installation/rpm.md index 2bb4916d6b3..ca2ebe31101 100644 --- a/docs/src/main/sphinx/installation/rpm.md +++ b/docs/src/main/sphinx/installation/rpm.md @@ -36,24 +36,23 @@ installation, you can manage the Trino server with the `service` command: service trino [start|stop|restart|status] ``` -```{eval-rst} -.. list-table:: ``service`` commands - :widths: 15, 85 - :header-rows: 1 - - * - Command - - Action - * - ``start`` - - Starts the server as a daemon and returns its process ID. - * - ``stop`` - - Shuts down a server started with either ``start`` or ``run``. Sends the - SIGTERM signal. - * - ``restart`` - - Stops and then starts a running server, or starts a stopped server, - assigning a new process ID. - * - ``status`` - - Prints a status line, either *Stopped pid* or *Running as pid*. -``` +:::{list-table} `service` commands +:widths: 15, 85 +:header-rows: 1 + +* - Command + - Action +* - `start` + - Starts the server as a daemon and returns its process ID. +* - `stop` + - Shuts down a server started with either `start` or `run`. Sends the + SIGTERM signal. +* - `restart` + - Stops and then starts a running server, or starts a stopped server, + assigning a new process ID. +* - `status` + - Prints a status line, either *Stopped pid* or *Running as pid*. +::: ## Installation directory structure diff --git a/docs/src/main/sphinx/security/built-in-system-access-control.md b/docs/src/main/sphinx/security/built-in-system-access-control.md index a2435c04cee..cca8b6c772a 100644 --- a/docs/src/main/sphinx/security/built-in-system-access-control.md +++ b/docs/src/main/sphinx/security/built-in-system-access-control.md @@ -20,27 +20,26 @@ contain a comma separated list of the access control property files to use Trino offers the following built-in system access control implementations: -```{eval-rst} -.. list-table:: - :widths: 20, 80 - :header-rows: 1 - - * - Name - - Description - * - ``default`` - - All operations are permitted, except for user impersonation and triggering - :doc:`/admin/graceful-shutdown`. - - This is the default access control if none are configured. - * - ``allow-all`` - - All operations are permitted. - * - ``read-only`` - - Operations that read data or metadata are permitted, but none of the - operations that write data or metadata are allowed. - * - ``file`` - - Authorization rules are specified in a config file. See - :doc:`file-system-access-control`. -``` +:::{list-table} +:widths: 20, 80 +:header-rows: 1 + +* - Name + - Description +* - `default` + - All operations are permitted, except for user impersonation and triggering + [](/admin/graceful-shutdown). + + This is the default access control if none are configured. +* - `allow-all` + - All operations are permitted. +* - `read-only` + - Operations that read data or metadata are permitted, but none of the + operations that write data or metadata are allowed. +* - `file` + - Authorization rules are specified in a config file. See + [](/security/file-system-access-control). +::: If you want to limit access on a system level in any other way than the ones listed above, you must implement a custom {doc}`/develop/system-access-control`. diff --git a/docs/src/main/sphinx/security/certificate.md b/docs/src/main/sphinx/security/certificate.md index 8c725ec81c2..dfabad8d291 100644 --- a/docs/src/main/sphinx/security/certificate.md +++ b/docs/src/main/sphinx/security/certificate.md @@ -79,21 +79,20 @@ http-server.authentication.type=CERTIFICATE,PASSWORD The following configuration properties are also available: -```{eval-rst} -.. list-table:: Configuration properties - :widths: 50 50 - :header-rows: 1 - - * - Property name - - Description - * - ``http-server.authentication.certificate.user-mapping.pattern`` - - A regular expression pattern to :doc:`map all user names - ` for this authentication type to the format - expected by Trino. - * - ``http-server.authentication.certificate.user-mapping.file`` - - The path to a JSON file that contains a set of :doc:`user mapping - rules ` for this authentication type. -``` +:::{list-table} Configuration properties +:widths: 50 50 +:header-rows: 1 + +* - Property name + - Description +* - `http-server.authentication.certificate.user-mapping.pattern` + - A regular expression pattern to [map all user + names](/security/user-mapping) for this authentication type to the format + expected by Trino. +* - `http-server.authentication.certificate.user-mapping.file` + - The path to a JSON file that contains a set of [user mapping + rules](/security/user-mapping) for this authentication type. +::: ## Use certificate authentication with clients diff --git a/docs/src/main/sphinx/security/file-system-access-control.md b/docs/src/main/sphinx/security/file-system-access-control.md index d545cb8222f..64ef7a61547 100644 --- a/docs/src/main/sphinx/security/file-system-access-control.md +++ b/docs/src/main/sphinx/security/file-system-access-control.md @@ -115,24 +115,25 @@ The following table summarizes the permissions required for each SQL command: Permissions required for executing functions: -```{eval-rst} -.. list-table:: - :widths: 30, 10, 15, 30 - :header-rows: 1 - - * - SQL command - - Catalog - - Function permission - - Note - * - ``SELECT function()`` - - ``read-only`` - - ``execute``, ``grant_execute*`` - - ``grant_execute`` is required when the function is used in a SECURITY DEFINER view. - * - ``SELECT FROM TABLE(table_function())`` - - ``read-only`` - - ``execute``, ``grant_execute*`` - - ``grant_execute`` is required when the function is used in a SECURITY DEFINER view. -``` +:::{list-table} +:widths: 30, 10, 20, 40 +:header-rows: 1 + +* - SQL command + - Catalog + - Function permission + - Note +* - `SELECT function()` + - + - `execute`, `grant_execute*` + - `grant_execute` is required when the function is used in a `SECURITY DEFINER` + view. +* - `SELECT FROM TABLE(table_function())` + - `all` + - `execute`, `grant_execute*` + - `grant_execute` is required when the function is used in a `SECURITY DEFINER` + view. +::: (system-file-auth-visibility)= diff --git a/docs/src/main/sphinx/security/jwt.md b/docs/src/main/sphinx/security/jwt.md index 1a0b15c0a92..32858227f0f 100644 --- a/docs/src/main/sphinx/security/jwt.md +++ b/docs/src/main/sphinx/security/jwt.md @@ -74,39 +74,34 @@ http-server.authentication.jwt.key-file=https://cluster.example.net/.well-known/ The following configuration properties are available: -```{eval-rst} -.. list-table:: Configuration properties for JWT authentication - :widths: 50 50 - :header-rows: 1 - - * - Property - - Description - * - ``http-server.authentication.jwt.key-file`` - - Required. Specifies either the URL to a JWKS service or the path to a - PEM or HMAC file, as described below this table. - * - ``http-server.authentication.jwt.required-issuer`` - - Specifies a string that must match the value of the JWT's - issuer (``iss``) field in order to consider this JWT valid. - The ``iss`` field in the JWT identifies the principal that issued the - JWT. - * - ``http-server.authentication.jwt.required-audience`` - - Specifies a string that must match the value of the JWT's - Audience (``aud``) field in order to consider this JWT valid. - The ``aud`` field in the JWT identifies the recipients that the - JWT is intended for. - * - ``http-server.authentication.jwt.principal-field`` - - String to identify the field in the JWT that identifies the - subject of the JWT. The default value is ``sub``. This field is used to - create the Trino principal. - * - ``http-server.authentication.jwt.user-mapping.pattern`` - - A regular expression pattern to :doc:`map all user names - ` for this authentication system to the format - expected by the Trino server. - * - ``http-server.authentication.jwt.user-mapping.file`` - - The path to a JSON file that contains a set of - :doc:`user mapping rules ` for this - authentication system. -``` +:::{list-table} Configuration properties for JWT authentication +:widths: 50 50 +:header-rows: 1 + +* - Property + - Description +* - `http-server.authentication.jwt.key-file` + - Required. Specifies either the URL to a JWKS service or the path to a PEM or + HMAC file, as described below this table. +* - `http-server.authentication.jwt.required-issuer` + - Specifies a string that must match the value of the JWT's issuer (`iss`) + field in order to consider this JWT valid. The `iss` field in the JWT + identifies the principal that issued the JWT. +* - `http-server.authentication.jwt.required-audience` + - Specifies a string that must match the value of the JWT's Audience (`aud`) + field in order to consider this JWT valid. The `aud` field in the JWT + identifies the recipients that the JWT is intended for. +* - `http-server.authentication.jwt.principal-field` + - String to identify the field in the JWT that identifies the subject of the + JWT. The default value is `sub`. This field is used to create the Trino + principal. +* - `http-server.authentication.jwt.user-mapping.pattern` + - A regular expression pattern to [map all user names](/security/user-mapping) + for this authentication system to the format expected by the Trino server. +* - `http-server.authentication.jwt.user-mapping.file` + - The path to a JSON file that contains a set of [user mapping + rules](/security/user-mapping) for this authentication system. +::: Use the `http-server.authentication.jwt.key-file` property to specify either: diff --git a/docs/src/main/sphinx/security/oauth2.md b/docs/src/main/sphinx/security/oauth2.md index 2bce1f2b15c..9143406977b 100644 --- a/docs/src/main/sphinx/security/oauth2.md +++ b/docs/src/main/sphinx/security/oauth2.md @@ -18,8 +18,8 @@ Set the callback/redirect URL to `https:///oauth2 when configuring an OAuth 2.0 authorization server like an OpenID Connect (OIDC) provider. -If Web UI is enabled, set the post-logout callback URL to -`https:///ui/logout/logout.html` when configuring +If Web UI is enabled, set the post-logout callback URL to +`https:///ui/logout/logout.html` when configuring an OAuth 2.0 authentication server like an OpenID Connect (OIDC) provider. Using {doc}`TLS ` and {doc}`a configured shared secret @@ -84,89 +84,91 @@ web-ui.authentication.type=oauth2 The following configuration properties are available: -```{eval-rst} -.. list-table:: OAuth2 configuration properties - :widths: 40 60 - :header-rows: 1 - - * - Property - - Description - * - ``http-server.authentication.type`` - - The type of authentication to use. Must be set to ``oauth2`` to enable - OAuth2 authentication for the Trino coordinator. - * - ``http-server.authentication.oauth2.issuer`` - - The issuer URL of the IdP. All issued tokens must have this in the ``iss`` field. - * - ``http-server.authentication.oauth2.access-token-issuer`` - - The issuer URL of the IdP for access tokens, if different. - All issued access tokens must have this in the ``iss`` field. - Providing this value while OIDC discovery is enabled overrides the value - from the OpenID provider metadata document. - Defaults to the value of ``http-server.authentication.oauth2.issuer``. - * - ``http-server.authentication.oauth2.auth-url`` - - The authorization URL. The URL a user's browser will be redirected to in - order to begin the OAuth 2.0 authorization process. Providing this value - while OIDC discovery is enabled overrides the value from the OpenID - provider metadata document. - * - ``http-server.authentication.oauth2.token-url`` - - The URL of the endpoint on the authorization server which Trino uses to - obtain an access token. Providing this value while OIDC discovery is - enabled overrides the value from the OpenID provider metadata document. - * - ``http-server.authentication.oauth2.jwks-url`` - - The URL of the JSON Web Key Set (JWKS) endpoint on the authorization - server. It provides Trino the set of keys containing the public key - to verify any JSON Web Token (JWT) from the authorization server. - Providing this value while OIDC discovery is enabled overrides the value - from the OpenID provider metadata document. - * - ``http-server.authentication.oauth2.userinfo-url`` - - The URL of the IdPs ``/userinfo`` endpoint. If supplied then this URL is - used to validate the OAuth access token and retrieve any associated - claims. This is required if the IdP issues opaque tokens. Providing this - value while OIDC discovery is enabled overrides the value from the OpenID - provider metadata document. - * - ``http-server.authentication.oauth2.client-id`` - - The public identifier of the Trino client. - * - ``http-server.authentication.oauth2.client-secret`` - - The secret used to authorize Trino client with the authorization server. - * - ``http-server.authentication.oauth2.additional-audiences`` - - Additional audiences to trust in addition to the client ID which is - always a trusted audience. - * - ``http-server.authentication.oauth2.scopes`` - - Scopes requested by the server during the authorization challenge. See: - https://tools.ietf.org/html/rfc6749#section-3.3 - * - ``http-server.authentication.oauth2.challenge-timeout`` - - Maximum :ref:`duration ` of the authorization challenge. - Default is ``15m``. - * - ``http-server.authentication.oauth2.state-key`` - - A secret key used by the SHA-256 - `HMAC `_ - algorithm to sign the state parameter in order to ensure that the - authorization request was not forged. Default is a random string - generated during the coordinator start. - * - ``http-server.authentication.oauth2.user-mapping.pattern`` - - Regex to match against user. If matched, the user name is replaced with - first regex group. If not matched, authentication is denied. Default is - ``(.*)`` which allows any user name. - * - ``http-server.authentication.oauth2.user-mapping.file`` - - File containing rules for mapping user. See :doc:`/security/user-mapping` - for more information. - * - ``http-server.authentication.oauth2.principal-field`` - - The field of the access token used for the Trino user principal. Defaults to ``sub``. Other commonly used fields include ``sAMAccountName``, ``name``, ``upn``, and ``email``. - * - ``http-server.authentication.oauth2.oidc.discovery`` - - Enable reading the `OIDC provider metadata `_. - Default is ``true``. - * - ``http-server.authentication.oauth2.oidc.discovery.timeout`` - - The timeout when reading OpenID provider metadata. Default is ``30s``. - * - ``http-server.authentication.oauth2.oidc.use-userinfo-endpoint`` - - Use the value of ``userinfo_endpoint`` `in the provider metadata `_. - When a ``userinfo_endpoint`` value is supplied this URL is used to - validate the OAuth 2.0 access token, and retrieve any associated claims. - This flag allows ignoring the value provided in the metadata document. - Default is ``true``. - * - ``http-server.authentication.oauth2.end-session-url`` - - The URL of the endpoint on the authorization server to which user's browser - will be redirected to so that End-User will be logged out from the authorization - server when logging out from Trino. -``` +:::{list-table} OAuth2 configuration properties +:widths: 40 60 +:header-rows: 1 + +* - Property + - Description +* - `http-server.authentication.type` + - The type of authentication to use. Must be set to `oauth2` to enable OAuth2 + authentication for the Trino coordinator. +* - `http-server.authentication.oauth2.issuer` + - The issuer URL of the IdP. All issued tokens must have this in the `iss` + field. +* - `http-server.authentication.oauth2.access-token-issuer` + - The issuer URL of the IdP for access tokens, if different. All issued access + tokens must have this in the `iss` field. Providing this value while OIDC + discovery is enabled overrides the value from the OpenID provider metadata + document. Defaults to the value of + `http-server.authentication.oauth2.issuer`. +* - `http-server.authentication.oauth2.auth-url` + - The authorization URL. The URL a user's browser will be redirected to in + order to begin the OAuth 2.0 authorization process. Providing this value + while OIDC discovery is enabled overrides the value from the OpenID provider + metadata document. +* - `http-server.authentication.oauth2.token-url` + - The URL of the endpoint on the authorization server which Trino uses to + obtain an access token. Providing this value while OIDC discovery is enabled + overrides the value from the OpenID provider metadata document. +* - `http-server.authentication.oauth2.jwks-url` + - The URL of the JSON Web Key Set (JWKS) endpoint on the authorization server. + It provides Trino the set of keys containing the public key to verify any + JSON Web Token (JWT) from the authorization server. Providing this value + while OIDC discovery is enabled overrides the value from the OpenID provider + metadata document. +* - `http-server.authentication.oauth2.userinfo-url` + - The URL of the IdPs `/userinfo` endpoint. If supplied then this URL is used + to validate the OAuth access token and retrieve any associated claims. This + is required if the IdP issues opaque tokens. Providing this value while OIDC + discovery is enabled overrides the value from the OpenID provider metadata + document. +* - `http-server.authentication.oauth2.client-id` + - The public identifier of the Trino client. +* - `http-server.authentication.oauth2.client-secret` + - The secret used to authorize Trino client with the authorization server. +* - `http-server.authentication.oauth2.additional-audiences` + - Additional audiences to trust in addition to the client ID which is + always a trusted audience. +* - `http-server.authentication.oauth2.scopes` + - Scopes requested by the server during the authorization challenge. See: + https://tools.ietf.org/html/rfc6749#section-3.3 +* - `http-server.authentication.oauth2.challenge-timeout` + - Maximum [duration](prop-type-duration) of the authorization challenge. + Default is `15m`. +* - `http-server.authentication.oauth2.state-key` + - A secret key used by the SHA-256 [HMAC](https://tools.ietf.org/html/rfc2104) + algorithm to sign the state parameter in order to ensure that the + authorization request was not forged. Default is a random string generated + during the coordinator start. +* - `http-server.authentication.oauth2.user-mapping.pattern` + - Regex to match against user. If matched, the user name is replaced with + first regex group. If not matched, authentication is denied. Default is + `(.*)` which allows any user name. +* - `http-server.authentication.oauth2.user-mapping.file` + - File containing rules for mapping user. See [](/security/user-mapping) for + more information. +* - `http-server.authentication.oauth2.principal-field` + - The field of the access token used for the Trino user principal. Defaults to + `sub`. Other commonly used fields include `sAMAccountName`, `name`, + `upn`, and `email`. +* - `http-server.authentication.oauth2.oidc.discovery` + - Enable reading the [OIDC provider metadata](https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderMetadata). + Default is `true`. +* - `http-server.authentication.oauth2.oidc.discovery.timeout` + - The timeout when reading OpenID provider metadata. Default is `30s`. +* - `http-server.authentication.oauth2.oidc.use-userinfo-endpoint` + - Use the value of `userinfo_endpoint` in the [provider + metadata](https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderMetadata). + When a `userinfo_endpoint` value is supplied this URL is used to validate + the OAuth 2.0 access token, and retrieve any associated claims. This flag + allows ignoring the value provided in the metadata document. Default is + `true`. +* - `http-server.authentication.oauth2.end-session-url` + - The URL of the endpoint on the authentication server to which the user's + browser is redirected to so that End-User is logged out from the + authentication server when logging out from Trino. +::: (trino-oauth2-refresh-tokens)= @@ -215,33 +217,30 @@ http-server.authentication.oauth2.scopes=openid,offline_access [or offline] The following configuration properties are available: -```{eval-rst} -.. list-table:: OAuth2 configuration properties for refresh flow - :widths: 40 60 - :header-rows: 1 - - * - Property - - Description - * - ``http-server.authentication.oauth2.refresh-tokens.issued-token.timeout`` - - Expiration time for an issued token, which is the Trino-encrypted token - that contains an access token and a refresh token. The timeout value must - be less than or equal to the :ref:`duration ` of the - refresh token expiration issued by the IdP. Defaults to ``1h``. The - timeout value is the maximum session time for an OAuth2-authenticated - client with refresh tokens enabled. For more details, see - :ref:`trino-oauth2-troubleshooting`. - * - ``http-server.authentication.oauth2.refresh-tokens.issued-token.issuer`` - - Issuer representing the coordinator instance, that is referenced in the - issued token, defaults to ``Trino_coordinator``. The current - Trino version is appended to the value. This is mainly used for - debugging purposes. - * - ``http-server.authentication.oauth2.refresh-tokens.issued-token.audience`` - - Audience representing this coordinator instance, that is used in the - issued token. Defaults to ``Trino_coordinator``. - * - ``http-server.authentication.oauth2.refresh-tokens.secret-key`` - - Base64-encoded secret key used to encrypt the generated token. - By default it's generated during startup. -``` +:::{list-table} OAuth2 configuration properties for refresh flow +:widths: 40 60 +:header-rows: 1 + +* - Property + - Description +* - `http-server.authentication.oauth2.refresh-tokens.issued-token.timeout` + - Expiration time for an issued token, which is the Trino-encrypted token that + contains an access token and a refresh token. The timeout value must be less + than or equal to the [duration](prop-type-duration) of the refresh token + expiration issued by the IdP. Defaults to `1h`. The timeout value is the + maximum session time for an OAuth2-authenticated client with refresh tokens + enabled. For more details, see [](trino-oauth2-troubleshooting). +* - `http-server.authentication.oauth2.refresh-tokens.issued-token.issuer` + - Issuer representing the coordinator instance, that is referenced in the + issued token, defaults to `Trino_coordinator`. The current Trino version is + appended to the value. This is mainly used for debugging purposes. +* - `http-server.authentication.oauth2.refresh-tokens.issued-token.audience` + - Audience representing this coordinator instance, that is used in the + issued token. Defaults to `Trino_coordinator`. +* - `http-server.authentication.oauth2.refresh-tokens.secret-key` + - Base64-encoded secret key used to encrypt the generated token. By default + it's generated during startup. +::: (trino-oauth2-troubleshooting)= diff --git a/docs/src/main/sphinx/sql/show-stats.md b/docs/src/main/sphinx/sql/show-stats.md index 56f2c0b407d..2c5f27af012 100644 --- a/docs/src/main/sphinx/sql/show-stats.md +++ b/docs/src/main/sphinx/sql/show-stats.md @@ -19,40 +19,39 @@ table lists the returned columns and what statistics they represent. Any additional statistics collected on the data source, other than those listed here, are not included. -```{eval-rst} -.. list-table:: Statistics - :widths: 20, 40, 40 - :header-rows: 1 +:::{list-table} Statistics +:widths: 20, 40, 40 +:header-rows: 1 - * - Column - - Description - - Notes - * - ``column_name`` - - The name of the column - - ``NULL`` in the table summary row - * - ``data_size`` - - The total size in bytes of all of the values in the column - - ``NULL`` in the table summary row. Available for columns of :ref:`string - ` data types with variable widths. - * - ``distinct_values_count`` - - The estimated number of distinct values in the column - - ``NULL`` in the table summary row - * - ``nulls_fractions`` - - The portion of the values in the column that are ``NULL`` - - ``NULL`` in the table summary row. - * - ``row_count`` - - The estimated number of rows in the table - - ``NULL`` in column statistic rows - * - ``low_value`` - - The lowest value found in this column - - ``NULL`` in the table summary row. Available for columns of :ref:`DATE - `, :ref:`integer `, - :ref:`floating-point `, and - :ref:`fixed-precision ` data types. - * - ``high_value`` - - The highest value found in this column - - ``NULL`` in the table summary row. Available for columns of :ref:`DATE - `, :ref:`integer `, - :ref:`floating-point `, and - :ref:`fixed-precision ` data types. -``` +* - Column + - Description + - Notes +* - `column_name` + - The name of the column + - `NULL` in the table summary row +* - `data_size` + - The total size in bytes of all of the values in the column + - `NULL` in the table summary row. Available for columns of + [string](string-data-types) data types with variable widths. +* - `distinct_values_count` + - The estimated number of distinct values in the column + - `NULL` in the table summary row +* - `nulls_fractions` + - The portion of the values in the column that are `NULL` + - `NULL` in the table summary row. +* - `row_count` + - The estimated number of rows in the table + - `NULL` in column statistic rows +* - `low_value` + - The lowest value found in this column + - `NULL` in the table summary row. Available for columns of + [DATE](date-data-type), [integer](integer-data-types), + [floating-point](floating-point-data-types), and + [fixed-precision](fixed-precision-data-types) data types. +* - `high_value` + - The highest value found in this column + - `NULL` in the table summary row. Available for columns of + [DATE](date-data-type), [integer](integer-data-types), + [floating-point](floating-point-data-types), and + [fixed-precision](fixed-precision-data-types) data types. + :::