From 772f172fb517d9d4087420f14dbd0c71e8badc3b Mon Sep 17 00:00:00 2001 From: voetberg Date: Fri, 2 Aug 2024 10:36:20 -0500 Subject: [PATCH 1/3] Make a script to pull default confg params into a json file, read in jsx, insert into config docs --- docs/index.md | 2 +- ...meters.md => configuration_parameters.mdx} | 445 +++++++++--------- tools/build_documentation.sh | 2 + tools/get_config.py | 15 + 4 files changed, 241 insertions(+), 223 deletions(-) rename docs/operator/{configuration_parameters.md => configuration_parameters.mdx} (54%) create mode 100755 tools/get_config.py diff --git a/docs/index.md b/docs/index.md index 69c6f390b6b..fc07acc4d96 100644 --- a/docs/index.md +++ b/docs/index.md @@ -51,7 +51,7 @@ pre-requisites section under each of the topics! - [Installing Rucio Daemons](operator/installing_daemons.md) - [Monitoring](operator/monitoring.md) - [Database](operator/database) -- [Configuration parameters](operator/configuration_parameters.md) +- [Configuration parameters](operator/configuration_parameters.mdx) ## Developer Documentation diff --git a/docs/operator/configuration_parameters.md b/docs/operator/configuration_parameters.mdx similarity index 54% rename from docs/operator/configuration_parameters.md rename to docs/operator/configuration_parameters.mdx index 98a1aa470e9..f7cc3aa3170 100644 --- a/docs/operator/configuration_parameters.md +++ b/docs/operator/configuration_parameters.mdx @@ -2,7 +2,27 @@ id: configuration_parameters title: Configuration parameters --- -Description of the configuration parameters splitted in three different places: + +export function Defaults({section, option}) { + var ConfigDefaults = require('../default_configurations.json'); + const noDefault = "No Default." + + try { + var defaultSetting = ConfigDefaults[section][option]; + if (defaultSetting == null) { + return noDefault + } else { + return "Default: " + defaultSetting + "." + + } + } + catch (e) { + console.log(e) + return noDefault + } +} + +Description of the configuration parameters split in three different places: Rucio configuration file (`rucio.cfg`), Rucio configuration table and RSE attributes. ## Rucio Configuration File (`rucio.cfg`) @@ -19,21 +39,22 @@ Rucio will look for the config in the following locations - ### Options and Defaults #### **accounts** - - **special_accounts** + - **special_accounts** + #### **alembic** - **cfg**: Path to the configuration file (.ini) for Alembic. Example: - `/opt/rucio/etc/alembic.ini`. No default. + `/opt/rucio/etc/alembic.ini`. + #### **api** - - **endpoints**: _(Optional)_ Endpoints separated by commas. Default: - `['accountlimits', 'accounts', 'config', 'credentials', 'dids', 'export', - 'heartbeats', 'identities', 'import', 'lifetime_exceptions', 'locks', - 'meta', 'ping', 'redirect', 'replicas', 'requests', 'rses', 'rules', - 'scopes', 'subscriptions']`. + - **endpoints**: _(Optional)_ Endpoints separated by commas. + #### **auditor** - **cache**: Path to the folder to store the `rucio-auditor` cache. Example: - `/opt/rucio/auditor-cache`. No default. + `/opt/rucio/auditor-cache`. + - **results**: Path to the folder to store the `rucio-auditor` - results. Example: `/opt/rucio/auditor-results`. No default. + results. Example: `/opt/rucio/auditor-results`. + - **threshold**: _(Optional)_ Floating number used in a sanity check, comparing the number of entries with the total number of files on the RSE: @@ -45,37 +66,41 @@ Rucio will look for the config in the following locations - if len(lost_replicas) > threshold * usage['files'] ``` - Default: `0.2`. + #### **bb8** - **dump_production_day**: _(Optional)_ Day of the week of the most recent dump. Values: `{Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, - Saturday}`. Default: `None`. + Saturday}`. + - **dump_url_template**: _(Optional)_ URL of the template (structure) of a - dump. Default: - `http://rucio-analytix.cern.ch:8080/LOCKS/GetFileFromHDFS?date=${date}&rse=${rse}`. + dump. + #### **bootstrap** - **gss_email**: _(Optional)_ Email of the Kerberos auth method which identity - is specified in `gss_identity`. - - **gss_identity**: _(Optional)_ Identity of the Kerberos auth method. + is specified in `gss_identity`. + + - **gss_identity**: _(Optional)_ Identity of the Kerberos auth method. + - **saml_email** - **saml_id** - **ssh_email**: _(Optional)_ Email of the SSH auth method which identity is - specified in `ssh_identity`. Default: `ph-adp-ddm-lab@cern.ch`. - - **ssh_identity**: _(Optional)_ SSH auth using an RSA key. Default: ``` - ssh-rsa - AAAAB3NzaC1yc2EAAAABIwAAAQEAq5LySllrQFpPL614sulXQ7wnIr1aGhGtl8b+HCB/0FhMSMTHwSjX78UbfqEorZV16rXrWPgUpvcbp2hqctw6eCbxwqcgu3uGWaeS5A0iWRw7oXUh6ydnVy89zGzX1FJFFDZ+AgiZ3ytp55tg1bjqqhK1OSC0pJxdNe878TRVVo5MLI0S/rZY2UovCSGFaQG2iLj14wz/YqI7NFMUuJFR4e6xmNsOP7fCZ4bGMsmnhR0GmY0dWYTupNiP5WdYXAfKExlnvFLTlDI5Mgh4Z11NraQ8pv4YE1woolYpqOc/IMMBBXFniTT4tC7cgikxWb9ZmFe+r4t6yCDpX4IL8L5GOQ== - ddmlab ``` + specified in `ssh_identity`. + + - **ssh_identity**: _(Optional)_ SSH auth using an RSA key. + - **userpass_email**: _(Optional)_ Email of the root account which name is - specified in `userpass_identity`. Default: `ph-adp-ddm-lab@cern.ch`. - - **userpass_identity**: _(Optional)_ Name of the root account. Default: - `ddmlab`. + specified in `userpass_identity`. + + - **userpass_identity**: _(Optional)_ Name of the root account. + - **userpass_pwd**: _(Optional)_ Password of the root account which name is - specified in `userpass_identity`. Default: `secret`. + specified in `userpass_identity`. - **x509_email**: _(Optional)_ Email of the X.509 identity specified in - `x509_identity`. Default: `ph-adp-ddm-lab@cern.ch`. - - **x509_identity**: _(Optional)_ Identity of the X.509 certificate. Default: - `emailAddress=ph-adp-ddm-lab@cern.ch,CN=DDMLAB Client - Certificate,OU=PH-ADP-CO,O=CERN,ST=Geneva,C=CH`. + `x509_identity`. + + - **x509_identity**: _(Optional)_ Identity of the X.509 certificate. + + #### **c3po** - **algorithms**: _(Optional)_ Placement algorithm. @@ -85,186 +110,170 @@ Rucio will look for the config in the following locations - Values: `{simple, t2_free_space, t2_free_space_only_pop, t2_free_space_only_pop_with_network}`. - Default: `None`. + - **ca_cert**: _(Optional)_ Path of the certificate for Elasticsearch. - Default: `False`. + - **elastic_index**: _(Optional)_ Index (database) to use in Elasticsearch. + - **elastic_pass**: _(Optional)_ Password of the username defined in - `elastic_user` to authenticate to Elasticsearch. No default. + `elastic_user` to authenticate to Elasticsearch. - **elastic_url**: URL of Elasticsearch. Example: - `http://aianalytics01.cern.ch:9200`. - - **elastic_user**: _(Optional)_ Username to authenticate to Elasticsearch. No - default. + `http://aianalytics01.cern.ch:9200`. + - **elastic_user**: _(Optional)_ Username to authenticate to Elasticsearch. - **placement_algorithm** - - **redis_host**: URL of the Redis database. - - **redis_port**: Port of the Redis database defined in `redis_host`. + - **redis_host**: URL of the Redis database. + - **redis_port**: Port of the Redis database defined in `redis_host`. #### **c3po-popularity** - **elastic_url** #### **c3po-site-mapper** - **ddm_url**: URL for DDM. Example: - `http://atlas-agis-api.cern.ch/request/ddmendpoint/query/list/?json`. + `http://atlas-agis-api.cern.ch/request/ddmendpoint/query/list/?json`. - **panda_url**: URL for PanDA. Example: - `http://atlas-agis-api.cern.ch/request/pandaqueue/query/list/?json`. + `http://atlas-agis-api.cern.ch/request/pandaqueue/query/list/?json`. #### **c3po-workload** - **panda_url**: URL for PanDA. Example: - `http://bigpanda.cern.ch/jobs/?category=analysis&jobstatus=running`. + `http://bigpanda.cern.ch/jobs/?category=analysis&jobstatus=running`. - **window**: Integer. #### **c3po-network-metrics** - **prefix**: - - **redis_host**: URL of the Redis database. - - **redis_port**: Port of the Redis database defined in `redis_host`. + - **redis_host**: URL of the Redis database. + - **redis_port**: Port of the Redis database defined in `redis_host`. #### **cache** - - **url**: _(Optional)_ URL of the cache. Default: `127.0.0.1:11211`. + - **url**: _(Optional)_ URL of the cache. - **use_external_cache_for_auth_tokens**: _(Optional)_ if True, use remote - cache provider for auth tokens. If False, use a private in-memory cache. - Default: `False` + cache provider for auth tokens. If False, use a private in-memory cache. #### **common** - - **extract_scope**: _(Optional)_ Default: `atlas`. - - **logdir**: Path of the directory for logs. Contains `auditor.log`. + - **extract_scope**: _(Optional)_ + - **logdir**: Path of the directory for logs. Contains `auditor.log`. - **logformat**: _(Optional)_ Formatter of the log. See [the logging formatter documentation](https://docs.python.org/3/library/logging.html#logging.Formatter) - **loglevel**: _(Optional)_. Set the root logger level to the specified level. Values: `{'NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'}`. See [documentation for logging levels](https://docs.python.org/3/library/logging.html#levels) - Default: `DEBUG`. For `rucio-auditor` default is `WARNING`. + For `rucio-auditor` default is `WARNING`. - **mailtemplatedir**: _(Optional)_ Path of the folder with mail templates - (.tmpl). Example: `/opt/rucio/etc/mail_templates`. - - **multi_vo**: _(Optional)_ Default: `False`. + (.tmpl). Example: `/opt/rucio/etc/mail_templates`. + - **multi_vo**: _(Optional)_ #### **conveyor** - - **allow_user_oidc_tokens**: _(Optional)_ Boolean. Default: `False`. - - **bring_online**: Integer, bring online timeout. Default: `43200`. + - **allow_user_oidc_tokens**: _(Optional)_ Boolean. + - **bring_online**: Integer, bring online timeout. - **cacert** - **cache_time**: _(Optional)_ Integer, expiration time in seconds passed to - the dogpile system. Default: `600`. - - **failover_scheme**: Failover schemes. Default: `None`. - - **filter_transfertool**: _(Optional)_ Default: `None`. + the dogpile system. + - **failover_scheme**: Failover schemes. + - **filter_transfertool**: _(Optional)_ - **ftshosts**: URL of the [File Transfer Service (FTS)](https://fts.web.cern.ch/) hosts (separated by commas). Example: `https://fts3-pilot.cern.ch:8446, https://fts3-pilot.cern.ch:8446`. - - **globus_auth_app**: _(Optional)_ Default: `None`. + - **globus_auth_app**: _(Optional)_ - **max_time_in_queue**: _(Optional)_ (separated by - commas). Default: `{}`. - - **poll_timeout**: Float, timeout. Default: `None`. - - **queue_mode**: _(Optional)_ Values: `{'strict', 'default'}`Default: - `default`. - - **request_oidc_audience**: _(Optional)_. Default: `fts:example`. - - **request_oidc_scope**: _(Optional)_. Default: `fts:submit-transfer`. - - **scheme**: _(Optional)_ Schemes to process (separated by commas). Default: - `None`. - - **submit_timeout**: _(Optional)_ Timeout. Default: `None`. - - **transfertool**: _(Optional)_ Default: `None`. - - **transfertype**: _(Optional)_. Values: `{bulk, single}`. Default: `single`. + commas) . + - **poll_timeout**: Float, timeout. + - **queue_mode**: _(Optional)_ Values: `{'strict', 'default'}` + - **request_oidc_audience**: _(Optional)_. + - **request_oidc_scope**: _(Optional)_. + - **scheme**: _(Optional)_ Schemes to process (separated by commas). + - **submit_timeout**: _(Optional)_ Timeout. + - **transfertool**: _(Optional)_ + - **transfertype**: _(Optional)_. Values: `{bulk, single}`. - **usercert**: Path to the certificate for the FTS3 implementation of a Rucio - transfertool. Default: `None`. - - **user_activities**: _(Optional)_ Default: `['user', - 'user_test']`. - - **user_transfers**: _(Optional)_ Default: `None`. - - **using_memcache**: _(Optional)_ Boolean. Default: `False`. - - **webdav_transfer_mode**: _(Optional)_. Default: `None`. + transfertool. + - **user_activities**: _(Optional)_ + - **user_transfers**: _(Optional)_ + - **using_memcache**: _(Optional)_ Boolean. + - **webdav_transfer_mode**: _(Optional)_. #### **core** - - **default_mail_from**: _(Optional)_ Default email. Default: `None`. + - **default_mail_from**: _(Optional)_ Default email. - **geoip_licence_key**: _(Optional)_ License key for GeoLite2. Get a free - licence key at [the signup page](https://www.maxmind.com/en/geolite2/signup). Default: - `NOLICENCE`. - - **use_temp_tables**: _(Optional)_ Use Rucio with temporary table workflows. - Default: `False`. + licence key at [the signup page](https://www.maxmind.com/en/geolite2/signup). + - **use_temp_tables**: _(Optional)_ Use Rucio with temporary table workflows. #### **client** {#client_config} - - **account**: Rucio account. Example: `root`. + - **account**: Rucio account. Example: `root`. - **auth_host**: URL of the host of the rucio authentication server. Example: - `https://rucio-auth-prod.cern.ch:443`. + `https://rucio-auth-prod.cern.ch:443`. - **auth_token_file_path**: _(Optional)_ If token file path is defined in the rucio.cfg file, use that file. Currently this prevents authenticating as - another user or VO. Default: `None`. + another user or VO. - **auth_type**: Type of authentication in rucio. Values: `{userpass, x509, - x509_proxy, gss, ssh, saml, oidc}`. + x509_proxy, gss, ssh, saml, oidc}`. - **ca_cert**: Path of the cert file for HTTPS. Example: - `/opt/rucio/etc/web/ca.crt`. + `/opt/rucio/etc/web/ca.crt`. - **client_cert**: Path of the X.509 client cert file. This can be overwitten by the `RUCIO_CLIENT_CERT` environment variable. Example: - `/opt/rucio/etc/web/client.crt`. + `/opt/rucio/etc/web/client.crt`. - **client_key**: Path of the X.509 client key file for the cert defined in `client_cert`. This can be overwitten by the `RUCIO_CLIENT_KEY` environment - variable. Example: `/opt/rucio/etc/web/client.key`. + variable. Example: `/opt/rucio/etc/web/client.key`. - **client_x509_proxy**: Path of the X.509 client proxy. Mandatory if - `auth_type = x509_proxy`. - - **oidc_audience**: _(Optional)_ Only used if `auth_type = oidc`. Default: - `None`. + `auth_type = x509_proxy`. + - **oidc_audience**: _(Optional)_ Only used if `auth_type = oidc`. - **oidc_auto**: _(Optional)_ Boolean. Only used if `auth_type = - oidc`. Default: `False`. - - **oidc_issuer**: _(Optional)_ Only used if `auth_type = oidc`. Default: - `None`. - - **oidc_password**: _(Optional)_ Only used if `auth_type = oidc`. Default: - `None`. + oidc`. + - **oidc_issuer**: _(Optional)_ Only used if `auth_type = oidc`. + - **oidc_password**: _(Optional)_ Only used if `auth_type = oidc`. - **oidc_refresh_lifetime**: _(Optional)_ Only used if `auth_type = - oidc`. Default: `None`. - - **oidc_scope**: _(Optional)_ Only used if `auth_type = oidc`. Default: - `openid profile`. - - **oidc_username**: _(Optional)_ Only used if `auth_type = oidc`. Default: - `None`. + oidc`. + - **oidc_scope**: _(Optional)_ Only used if `auth_type = oidc`. + - **oidc_username**: _(Optional)_ Only used if `auth_type = oidc`. - **password**: Password of the user specified in `username`. Mandatory if `auth_type = userpass` or `auth_type = saml`. - **protocol_stat_retries**: _(Optional)_ Integer, number of retries if stat file fails. The time of the retries are: 1s, 2s, 4s, 8s, 16s, 32s - later. Default: `6`. + later. - **request_retries**: _(Optional)_ Integer, number of retries if an - unauthorized error is returned. Default: `3`. + unauthorized error is returned. - **rucio_host**: URL of rucio host. Example: `https://rucio-server-prod.cern.ch:443`. - **ssh_private_key**: Path of the SSH private key. Mandatory if `auth_type = - ssh`. Example: `$HOME/.ssh/id_rsa`. - - **username**: Mandatory if `auth_type = userpass` or `auth_type = saml`. - - **vo**: _(Optional)_ VO name. Default: `def`. + ssh`. Example: `$HOME/.ssh/id_rsa`. + - **username**: Mandatory if `auth_type = userpass` or `auth_type = saml`. + - **vo**: _(Optional)_ VO name. #### **credentials** - - **gcs**: _(Optional)_ Path of the Google Cloud Storage credentials. Default: - `/opt/rucio/etc/google-cloud-storage-test.json`. - - **signature_lifetime**: _(Optional)_ ?. Default: `600`. + - **gcs**: _(Optional)_ Path of the Google Cloud Storage credentials. + - **signature_lifetime**: _(Optional)_ ?. #### **database** {#db_config} - **default**: Type of the SQL connection. Values: `{mysql, postgresql, - sqlite, oracle}`. + sqlite, oracle}`. - **echo**: Enable echo for database logs. Values: `{0 (disable), 1 - (enable)}`. - - **echo_pool**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.echo_pool) - - **max_overflow**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.max_overflow) - - **pool_recycle**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_recycle) - - **pool_reset_on_return**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_reset_on_return) - - **pool_size**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_size) - - **pool_timeout**: [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_timeout) + (enable)}`. + - **echo_pool**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.echo_pool) + - **max_overflow**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.max_overflow) + - **pool_recycle**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_recycle) + - **pool_reset_on_return**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_reset_on_return) + - **pool_size**: See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_size) + - **pool_timeout**: [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_timeout) - **poolclass**: Which connection pooling mechanism to use. Values: `nullpool` (disables pooling), `queuepool` (default for all but SQLite engine), or - `singletonthreadpool` (default for SQLite engine). - See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.poolclass) + `singletonthreadpool` (default for SQLite engine). + See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.poolclass) - **schema**: _(Optional)_ Schema to be applied to a database, if not set in - config, try to create automatically. - - **use_threadlocal** + config, try to create automatically. + - **use_threadlocal** #### **download** - **transfer_speed_timeout**: _(Optional)_ Minimum allowed average transfer speed (in KBps). Default: `500`. Used to dynamically compute the timeout if - `--transfer-timeout` not set. Is not supported for `--pfn`. + `--transfer-timeout` not set. Is not supported for `--pfn`. - **transfer_timeout**: _(Optional)_ Transfer timeout (in seconds). Default: computed dynamically from `--transfer-speed-timeout`. If set to any value >= - 0, `--transfer-speed-timeout` is ignored. + 0, `--transfer-speed-timeout` is ignored. #### **es-atlas** - - **ca_cert**: _(Optional)_ Path of the certificate for Elasticsearch. No - default. + - **ca_cert**: _(Optional)_ Path of the certificate for Elasticsearch. - **password**: _(Optional)_ Password of the username defined in `username` to - authenticate to Elasticsearch. No default. + authenticate to Elasticsearch. - **url**: _(Optional)_ URL of Elasticsearch. Example: - `http://aianalytics01.cern.ch:9200`. No default. - - **username**: _(Optional)_ Username to authenticate to Elasticsearch. No - default. + `http://aianalytics01.cern.ch:9200`. + - **username**: _(Optional)_ Username to authenticate to Elasticsearch. #### **hermes** - **elastic_endpoint**: _(Optional)_ URL of Elasticsearch. Example: `http://aianalytics01.cern.ch:9200`. Mandatory if `elastic` is specified in - `services_list`. + `services_list`. - **influxdb_endpoint**: _(Optional)_ URL of InfluxDB. Mandatory if `influx` - is specified in `services_list`. + is specified in `services_list`. #### **importer** - - **attr_sync_method**: _(Optional)_ Values: `{append, edit, hard}`. Default: - `edit`. - - **rse_sync_method**: _(Optional)_ Values: `{append, edit, hard}`. Default: - `edit`. + - **attr_sync_method**: _(Optional)_ Values: `{append, edit, hard}`. + - **rse_sync_method**: _(Optional)_ Values: `{append, edit, hard}`. #### **injector** - **adler32** - **bytes** @@ -273,173 +282,165 @@ Rucio will look for the config in the following locations - #### **lifetime** - **directory**: _(Optional)_ Path to the policies directory with JSON files named `config_DTYPE.json`, where `DTYPE` is a value in `{data, mc, valid, - other}`. Default: `/opt/rucio/etc/policies`. + other}`. #### **logging** - - ***CFG_OPTION***: _(Optional)_ ?. Default: `None`. + - ***CFG_OPTION***: _(Optional)_ ?. #### **messaging-cache** - **account** - - **broker_virtual_host**: _(Optional)_ ? + - **broker_virtual_host**: _(Optional)_ ? - **brokers**: Default message broker name for `rucio-cache-client`. Ignored - if `rucio-cache-client` executed with `--broker`. + if `rucio-cache-client` executed with `--broker`. - **destination**: Default message broker topic for `rucio-cache-client`. Ignored if `rucio-cache-client` executed with - `--destination`. + `--destination`. - port - **ssl_cert_file**: Default certificate file for `rucio-cache-client`. Ignored if `rucio-cache-client` executed with - `--certificate`. + `--certificate`. - **ssl_key_file**: Default certificate key file for `rucio-cache-client`. Ignored if `rucio-cache-client` executed with - `--certificate-key`. + `--certificate-key`. - **voname** #### **messaging-fts3** - - **broker_virtual_host**: _(Optional)_ ?. No default. + - **broker_virtual_host**: _(Optional)_ ?. - **brokers**: Brokers separated by commas. Example: `dashb-test-mb.cern.ch`. - **destination**: Name of the destination topic. Example: - `/topic/transfer.fts_monitoring_queue_state`. - - **nonssl_port**: _(Optional)_ Port of the broker if `use_ssl` is not set. + `/topic/transfer.fts_monitoring_queue_state`. + - **nonssl_port**: _(Optional)_ Port of the broker if `use_ssl` is not set. - **password**: _(Optional)_ Password of the `username`. Only used if - `use_ssl` is not set. No default. + `use_ssl` is not set. - **port** - - **ssl_cert_file**: _(Optional)_ Path of the certificate file. No default. + - **ssl_cert_file**: _(Optional)_ Path of the certificate file. - **ssl_key_file**: _(Optional)_ Path of the certificate key file defined in - `ssl_cert_file`. No default. - - **use_ssl**: _(Optional)_ Boolean. Default: `True`. - - **username**: _(Optional)_ Username of the broker. Only used if `use_ssl` is - not set. No default. + `ssl_cert_file`. + - **use_ssl**: _(Optional)_ Boolean. + - **username**: _(Optional)_ Username of the broker. Only used if `username` is + not set. - **voname** #### **messaging-hermes** - - **broker_virtual_host**: _(Optional)_ No default. - - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. + - **broker_virtual_host**: _(Optional)_ + - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. - **destination**: Name of the destination topic. Example: - `/topic/rucio.events`. - - **email_from**: Example: `Rucio `. - - **email_test**: Example: `spamspamspam@cern.ch`. - - **nonssl_port**: _(Optional)_ Port of the broker if `use_ssl` is not set. + `/topic/rucio.events`. + - **email_from**: Example: `Rucio `. + - **email_test**: Example: `spamspamspam@cern.ch`. + - **nonssl_port**: _(Optional)_ Port of the broker if `use_ssl` is not set. - **password**: _(Optional)_ Password of the `username`. Mandatory if - `use_ssl` is not set. No default. - - **port**: _(Optional)_ Port of the broker if `use_ssl` is set. - - **ssl_cert_file**: _(Optional)_ Path of the certificate file. No - default. Mandatory if `use_ssl` is set. + `use_ssl` is not set. + - **port**: _(Optional)_ Port of the broker if `use_ssl` is set. + - **ssl_cert_file**: _(Optional)_ Path of the certificate file. Mandatory if `use_ssl` is set. - **ssl_key_file**: _(Optional)_ Path of the certificate key file defined in - `ssl_cert_file`. No default. Mandatory if `use_ssl` is set. - - **use_ssl**: _(Optional)_ Boolean. Default: `True`. + `ssl_cert_file`. No default. Mandatory if `use_ssl` is set. + - **use_ssl**: _(Optional)_ Boolean. - **username**: _(Optional)_ Username of the broker. Mandatory if `use_ssl` is - not set. No default. + not set. - **voname** #### **metadata** - **plugins**: _(Optional)_ Metadata handler modules separated by - commas. Default: `rucio.core.did_meta_plugins.json_meta.JSONDidMeta`. + commas. #### **monitor** - - **enable_metrics**: _(Optional)_ Enable `statsd` metrics. Boolean. Default: - `False`. + - **enable_metrics**: _(Optional)_ Enable `statsd` metrics. Boolean. - **carbon_server**: _(Optional)_ Hostname or IP address of the `statsd` - server. Default: `localhost` - - **carbon_port**: _(Optional)_ Port of the `statsd` server. Default: `8125`. + server. + - **carbon_port**: _(Optional)_ Port of the `statsd` server. - **user_scope**: _(Optional)_ Prefix to distinguish and group stats from an - application or environment. Default: `rucio`. - - **metrics_port**: _(Optional)_ Port of Prometheus Python Client. Default: - `8080`. + application or environment. + - **metrics_port**: _(Optional)_ Port of Prometheus Python Client. #### **nagios** - **fts_servers** - **proxy** - **rfcproxy** #### **nongrid-trace** - - **broker_virtual_host**: _(Optional)_ ?. No default. - - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. - - **logformat**: _(Optional)_ Formatter of the log. See [logging documentation](https://docs.python.org/3/library/logging.html#logging.Formatter). + - **broker_virtual_host**: _(Optional)_ ?. + - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. + - **logformat**: _(Optional)_ Formatter of the log. See [logging documentation](https://docs.python.org/3/library/logging.html#logging.Formatter). - **loglevel**: _(Optional)_ Set the root logger level to the specified level. Values: `{'NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'}`. See [logging documentation](https://docs.python.org/3/library/logging.html#levels). - Default: `DEBUG`. + Default: - **password**: Password of the `username`. - - **topic**: Name of the destination topic. - - **tracedir**: _(Optional)_ Path of the directory for traces. Default: - `/var/log/rucio`. - - **username**: Username of the broker. + - **topic**: Name of the destination topic. + - **tracedir**: _(Optional)_ Path of the directory for traces. + - **username**: Username of the broker. #### **oidc** - - **admin_issuer**: Example: `wlcg`. - - **default_jwt_refresh_lifetime**: _(Optional)_ Integer. Default: `96`. - - **exchange_grant_type**: _(Optional)_ Default: - `urn:ietf:params:oauth:grant-type:token-exchange`. - - **expected_audience**: _(Optional)_ Default: `rucio`. - - **expected_scope**: _(Optional)_ Default: `openid profile`. + - **admin_issuer**: Example: `wlcg`. + - **default_jwt_refresh_lifetime**: _(Optional)_ Integer. + - **exchange_grant_type**: _(Optional)_ + - **expected_audience**: _(Optional)_ + - **expected_scope**: _(Optional)_ - **idpsecrets**: Path of the idpsecrets JSON. Example: - `/opt/rucio/etc/idpsecrets.json`. + `/opt/rucio/etc/idpsecrets.json`. #### **permission** - **policy**: _(Optional)_ Permission policy. Values: `{atlas, belleii, cms, - generic, generic_multi_vo}`. Default: `generic`. + generic, generic_multi_vo}`. #### **policy** - **lfn2pfn_algorithm_default**: _(Optional)_ Default algorithm name for - LFN2PFN translation for this server. Default: `hash`. + LFN2PFN translation for this server. - **package** - **package-*VO*** - **permission**: Same as `policy/permission`. - **schema** - - **scratchdisk_lifetime**: _(Optional)_ Integer. Default: `14`. - - **support**: _(Optional)_ Contact information. - - **support_rucio**: _(Optional)_ Rucio contact information. Default: - `https://github.com/rucio/rucio/issues`. + - **scratchdisk_lifetime**: _(Optional)_ Integer. + - **support**: _(Optional)_ Contact information. + - **support_rucio**: _(Optional)_ Rucio contact information. #### **saml** - **config_path**: Path to the SAML config folder. Example: - `/opt/rucio/lib/rucio/web/ui/common/saml/`. + `/opt/rucio/lib/rucio/web/ui/common/saml/`. #### **test** - **cacert**: Path of the CA certificate for tests. Example: - `/opt/rucio/etc/web/ca.crt` + `/opt/rucio/etc/web/ca.crt` - **usercert** Path of the user certificate for tests. Example: - `/opt/rucio/etc/web/usercert.pem` - - **userkey**: Path of the user certificate key for tests. + `/opt/rucio/etc/web/usercert.pem` + - **userkey**: Path of the user certificate key for tests. #### **trace** - - **broker_virtual_host**: _(Optional)_ No default. - - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. - - **logformat**: _(Optional)_ Formatter of the log. See [logging documentation](https://docs.python.org/3/library/logging.html#logging.Formatter). + - **broker_virtual_host**: _(Optional)_ + - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. + - **logformat**: _(Optional)_ Formatter of the log. See [logging documentation](https://docs.python.org/3/library/logging.html#logging.Formatter). - **loglevel**: _(Optional)_ Set the root logger level to the specified level. Values: `{'NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'}`. See [logging documentation](https://docs.python.org/3/library/logging.html#levels). - Default: `DEBUG`. + - **password**: Password of the `username`. - - **port**: _(Optional)_ Port of the broker. Example: `61013`. - - **topic**: Name of the destination topic. Example: `/topic/rucio.tracer`. - - **tracedir**: _(Optional)_ Path of the directory for traces. Default: - `/var/log/rucio/trace`. - - **username**: Username of the broker. + - **port**: _(Optional)_ Port of the broker. + - **topic**: Name of the destination topic. Example: `/topic/rucio.tracer`. + - **tracedir**: _(Optional)_ Path of the directory for traces. + - **username**: Username of the broker. #### **tracer-kronos** - - **broker_virtual_host**: _(Optional)_ No default. - - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. - - **chunksize**: Integer + - **broker_virtual_host**: _(Optional)_ + - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. + - **chunksize**: Integer - **excluded_usrdns**: Separated by commas. Example: `CN=proxy,CN=Robot: Ganga Robot,CN=722147,CN=gangarbt,OU=Users,OU=Organic - Units,DC=cern,DC=ch`. + Units,DC=cern,DC=ch`. - **password**: _(Optional)_ Password of the `username`. Mandatory if - `use_ssl` is not set. No default. - - **port**: Port of the broker. - - **prefetch_size**: `activemq.prefetchSize`, see [activemq documentation](https://activemq.apache.org/what-is-the-prefetch-limit-for) + `use_ssl` is not set. + - **port**: Port of the broker. + - **prefetch_size**: `activemq.prefetchSize`, see [activemq documentation](https://activemq.apache.org/what-is-the-prefetch-limit-for) - **queue**: The topic or queue to subscribe to. Example: - `/queue/Consumer.kronos.rucio.tracer`. + `/queue/Consumer.kronos.rucio.tracer`. - **reconnect_attempts**: Maximum attempts to reconnect. Integer. Example: - `100`. - - **ssl_cert_file**: _(Optional)_ Path of the certificate file. No - default. Mandatory if `use_ssl` is set. + `100`. + - **ssl_cert_file**: _(Optional)_ Path of the certificate file. Mandatory if `use_ssl` is set. - **ssl_key_file**: _(Optional)_ Path of the certificate key file defined in - `ssl_cert_file`. No default. Mandatory if `use_ssl` is set. + `ssl_cert_file`. Mandatory if `use_ssl` is set. - **subscription_id**: A unique id to represent the subscription. Example: - `rucio-tracer-listener`. - - **use_ssl**: _(Optional)_ Boolean. Default: `True`. + `rucio-tracer-listener`. + - **use_ssl**: _(Optional)_ Boolean. - **username**: _(Optional)_ Username of the broker. Mandatory if `use_ssl` is - not set. No default. + not set. #### **transmogrifier** - **maxdids** #### **upload** - **transfer_timeout**: _(Optional)_ Transfer timeout (seconds, - integer). Default: `360`. + integer). #### **webui** - - **auth_issuer**: _(Optional)_ Mandatory if `auth_type` = `oidc`. No default. + - **auth_issuer**: _(Optional)_ Mandatory if `auth_type` = `oidc`. - **auth_type**: _(Optional)_ Preferred server side config for webui - authentication. Values: `{oidc, None}`. Default: `None`. + authentication. Values: `{oidc, None}`. - **usercert** - **urls**: A CSV specifying urls of Rucio WebUI 2.0 clients. Required for - correctly handling pre-flight CORS requests. + correctly handling pre-flight CORS requests. ## Rucio configuration table * Checked only if Section/Option pair is not in the Configuration File diff --git a/tools/build_documentation.sh b/tools/build_documentation.sh index 533104cd9c3..443e78066f9 100755 --- a/tools/build_documentation.sh +++ b/tools/build_documentation.sh @@ -6,6 +6,8 @@ SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) AUTO_GENERATED=$SCRIPT_DIR/auto_generated DOCS=$SCRIPT_DIR/../docs +echo "Generating dynamic default configuration files..." +"$SCRIPT_DIR"/get_config.py $DOCS "$SCRIPT_DIR"/generate_dynamic_files.sh diff --git a/tools/get_config.py b/tools/get_config.py new file mode 100755 index 00000000000..02f751453cd --- /dev/null +++ b/tools/get_config.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python3 + +import configparser +import json +import sys +config = configparser.RawConfigParser() +config.read("/opt/rucio/etc/rucio.cfg") + +json_config = {} +for key in config.sections(): + json_config[key] = { + sub_key: item for sub_key, item in config.items(key) + } + +json.dump(json_config, open(f"{sys.argv[1]}/default_configurations.json", 'w')) \ No newline at end of file From b51e9f0673f1faa5a2d7b1dbf46d55a5f9c537d7 Mon Sep 17 00:00:00 2001 From: voetberg Date: Fri, 2 Aug 2024 10:51:00 -0500 Subject: [PATCH 2/3] Reformat to make linting happy --- tools/build_documentation.sh | 2 +- tools/get_config.py | 11 +++++------ 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/tools/build_documentation.sh b/tools/build_documentation.sh index 443e78066f9..32e99916bd7 100755 --- a/tools/build_documentation.sh +++ b/tools/build_documentation.sh @@ -7,7 +7,7 @@ AUTO_GENERATED=$SCRIPT_DIR/auto_generated DOCS=$SCRIPT_DIR/../docs echo "Generating dynamic default configuration files..." -"$SCRIPT_DIR"/get_config.py $DOCS +"$SCRIPT_DIR"/get_config.py "$DOCS" "$SCRIPT_DIR"/generate_dynamic_files.sh diff --git a/tools/get_config.py b/tools/get_config.py index 02f751453cd..dda02446b30 100755 --- a/tools/get_config.py +++ b/tools/get_config.py @@ -1,15 +1,14 @@ #!/usr/bin/env python3 import configparser -import json +import json import sys + config = configparser.RawConfigParser() config.read("/opt/rucio/etc/rucio.cfg") json_config = {} -for key in config.sections(): - json_config[key] = { - sub_key: item for sub_key, item in config.items(key) - } +for key in config.sections(): + json_config[key] = {sub_key: item for sub_key, item in config.items(key)} -json.dump(json_config, open(f"{sys.argv[1]}/default_configurations.json", 'w')) \ No newline at end of file +json.dump(json_config, open(f"{sys.argv[1]}/default_configurations.json", "w")) From 1728ccf672ac13dd39354a2607485481fb885120 Mon Sep 17 00:00:00 2001 From: voetberg Date: Fri, 2 Aug 2024 11:06:53 -0500 Subject: [PATCH 3/3] Remove placeholder --Not in code-- and --??-- --- docs/operator/configuration_parameters.mdx | 150 ++++++++++----------- 1 file changed, 73 insertions(+), 77 deletions(-) diff --git a/docs/operator/configuration_parameters.mdx b/docs/operator/configuration_parameters.mdx index f7cc3aa3170..ba835cc03b0 100644 --- a/docs/operator/configuration_parameters.mdx +++ b/docs/operator/configuration_parameters.mdx @@ -39,8 +39,7 @@ Rucio will look for the config in the following locations - ### Options and Defaults #### **accounts** - - **special_accounts** - + - **special_accounts** #### **alembic** - **cfg**: Path to the configuration file (.ini) for Alembic. Example: `/opt/rucio/etc/alembic.ini`. @@ -81,8 +80,8 @@ Rucio will look for the config in the following locations - - **gss_identity**: _(Optional)_ Identity of the Kerberos auth method. - - **saml_email** - - **saml_id** + - **saml_email** + - **saml_id** - **ssh_email**: _(Optional)_ Email of the SSH auth method which identity is specified in `ssh_identity`. @@ -111,20 +110,18 @@ Rucio will look for the config in the following locations - t2_free_space_only_pop_with_network}`. - - **ca_cert**: _(Optional)_ Path of the certificate for Elasticsearch. - - - **elastic_index**: _(Optional)_ Index (database) to use in Elasticsearch. - + - **ca_cert**: _(Optional)_ Path of the certificate for Elasticsearch. + - **elastic_index**: _(Optional)_ Index (database) to use in Elasticsearch. - **elastic_pass**: _(Optional)_ Password of the username defined in `elastic_user` to authenticate to Elasticsearch. - **elastic_url**: URL of Elasticsearch. Example: `http://aianalytics01.cern.ch:9200`. - **elastic_user**: _(Optional)_ Username to authenticate to Elasticsearch. - - **placement_algorithm** + - **placement_algorithm** - **redis_host**: URL of the Redis database. - **redis_port**: Port of the Redis database defined in `redis_host`. #### **c3po-popularity** - - **elastic_url** + - **elastic_url** #### **c3po-site-mapper** - **ddm_url**: URL for DDM. Example: `http://atlas-agis-api.cern.ch/request/ddmendpoint/query/list/?json`. @@ -133,9 +130,9 @@ Rucio will look for the config in the following locations - #### **c3po-workload** - **panda_url**: URL for PanDA. Example: `http://bigpanda.cern.ch/jobs/?category=analysis&jobstatus=running`. - - **window**: Integer. + - **window**: Integer. #### **c3po-network-metrics** - - **prefix**: + - **prefix**: - **redis_host**: URL of the Redis database. - **redis_port**: Port of the Redis database defined in `redis_host`. #### **cache** @@ -143,7 +140,7 @@ Rucio will look for the config in the following locations - - **use_external_cache_for_auth_tokens**: _(Optional)_ if True, use remote cache provider for auth tokens. If False, use a private in-memory cache. #### **common** - - **extract_scope**: _(Optional)_ + - **extract_scope**: _(Optional)_ - **logdir**: Path of the directory for logs. Contains `auditor.log`. - **logformat**: _(Optional)_ Formatter of the log. See [the logging formatter documentation](https://docs.python.org/3/library/logging.html#logging.Formatter) - **loglevel**: _(Optional)_. Set the root logger level to the specified @@ -154,21 +151,20 @@ Rucio will look for the config in the following locations - For `rucio-auditor` default is `WARNING`. - **mailtemplatedir**: _(Optional)_ Path of the folder with mail templates (.tmpl). Example: `/opt/rucio/etc/mail_templates`. - - **multi_vo**: _(Optional)_ + - **multi_vo**: _(Optional)_ #### **conveyor** - **allow_user_oidc_tokens**: _(Optional)_ Boolean. - **bring_online**: Integer, bring online timeout. - - **cacert** + - **cacert** - **cache_time**: _(Optional)_ Integer, expiration time in seconds passed to the dogpile system. - **failover_scheme**: Failover schemes. - - **filter_transfertool**: _(Optional)_ + - **filter_transfertool**: _(Optional)_ - **ftshosts**: URL of the [File Transfer Service (FTS)](https://fts.web.cern.ch/) hosts (separated by commas). Example: - `https://fts3-pilot.cern.ch:8446, https://fts3-pilot.cern.ch:8446`. - - **globus_auth_app**: _(Optional)_ - - **max_time_in_queue**: _(Optional)_ (separated by + `https://fts3-pilot.cern.ch:8446, https://fts3-pilot.cern.ch:8446`. + - **globus_auth_app**: _(Optional)_ + - **max_time_in_queue**: _(Optional)_ (separated by commas) . - **poll_timeout**: Float, timeout. - **queue_mode**: _(Optional)_ Values: `{'strict', 'default'}` @@ -176,12 +172,12 @@ Rucio will look for the config in the following locations - - **request_oidc_scope**: _(Optional)_. - **scheme**: _(Optional)_ Schemes to process (separated by commas). - **submit_timeout**: _(Optional)_ Timeout. - - **transfertool**: _(Optional)_ + - **transfertool**: _(Optional)_ - **transfertype**: _(Optional)_. Values: `{bulk, single}`. - **usercert**: Path to the certificate for the FTS3 implementation of a Rucio transfertool. - - **user_activities**: _(Optional)_ - - **user_transfers**: _(Optional)_ + - **user_activities**: _(Optional)_ + - **user_transfers**: _(Optional)_ - **using_memcache**: _(Optional)_ Boolean. - **webdav_transfer_mode**: _(Optional)_. #### **core** @@ -189,7 +185,7 @@ Rucio will look for the config in the following locations - - **geoip_licence_key**: _(Optional)_ License key for GeoLite2. Get a free licence key at [the signup page](https://www.maxmind.com/en/geolite2/signup). - **use_temp_tables**: _(Optional)_ Use Rucio with temporary table workflows. -#### **client** {#client_config} +#### **client** - **account**: Rucio account. Example: `root`. - **auth_host**: URL of the host of the rucio authentication server. Example: `https://rucio-auth-prod.cern.ch:443`. @@ -233,7 +229,7 @@ Rucio will look for the config in the following locations - #### **credentials** - **gcs**: _(Optional)_ Path of the Google Cloud Storage credentials. - **signature_lifetime**: _(Optional)_ ?. -#### **database** {#db_config} +#### **database** - **default**: Type of the SQL connection. Values: `{mysql, postgresql, sqlite, oracle}`. - **echo**: Enable echo for database logs. Values: `{0 (disable), 1 @@ -250,7 +246,7 @@ Rucio will look for the config in the following locations - See [sqlalchemy documentation](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.poolclass) - **schema**: _(Optional)_ Schema to be applied to a database, if not set in config, try to create automatically. - - **use_threadlocal** + - **use_threadlocal** #### **download** - **transfer_speed_timeout**: _(Optional)_ Minimum allowed average transfer speed (in KBps). Default: `500`. Used to dynamically compute the timeout if @@ -275,10 +271,10 @@ Rucio will look for the config in the following locations - - **attr_sync_method**: _(Optional)_ Values: `{append, edit, hard}`. - **rse_sync_method**: _(Optional)_ Values: `{append, edit, hard}`. #### **injector** - - **adler32** - - **bytes** - - **file** - - **md5** + - **adler32** + - **bytes** + - **file** + - **md5** #### **lifetime** - **directory**: _(Optional)_ Path to the policies directory with JSON files named `config_DTYPE.json`, where `DTYPE` is a value in `{data, mc, valid, @@ -286,7 +282,7 @@ Rucio will look for the config in the following locations - #### **logging** - ***CFG_OPTION***: _(Optional)_ ?. #### **messaging-cache** - - **account** + - **account** - **broker_virtual_host**: _(Optional)_ ? - **brokers**: Default message broker name for `rucio-cache-client`. Ignored if `rucio-cache-client` executed with `--broker`. @@ -300,7 +296,7 @@ Rucio will look for the config in the following locations - - **ssl_key_file**: Default certificate key file for `rucio-cache-client`. Ignored if `rucio-cache-client` executed with `--certificate-key`. - - **voname** + - **voname** #### **messaging-fts3** - **broker_virtual_host**: _(Optional)_ ?. - **brokers**: Brokers separated by commas. Example: `dashb-test-mb.cern.ch`. @@ -309,21 +305,21 @@ Rucio will look for the config in the following locations - - **nonssl_port**: _(Optional)_ Port of the broker if `use_ssl` is not set. - **password**: _(Optional)_ Password of the `username`. Only used if `use_ssl` is not set. - - **port** + - **port** - **ssl_cert_file**: _(Optional)_ Path of the certificate file. - **ssl_key_file**: _(Optional)_ Path of the certificate key file defined in `ssl_cert_file`. - **use_ssl**: _(Optional)_ Boolean. - **username**: _(Optional)_ Username of the broker. Only used if `username` is not set. - - **voname** + - **voname** #### **messaging-hermes** - - **broker_virtual_host**: _(Optional)_ + - **broker_virtual_host**: _(Optional)_ - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. - **destination**: Name of the destination topic. Example: `/topic/rucio.events`. - - **email_from**: Example: `Rucio `. - - **email_test**: Example: `spamspamspam@cern.ch`. + - **email_from**: Example: `Rucio `. + - **email_test**: Example: `spamspamspam@cern.ch`. - **nonssl_port**: _(Optional)_ Port of the broker if `use_ssl` is not set. - **password**: _(Optional)_ Password of the `username`. Mandatory if `use_ssl` is not set. @@ -334,7 +330,7 @@ Rucio will look for the config in the following locations - - **use_ssl**: _(Optional)_ Boolean. - **username**: _(Optional)_ Username of the broker. Mandatory if `use_ssl` is not set. - - **voname** + - **voname** #### **metadata** - **plugins**: _(Optional)_ Metadata handler modules separated by commas. @@ -347,9 +343,9 @@ Rucio will look for the config in the following locations - application or environment. - **metrics_port**: _(Optional)_ Port of Prometheus Python Client. #### **nagios** - - **fts_servers** - - **proxy** - - **rfcproxy** + - **fts_servers** + - **proxy** + - **rfcproxy** #### **nongrid-trace** - **broker_virtual_host**: _(Optional)_ ?. - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. @@ -366,9 +362,9 @@ Rucio will look for the config in the following locations - #### **oidc** - **admin_issuer**: Example: `wlcg`. - **default_jwt_refresh_lifetime**: _(Optional)_ Integer. - - **exchange_grant_type**: _(Optional)_ - - **expected_audience**: _(Optional)_ - - **expected_scope**: _(Optional)_ + - **exchange_grant_type**: _(Optional) + - **expected_audience**: _(Optional)_ + - **expected_scope**: _(Optional)_ - **idpsecrets**: Path of the idpsecrets JSON. Example: `/opt/rucio/etc/idpsecrets.json`. #### **permission** @@ -377,10 +373,10 @@ Rucio will look for the config in the following locations - #### **policy** - **lfn2pfn_algorithm_default**: _(Optional)_ Default algorithm name for LFN2PFN translation for this server. - - **package** - - **package-*VO*** + - **package** + - **package-*VO*** - **permission**: Same as `policy/permission`. - - **schema** + - **schema** - **scratchdisk_lifetime**: _(Optional)_ Integer. - **support**: _(Optional)_ Contact information. - **support_rucio**: _(Optional)_ Rucio contact information. @@ -394,7 +390,7 @@ Rucio will look for the config in the following locations - `/opt/rucio/etc/web/usercert.pem` - **userkey**: Path of the user certificate key for tests. #### **trace** - - **broker_virtual_host**: _(Optional)_ + - **broker_virtual_host**: _(Optional)_ - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. - **logformat**: _(Optional)_ Formatter of the log. See [logging documentation](https://docs.python.org/3/library/logging.html#logging.Formatter). - **loglevel**: _(Optional)_ Set the root logger level to the specified level. @@ -407,7 +403,7 @@ Rucio will look for the config in the following locations - - **tracedir**: _(Optional)_ Path of the directory for traces. - **username**: Username of the broker. #### **tracer-kronos** - - **broker_virtual_host**: _(Optional)_ + - **broker_virtual_host**: _(Optional)_ - **brokers**: Brokers separated by commas. Example: `atlas-test-mb.cern.ch`. - **chunksize**: Integer - **excluded_usrdns**: Separated by commas. Example: @@ -430,7 +426,7 @@ Rucio will look for the config in the following locations - - **username**: _(Optional)_ Username of the broker. Mandatory if `use_ssl` is not set. #### **transmogrifier** - - **maxdids** + - **maxdids** #### **upload** - **transfer_timeout**: _(Optional)_ Transfer timeout (seconds, integer). @@ -438,7 +434,7 @@ Rucio will look for the config in the following locations - - **auth_issuer**: _(Optional)_ Mandatory if `auth_type` = `oidc`. - **auth_type**: _(Optional)_ Preferred server side config for webui authentication. Values: `{oidc, None}`. - - **usercert** + - **usercert** - **urls**: A CSV specifying urls of Rucio WebUI 2.0 clients. Required for correctly handling pre-flight CORS requests. @@ -459,19 +455,19 @@ a setting already specified in the Configuration File. #### **automatix** - **account**: _(Optional)_ Account to use. Default: `root`. - **dataset_lifetime**: _(Optional)_. Default: `0`. - - **did_prefix**: _(Optional)_ Default: ` `. + - **did_prefix**: _(Optional)_ Default: ` `. - ***DIDTYPE*\_pattern**: _(Optional)_ Separated by `separator` char. No default. - **rses**: Separated by commas. - **scope**: _(Optional)_ Default: `test`. - **separator**: _(Optional)_ Separator char. No default. - - **set_metadata**: _(Optional)_ Default: `True`. + - **set_metadata**: _(Optional)_ Default: `True`. - **sites**: Separated by commas _(to be deprecated, please use `rses`)_. - **sleep_time**: _(Optional)_ Integer. Default: `30`. #### **clientcachemap** - - ***client_location['site']*** + - ***client_location['site']*** #### **conveyor** - - **activity-source-strategy**: _(Optional)_ Default: `{}`. + - **activity-source-strategy**: _(Optional)_ Default: `{}`. - **default-source-strategy**: _(Optional)_ Default: `orderly`. #### **hermes** - **services_list**: List of services separated by commas. Values: `{activemq, @@ -495,7 +491,7 @@ a setting already specified in the Configuration File. `None`. - **nb_workers_by_hostname**: _(Optional)_ Integer. Default: `5`. #### **root-proxy-internal** - - ***client_location['site']*** + - ***client_location['site']*** #### **rules** - **apply_rule_max_partition_size**: _(Optional)_ Integer . Default: `2000`. - **force_epoch_when_detach**: _(Optional)_ Purge setting of the @@ -530,36 +526,36 @@ and only contains information about the specific RSE's for the Rucio instance. Read more about RSEs [here](../started/concepts/rucio_storage_element.md) and how to set them up [here](./configuration.md#creating-new-rses). -- **associated_sites**: Separated by commas. Default: `None`. +- **associated_sites**: Separated by commas. Default: `None`. - **auto_approve_bytes**: Auto Approve Limit. Example: `500GB`. No default. -- **auto_approve_files**: No default. +- **auto_approve_files**: > No default. - **available_for_multihop**: Boolean. If True, allow to use this RSE as an intermediate hop in a multi-hop transfer. Default: `False`. -- **block_manual_approval**: Boolean. Default: `False`. -- **country**: No default. -- **greedyDeletion**: Boolean. Default: `False`. -- **hop_penalty**: Integer. Cost of passing via this RSE in multi-hop +- **block_manual_approval**: Boolean. Default: `False`. +- **country**: No default. +- **greedyDeletion**: Boolean. Default: `False`. +- **hop_penalty**: Integer. Cost of passing via this RSE in multi-hop transfers. Overrides the global `transfers/hop_penalty` configuration value for this particular RSE. Has only meaning if `available_for_multihop` attribute is True on the RSE. No default value. -- **istape**: Boolean. Default: `False`. -- **mock**: Boolean. Default: `False`. -- **naming_convention**: Default: `None`. -- **physgroup**: Default: ` `. -- **quota_approvers**: Separated by commas. Default: `None`. -- **restricted_read** Boolean. If True, only allow transfers +- **istape**: Boolean. Default: `False`. +- **mock**: Boolean. Default: `False`. +- **naming_convention**: Default: `None`. +- **physgroup**: Default: ` `. +- **quota_approvers**: Separated by commas. Default: `None`. +- **restricted_read** Boolean. If True, only allow transfers from this RSE if started by an account with admin privileges. Default: `False` -- **restricted_write** Boolean. Same as `restricted_read`, but for +- **restricted_write** Boolean. Same as `restricted_read`, but for transfers towards this RSE. Default: `False` -- **rule_approvers**: Separated by commas. No default. -- **rule_deleters**: Separated by commas. No default. -- **site** -- **skip_upload_stat**: Boolean. Default: `False`. -- **source_for_total_space**: Default: `storage`. -- **source_for_used_space**: Default: `storage`. -- **staging_buffer** +- **rule_approvers**: Separated by commas. No default. +- **rule_deleters**: Separated by commas. No default. +- **site** +- **skip_upload_stat**: Boolean. Default: `False`. +- **source_for_total_space**: Default: `storage`. +- **source_for_used_space**: Default: `storage`. +- **staging_buffer** - **tier**: Datacenter TIER. Integer (from 1 to 4). No default. -- **type**: Values: `{LOCALGROUPDISK, LOCALGROUPTAPE, GROUPDISK, +- **type**: Values: `{LOCALGROUPDISK, LOCALGROUPTAPE, GROUPDISK, SCRATCHDISK, MOCK, TEST, DATADISK}`. Default: ` `. - **verify_checksum**: Boolean. No default.