Skip to content

Commit

Permalink
Merge pull request #458 from haiwen/13.0_opt_structures
Browse files Browse the repository at this point in the history 
13.0 opt structures
  • Loading branch information
@freeplant
freeplant authored Feb 24, 2025
2 parents 8fb7d6e + c737ef2 commit e81551c
Show file tree
Hide file tree
Showing 4 changed files with 41 additions and 49 deletions.
80 changes: 37 additions & 43 deletions manual/extension/metadata-server.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,77 +5,70 @@ Metadata server aims to provide metadata management for your libraries, so as to
## Deployment

!!! note "Prerequisites"
The startup of Metadata server requires using ***Redis*** as the cached server (it should be the default cached server in Seafile 13.0). So you must deploy *Redis* for Seafile, then modify [`seafile.conf`](../config/seafile-conf.md#cache-pro-edition-only), [`seahub_settings.py`](https://docs.djangoproject.com/en/4.2/topics/cache/#redis) and [`seafevents.conf`](../config/seafevents-conf.md) to enable it before deploying metadata server.
The startup of Metadata server requires using ***Redis*** as the cache server (it should be the default cache server in Seafile 13.0). So you must deploy *Redis* for Seafile, then modify [`seafile.conf`](../config/seafile-conf.md#cache-pro-edition-only), [`seahub_settings.py`](https://docs.djangoproject.com/en/4.2/topics/cache/#redis) and [`seafevents.conf`](../config/seafevents-conf.md) to enable it before deploying metadata server.

### Download YML file
### Download docker-compose file

Please download YML file **in your `seafile-server.yml` directory** by following command:
Please download the file by following command:

=== "Deploy in the same machine with Seafile"

!!! note
You have to download this file to the directory same as `seafile-server.yml`

```sh
wget https://manual.seafile.com/13.0/repo/docker/md-server.yml
```

=== "Standalone"

!!! note
For standalone deployment, the metadata server only supports Seafile using the storage backend such as **S3**.

```sh
wget https://manual.seafile.com/13.0/repo/docker/metadata-server/md-server.yml
wget -O .env https://manual.seafile.com/13.0/repo/docker/metadata-server/env
```

!!! warning "Important for standalone deployment"
Starting from Seafile 13.0, metadata server supports standalone deployment, but it still needs to maintain access to the original Seafile data directory. **Please make sure that the machine where metadata server is deployed can access the directory normally**.

### Modify `.env`

!!! success "Faster configuration"
Metadata-server docker we use an one-time configuration file, which will be generated directly through the environment variable, eliminating the need for you to repeatedly write configuration files (i.e., **no `md-server.conf`**), which will significantly improve your deployment efficiency.

Normally, you only need to set a `METADATA_SERVER_SECRET_KEY` for the metadata server in `.env`, and the metadata of the library will be stored locally (default is `opt/md-data`):

```env
METADATA_SERVER_SECRET_KEY=<your metadata server secret key>
MD_DATA=/opt/md-data
```

!!! warning "About `METADATA_SERVER_SECRET_KEY`"
It is used to Seafile connect to metadata server, which can be generated by `pwgen -s 40 1`. However, we **do not recommend** to use the same as the original `JWT_PRIVATE_KEY` in `.env` **for security reasons**.

The following table is all the related environment variables with metadata-server:
By default, you don't need to add additional variables to your `.env` (except for standalone deployment) to get the metadata server started, because it will read the exact same configuration as the Seafile server (including `JWT_PRIVATE_KEY` ) and keep the repository metadata locally (default `/opt/md-server`). The following table is all the related environment variables with metadata-server:

| Variables | Description | Required |
| --- | --- | --- |
| `METADATA_SERVER_SECRET_KEY` | The JWT key used to connect with Seafile server | **Required** |
| `MD_STORAGE_TYPE` | The type of backend storage. Options: `file` (local storage), `s3`, `oss`. | Optional, default `file` |
| `MD_DATA` | Directory where local data (only valid in `MD_STORAGE_TYPE=file`) and cache are located. | Optional, default `/opt/md-data` |
| `SEAFILE_VOLUME` | Directory for Seafile data | Optional, default `/opt/seafile-data` |
| `JWT_PRIVATE_KEY` | The JWT key used to connect with Seafile server | **Required** |
| `MD_DATA` | Directory where metadata and cache are located. | Optional, default `/opt/md-data` |
| `MD_MAX_CACHE_SIZE` | The maximum cache size. | Optional, default `1GB` |
| `MD_S3_HOST` | Host of s3 backend. | Optional |
| `MD_S3_AWS_REGION` | Region of *AWS* s3 backend. | Optional |
| `MD_S3_USE_HTTPS` | Use https connecting to S3 backend. | Optional, default `true` |
| `MD_S3_BUCKET` | Name of S3 bucket for storaging metadata. | **Required** when `MD_STORAGE_TYPE=s3` |
| `MD_S3_PATH_STYLE_REQUEST` | S3 backend use path style request. | Optional, default `false` |
| `MD_S3_KEY_ID` | S3 backend authorization key ID. | **Required** when `MD_STORAGE_TYPE=s3` |
| `MD_S3_KEY` | S3 backend authorization key secret. | **Required** when `MD_STORAGE_TYPE=s3` |
| `MD_S3_USE_V4_SIGNATURE` | Use V4 signature to S3 storage backend. | Optional, default `true` |
| `MD_S3_SSE_C_KEY` | S3 SSE-C key. | Optional |
| `MD_OSS_HOST` | OSS backend host. | Optional |
| `MD_OSS_REGION` | OSS backend region. | Optional |
| `MD_OSS_BUCKET` | Name of OSS bucket for storaging metadata. | **Required** when `MD_STORAGE_TYPE=oss` |
| `MD_OSS_KEY_ID` | OSS backend authorization key ID. | **Required** when `MD_STORAGE_TYPE=oss` |
| `MD_OSS_KEY` | OSS backend authorization key secret. | **Required** when `MD_STORAGE_TYPE=oss` |
| `MD_STORAGE_TYPE` | The type of Seafile backend storage. Options: `file` (local storage), `s3`, `oss`. | Optional, default `file` and `s3` in deploying metadata server in the same machine with Seafile and standalone respective |
| `REDIS_HOST` | Your *Redis* service host. | Optional, default `redis` |
| `REDIS_PORT` | Your *Redis* service port. | Optional, default `6379` |
| `REDIS_PASSWORD` | Your *Redis* access password. | Optional |


!!! tip "More descriptions about *Optional* and *Required* in above table"
- In the table above, although many items are marked as *Required*, some of them are only required when the storage backend specified by `MD_STORAGE_TYPE` is used:
- `MD_S3_xxx` only valid in `MD_STORAGE_TYPE=s3`
- `MD_OSS_xxx` only valid in `MD_STORAGE_TYPE=oss`
And here is other optional values according to your `MD_STORAGE_TYPE` setting:

- `MD_STORAGE_TYPE=file` (only for deploying the metadata server in the same machine with Seafile)

| Variables | Description | Required |
| --- | --- | --- |
| `SEAFILE_VOLUME` | Directory for Seafile data | Optional, default `/opt/seafile-data` |


- `MD_STORAGE_TYPE=s3`

- If `MD_STORAGE_TYPE` is `s3` or `oss`, although `MD_xxx_HOST` and `MD_xxx_REGION` are marked as ***Optional***, when you use one of these two storage backends, **you must specify at least one of `MD_xxx_HOST` and `MD_xxx_REGION`, otherwise the service will fail to start**.
| Variables | Description | Required |
| --- | --- | --- |
| `MD_S3_HOST` | Host of s3 backend. | Optional |
| `MD_S3_AWS_REGION` | Region of *AWS* s3 backend. | Optional |
| `MD_S3_USE_HTTPS` | Use https connecting to S3 backend. | Optional, default `true` |
| `MD_S3_BUCKET` | Name of S3 bucket for storaging metadata. | **Required** |
| `MD_S3_PATH_STYLE_REQUEST` | S3 backend use path style request. | Optional, default `false` |
| `MD_S3_KEY_ID` | S3 backend authorization key ID. | **Required** |
| `MD_S3_KEY` | S3 backend authorization key secret. | **Required** |
| `MD_S3_USE_V4_SIGNATURE` | Use V4 signature to S3 storage backend. | Optional, default `true` |
| `MD_S3_SSE_C_KEY` | S3 SSE-C key. | Optional |

### Modify `seahub_settings.py`

Expand All @@ -84,13 +77,13 @@ To enable metadata server in Seafile, please add the following field in your `se
=== "Deploy in the same machine with Seafile"
```py
ENABLE_METADATA_MANAGEMENT = True
METADATA_SERVER_SECRET_KEY = '<your metadata server secret key> '
METADATA_SERVER_SECRET_KEY = '<your JWT key> '
METADATA_SERVER_URL = 'http://metadata-server:8084'
```
=== "Standalone"
```py
ENABLE_METADATA_MANAGEMENT = True
METADATA_SERVER_SECRET_KEY = '<your metadata server secret key> '
METADATA_SERVER_SECRET_KEY = '<your JWT key> '
METADATA_SERVER_URL = 'http://<your metadata-server host>:8084'
```

Expand All @@ -99,6 +92,7 @@ To enable metadata server in Seafile, please add the following field in your `se
You can use following command to start metadata server (and the Seafile service also have to restart):

```sh
docker compose down
docker compose up -d
```

Expand Down
2 changes: 1 addition & 1 deletion manual/repo/docker/md-server.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ services:
#ports:
# - ${MD_PORT:-8084}:${MD_PORT:-8084}
environment:
- JWT_PRIVATE_KEY=${METADATA_SERVER_SECRET_KEY:?Variable is not set or empty}
- JWT_PRIVATE_KEY=${JWT_PRIVATE_KEY:?Variable is not set or empty}
- SEAFILE_MYSQL_DB_HOST=${SEAFILE_MYSQL_DB_HOST:-db}
- SEAFILE_MYSQL_DB_PORT=${SEAFILE_MYSQL_DB_PORT:-3306}
- SEAFILE_MYSQL_DB_USER=${SEAFILE_MYSQL_DB_USER:-seafile}
Expand Down
7 changes: 3 additions & 4 deletions manual/repo/docker/metadata-server/env
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,8 @@ JWT_PRIVATE_KEY=
MD_PORT=8084 # not recommond to modify
MD_LOG_LEVEL=info
MD_MAX_CACHE_SIZE=1GB
MD_STORAGE_TYPE=file
MD_STORAGE_TYPE=s3
MD_DATA=/opt/md-data
SEAFILE_VOLUME=/opt/seafile-data

# for MySQL
SEAFILE_MYSQL_DB_HOST=
Expand All @@ -16,7 +15,7 @@ SEAFILE_MYSQL_DB_PASSWORD=
SEAFILE_MYSQL_DB_SEAFILE_DB_NAME=seafile_db
SEAFILE_LOG_TO_STDOUT=false

#
# for s3 configurations
MD_S3_HOST=
MD_S3_AWS_REGION=
MD_S3_USE_HTTPS=true
Expand All @@ -27,7 +26,7 @@ MD_S3_KEY=
MD_S3_USE_V4_SIGNATURE=true
MD_S3_SSE_C_KEY=

#
# for oss configurations
MD_OSS_HOST=
MD_OSS_REGION=
MD_OSS_BUCKET=
Expand Down
1 change: 0 additions & 1 deletion manual/repo/docker/metadata-server/md-server.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ services:
container_name: metadata-server
volumes:
- ${MD_DATA:-/opt/md-data}:/data
- ${SEAFILE_VOLUME:-/opt/seafile-data}/seafile:/opt/seafile
ports:
- ${MD_PORT:-8084}:${MD_PORT:-8084}
environment:
Expand Down

0 comments on commit e81551c

Please sign in to comment.