Skip to content

Commit

Permalink
Merge pull request #221 from icey-yu/doc
Browse files Browse the repository at this point in the history
feat: cluster deployment doc
  • Loading branch information
skiffer-git authored Dec 3, 2024
2 parents 40e9c49 + 3481177 commit 3741408
Show file tree
Hide file tree
Showing 11 changed files with 211 additions and 26 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
95 changes: 95 additions & 0 deletions docs/guides/gettingStarted/cluster.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
title: '集群部署说明'
sidebar_position: 6
---



## 集群部署说明

`open-im-server`支持集群部署。以下是集群部署的步骤:

1. 修改`kafka``minio``mongodb``etcd(discovery.yml)``redis`中的地址,将其配置到正确的组件地址中。**保证连接组件的各个端口可访问。**

2. 修改各个`rpc`组件的`registerIP`为设置部署`etcd`的服务器可访问的`ip`地址,**并保证各个端口可被访问**。如需启用`prometheus`,还需要保证各个组件的`prometheus.port`端口可被访问。

![rpc0](./cluster.assets/rpc0.png)

部署`promtheus`的机器需要修改`prometheus.yml`中的`targets`

![prometheus0](./cluster.assets/prometheus0.png)

4. 可在`open-im-server/start-config.yml`中修改各个组件数量。

5. 修改`kafka`广播给主机的端口。如果使用`open-im-server`中的`docker-compose.yml`部署,修改`service.kafka.environment.KAFKA_CFG_ADVERTISED_LISTENERS`中的`EXTERNAL`为访问`kafka`组件的地址。
例如:`KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,EXTERNAL://192.168.2.36:19094`

6. 如果通过集群模式部署`redis`,则需将`redis.yml`中的`clusterMode`设置为`true`

7. 如果部署了多个`api`或者`WebSocket`监听端口,可使用`nginx`方便快捷地实现负载均衡,参考配置如下:
```yaml
# nginx.conf

events {
worker_connections 1024;
}

http {
upstream websocket_cluster {
server 192.168.2.10:10101; # WebSocket address
server 192.168.2.11:10001;
}

upstream api_cluster {
server 192.168.2.10:10102; # api address
server 192.168.2.11:10002;
}

# WebSocket
server {
listen 10001;

server_name _;

location / {
proxy_pass http://websocket_cluster;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}

# API
server {
listen 10002;

server_name _;

location / {
proxy_pass http://api_cluster;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
}

```

如果决定使用`docker`启动`nginx`,可参考以下命令(linux系统):

```shell
docker run --name nginx -p 10001:10001 -p 10002:10002 -v $(pwd)/nginx-config:/etc/nginx/conf.d -v $(pwd)/nginx.conf:/etc/nginx/nginx.conf -d nginx
```

如使用不同系统修改挂载的目录即可。

8. 启动服务。

## **常见问题/注意事项**

1. 多台机器部署需要保证时钟一致,服务才可正常运行。例如`token`的签发允许各个机器的时钟误差在`5s`以内。
2. 组件端口无法访问:通过回环地址检测组件启动是否正常,若回环地址可访问,则检查是否被防火墙规则过滤。
14 changes: 7 additions & 7 deletions docs/restapi/commonConfigs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ title: 常用配置项修改说明

## 常用配置项

| 配置字段名称 | 配置项所在文件 | 配置项说明 | 修改说明 |
| ---------------------------- | ---------------------- | ------------------------------- | -------------------------------------- |
| `multiLoginPolicy` | `share.yml` | 客户端多端互踢及在线策略 | 0: 默认可多设备多端同时登陆, 1: 全端可同时登录但同端只能保持一个设备在线, 5: PC 端可同时在线, 其他端仅允许一个设备登录 |
| `secret` | `share.yml` | 服务端密钥,用于生成 adminToken | 默认为 `openIM123`,可更改为任意字符串 |
| `tokenPolicy.expire` | `openim-rpc-auth.yml` | token 过期时间设置 | expire 的单位为天,默认为 90 |
| `friendVerify` | `openim-rpc-msg.yml` | 发送消息是否需要好友关系 | 默认为 false |
| `enableHistoryForNewMembers` | `openim-rpc-group.yml` | 新入群用户是否可以查看历史消息 | 默认为 true |
| 配置字段名称 | 配置项所在文件 | 配置项说明 | 修改说明 |
| ---------------------------- | ---------------------- | ------------------------------- | ------------------------------------------------------------ |
| `multiLoginPolicy` | `share.yml` | 客户端多端互踢及在线策略 | 0: 默认可多设备多端同时登陆1: 全端可同时登录但同端只能保持一个设备在线;4:各个平台类型允许一个设备登录(目前支持Mobile、MiniWeb、Web、PC、Pad五种平台类型);5: PC 端可同时在线, 其他端仅允许一个设备登录 |
| `secret` | `share.yml` | 服务端密钥,用于生成 adminToken | 默认为 `openIM123`,可更改为任意字符串 |
| `tokenPolicy.expire` | `openim-rpc-auth.yml` | token 过期时间设置 | expire 的单位为天,默认为 90 |
| `friendVerify` | `openim-rpc-msg.yml` | 发送消息是否需要好友关系 | 默认为 false |
| `enableHistoryForNewMembers` | `openim-rpc-group.yml` | 新入群用户是否可以查看历史消息 | 默认为 true |
| `object` | `openim-rpc-third.yml` | 选择不同的对象存储 | 参考 [Object](#object) |

## Object
Expand Down
Binary file added docs/restapi/webhooks/assets/config.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/restapi/webhooks/assets/config.png
Binary file not shown.
6 changes: 2 additions & 4 deletions docs/restapi/webhooks/example.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,10 @@ hide_title: true

## 1. 修改配置文件

参照下面的模板修改 open-im-server 中的 `config/webhooks.yml` 配置。

![PC Web Interface](./assets/config.png)
将回调的`enable`设置为`true`,表示启动该回调。如需要添加新的回调参照 open-im-server 中的 `config/webhooks.yml` 配置,并在代码层面相应修改。

> **提示**:
>
> - `url` 为回调 URL。
> -`afterSendSingleMsg.enable` 设置为 `true` 时,启用该回调。
Expand Down Expand Up @@ -88,4 +87,3 @@ func (m *ChatApi) CallbackExample(c *gin.Context) {

![PC Web Interface](./assets/result.png)


28 changes: 13 additions & 15 deletions docs/restapi/webhooks/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,17 @@ hide_title: true
## ⚙️ 配置说明

```yaml
callback:
url: http://127.0.0.1:10006/callbackExample
beforeCreateGroup:
enable: true
timeout: 5
failedContinue: true

url: http://127.0.0.1:10006/callbackExample
beforeSendSingleMsg:
enable: false
timeout: 5
failedContinue: true
```
- url:APP业务服务器 callback 地址,支持HTTP/HTTPS
- enable:是否开启此回调,true 为开启
- url:APP业务服务器 `callback` 地址,支持HTTP/HTTPS
- enable:是否开启此回调,`true` 为开启
- timeout:响应超时时间(秒)
- failedContinue:超时或者API失败后是否继续执行流程,一般只对before类型回调有效,true 为继续执行
- failedContinue:超时或者API失败后是否继续执行流程,一般只对`before`类型回调有效,`true` 为继续执行

## 🔄 回调协议

Expand All @@ -45,11 +43,11 @@ callback:

### 请求参数说明

| 参数 | 说明 |
| ----------------- | ---------------------------------------- |
| {WEBHOOK_ADDRESS} | webhooks.yml 中的 url 字段,域名或主机名 |
| callbackCommand | 回调命令,可在各个回调方法说明中具体查看 |
| contenttype | 请求入参,为`json`类型 |
| 参数 | 说明 |
| ----------------- | ------------------------------------------------------------ |
| {WEBHOOK_ADDRESS} | webhooks.yml 中的 url 字段,域名或主机名,例如`http://127.0.0.1:10006/callbackExample` |
| callbackCommand | 回调命令,可在各个回调方法说明中具体查看 |
| contenttype | 请求入参,为`json`类型 |

### Header

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
title: 'Cluster Deployment Instructions'
sidebar_position: 6
---

## Cluster Deployment Instructions

`open-im-server` supports cluster deployment. The following are the steps for cluster deployment:

1. Modify the addresses in `kafka`, `minio`, `mongodb`, `etcd(discovery.yml)`, and `redis`, configuring them to the correct component addresses. **Ensure that all ports of the connected components are accessible.**

2. Modify the `registerIP` of each `rpc` component to the IP address accessible by the server where `etcd` is deployed, **and ensure that all ports are accessible**. If you need to enable `prometheus`, also ensure that the `prometheus.port` of each component is accessible.

![image-20241203155731369](./cluster.assets/rpc0.png)

The machine deploying `prometheus` needs to modify the `targets` in `prometheus.yml`.

![prometheus0](./cluster.assets/prometheus0.png)

3. You can modify the number of each component in `open-im-server/start-config.yml`.

4. Modify the port that `kafka` broadcasts to the host. If deploying using `docker-compose.yml` in `open-im-server`, change the `EXTERNAL` in `service.kafka.environment.KAFKA_CFG_ADVERTISED_LISTENERS` to the address accessing the `kafka` component.

For example: `KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,EXTERNAL://192.168.2.36:19094`.

5. If deploying `redis` in cluster mode, set `clusterMode` in `redis.yml` to `true`.

7. If you deploy multiple `api` or `WebSocket` listening ports, you can use `nginx` to easily implement load balancing. The configuration reference is as follows:
```yaml
# nginx.conf

events {
worker_connections 1024;
}

http {
upstream websocket_cluster {
server 192.168.2.10:10101; # WebSocket address
server 192.168.2.11:10001;
}

upstream api_cluster {
server 192.168.2.10:10102; # API address
server 192.168.2.11:10002;
}

# WebSocket
server {
listen 10001;

server_name _;

location / {
proxy_pass http://websocket_cluster;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}

# API
server {
listen 10002;

server_name _;

location / {
proxy_pass http://api_cluster;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
}
```

If you decide to start `nginx` using `docker`, you can refer to the following command (Linux system):

```shell
docker run --name nginx -p 10001:10001 -p 10002:10002 -v $(pwd)/nginx-config:/etc/nginx/conf.d -v $(pwd)/nginx.conf:/etc/nginx/nginx.conf -d nginx
```

If using a different system, adjust the mounted directories accordingly.

7. Start the services.

## **Common Issues/Notes**

1. When deploying on multiple machines, ensure clock synchronization for the services to run properly. For example, the issuance of `tokens` allows each machine's clock to have a time difference within `5s`.

2. Component ports are inaccessible: Check whether the component starts normally via the loopback address. If the loopback address is accessible, then check if firewall rules are blocking access.

0 comments on commit 3741408

Please sign in to comment.