-
Notifications
You must be signed in to change notification settings - Fork 30
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #221 from icey-yu/doc
feat: cluster deployment doc
- Loading branch information
Showing
11 changed files
with
211 additions
and
26 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,95 @@ | ||
--- | ||
title: '集群部署说明' | ||
sidebar_position: 6 | ||
--- | ||
|
||
|
||
|
||
## 集群部署说明 | ||
|
||
`open-im-server`支持集群部署。以下是集群部署的步骤: | ||
|
||
1. 修改`kafka`、`minio`、`mongodb`、`etcd(discovery.yml)`、`redis`中的地址,将其配置到正确的组件地址中。**保证连接组件的各个端口可访问。** | ||
|
||
2. 修改各个`rpc`组件的`registerIP`为设置部署`etcd`的服务器可访问的`ip`地址,**并保证各个端口可被访问**。如需启用`prometheus`,还需要保证各个组件的`prometheus.port`端口可被访问。 | ||
|
||
![rpc0](./cluster.assets/rpc0.png) | ||
|
||
部署`promtheus`的机器需要修改`prometheus.yml`中的`targets`。 | ||
|
||
![prometheus0](./cluster.assets/prometheus0.png) | ||
|
||
4. 可在`open-im-server/start-config.yml`中修改各个组件数量。 | ||
|
||
5. 修改`kafka`广播给主机的端口。如果使用`open-im-server`中的`docker-compose.yml`部署,修改`service.kafka.environment.KAFKA_CFG_ADVERTISED_LISTENERS`中的`EXTERNAL`为访问`kafka`组件的地址。 | ||
例如:`KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,EXTERNAL://192.168.2.36:19094`。 | ||
|
||
6. 如果通过集群模式部署`redis`,则需将`redis.yml`中的`clusterMode`设置为`true`。 | ||
|
||
7. 如果部署了多个`api`或者`WebSocket`监听端口,可使用`nginx`方便快捷地实现负载均衡,参考配置如下: | ||
```yaml | ||
# nginx.conf | ||
|
||
events { | ||
worker_connections 1024; | ||
} | ||
|
||
http { | ||
upstream websocket_cluster { | ||
server 192.168.2.10:10101; # WebSocket address | ||
server 192.168.2.11:10001; | ||
} | ||
|
||
upstream api_cluster { | ||
server 192.168.2.10:10102; # api address | ||
server 192.168.2.11:10002; | ||
} | ||
|
||
# WebSocket | ||
server { | ||
listen 10001; | ||
|
||
server_name _; | ||
|
||
location / { | ||
proxy_pass http://websocket_cluster; | ||
proxy_http_version 1.1; | ||
proxy_set_header Upgrade $http_upgrade; | ||
proxy_set_header Connection 'upgrade'; | ||
proxy_set_header Host $host; | ||
proxy_cache_bypass $http_upgrade; | ||
} | ||
} | ||
|
||
# API | ||
server { | ||
listen 10002; | ||
|
||
server_name _; | ||
|
||
location / { | ||
proxy_pass http://api_cluster; | ||
proxy_set_header Host $host; | ||
proxy_set_header X-Real-IP $remote_addr; | ||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | ||
proxy_set_header X-Forwarded-Proto $scheme; | ||
} | ||
} | ||
} | ||
|
||
``` | ||
|
||
如果决定使用`docker`启动`nginx`,可参考以下命令(linux系统): | ||
|
||
```shell | ||
docker run --name nginx -p 10001:10001 -p 10002:10002 -v $(pwd)/nginx-config:/etc/nginx/conf.d -v $(pwd)/nginx.conf:/etc/nginx/nginx.conf -d nginx | ||
``` | ||
|
||
如使用不同系统修改挂载的目录即可。 | ||
|
||
8. 启动服务。 | ||
|
||
## **常见问题/注意事项** | ||
|
||
1. 多台机器部署需要保证时钟一致,服务才可正常运行。例如`token`的签发允许各个机器的时钟误差在`5s`以内。 | ||
2. 组件端口无法访问:通过回环地址检测组件启动是否正常,若回环地址可访问,则检查是否被防火墙规则过滤。 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file added
BIN
+49.1 KB
...lugin-content-docs-guides/current/gettingStarted/cluster.assets/prometheus0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+39.8 KB
...aurus-plugin-content-docs-guides/current/gettingStarted/cluster.assets/rpc0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
94 changes: 94 additions & 0 deletions
94
i18n/en/docusaurus-plugin-content-docs-guides/current/gettingStarted/cluster.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,94 @@ | ||
--- | ||
title: 'Cluster Deployment Instructions' | ||
sidebar_position: 6 | ||
--- | ||
|
||
## Cluster Deployment Instructions | ||
|
||
`open-im-server` supports cluster deployment. The following are the steps for cluster deployment: | ||
|
||
1. Modify the addresses in `kafka`, `minio`, `mongodb`, `etcd(discovery.yml)`, and `redis`, configuring them to the correct component addresses. **Ensure that all ports of the connected components are accessible.** | ||
|
||
2. Modify the `registerIP` of each `rpc` component to the IP address accessible by the server where `etcd` is deployed, **and ensure that all ports are accessible**. If you need to enable `prometheus`, also ensure that the `prometheus.port` of each component is accessible. | ||
|
||
![image-20241203155731369](./cluster.assets/rpc0.png) | ||
|
||
The machine deploying `prometheus` needs to modify the `targets` in `prometheus.yml`. | ||
|
||
![prometheus0](./cluster.assets/prometheus0.png) | ||
|
||
3. You can modify the number of each component in `open-im-server/start-config.yml`. | ||
|
||
4. Modify the port that `kafka` broadcasts to the host. If deploying using `docker-compose.yml` in `open-im-server`, change the `EXTERNAL` in `service.kafka.environment.KAFKA_CFG_ADVERTISED_LISTENERS` to the address accessing the `kafka` component. | ||
|
||
For example: `KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,EXTERNAL://192.168.2.36:19094`. | ||
|
||
5. If deploying `redis` in cluster mode, set `clusterMode` in `redis.yml` to `true`. | ||
|
||
7. If you deploy multiple `api` or `WebSocket` listening ports, you can use `nginx` to easily implement load balancing. The configuration reference is as follows: | ||
```yaml | ||
# nginx.conf | ||
|
||
events { | ||
worker_connections 1024; | ||
} | ||
|
||
http { | ||
upstream websocket_cluster { | ||
server 192.168.2.10:10101; # WebSocket address | ||
server 192.168.2.11:10001; | ||
} | ||
|
||
upstream api_cluster { | ||
server 192.168.2.10:10102; # API address | ||
server 192.168.2.11:10002; | ||
} | ||
|
||
# WebSocket | ||
server { | ||
listen 10001; | ||
|
||
server_name _; | ||
|
||
location / { | ||
proxy_pass http://websocket_cluster; | ||
proxy_http_version 1.1; | ||
proxy_set_header Upgrade $http_upgrade; | ||
proxy_set_header Connection 'upgrade'; | ||
proxy_set_header Host $host; | ||
proxy_cache_bypass $http_upgrade; | ||
} | ||
} | ||
|
||
# API | ||
server { | ||
listen 10002; | ||
|
||
server_name _; | ||
|
||
location / { | ||
proxy_pass http://api_cluster; | ||
proxy_set_header Host $host; | ||
proxy_set_header X-Real-IP $remote_addr; | ||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | ||
proxy_set_header X-Forwarded-Proto $scheme; | ||
} | ||
} | ||
} | ||
``` | ||
|
||
If you decide to start `nginx` using `docker`, you can refer to the following command (Linux system): | ||
|
||
```shell | ||
docker run --name nginx -p 10001:10001 -p 10002:10002 -v $(pwd)/nginx-config:/etc/nginx/conf.d -v $(pwd)/nginx.conf:/etc/nginx/nginx.conf -d nginx | ||
``` | ||
|
||
If using a different system, adjust the mounted directories accordingly. | ||
|
||
7. Start the services. | ||
|
||
## **Common Issues/Notes** | ||
|
||
1. When deploying on multiple machines, ensure clock synchronization for the services to run properly. For example, the issuance of `tokens` allows each machine's clock to have a time difference within `5s`. | ||
|
||
2. Component ports are inaccessible: Check whether the component starts normally via the loopback address. If the loopback address is accessible, then check if firewall rules are blocking access. |