Skip to content

Commit

Permalink
Added kibana (#2286)
Browse files Browse the repository at this point in the history
Added kibana to make elastic management easier.
PR #1710 did this. 
PR #1714 revert this.
This PR did again and fix some bugs.

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
  • Loading branch information
yuzhichang authored Sep 6, 2024
1 parent 925dd2a commit 6b23308
Show file tree
Hide file tree
Showing 7 changed files with 68 additions and 24 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ Try our demo at [https://demo.ragflow.io](https://demo.ragflow.io).
* Running on http://x.x.x.x:9380
INFO:werkzeug:Press CTRL+C to quit
```
> If you skip this confirmation step and directly log in to RAGFlow, your browser may prompt a `network anomaly` error because, at that moment, your RAGFlow may not be fully initialized.
> If you skip this confirmation step and directly log in to RAGFlow, your browser may prompt a `network abnormal` error because, at that moment, your RAGFlow may not be fully initialized.
5. In your web browser, enter the IP address of your server and log in to RAGFlow.
> With the default settings, you only need to enter `http://IP_OF_YOUR_MACHINE` (**sans** port number) as the default HTTP serving port `80` can be omitted when using the default configurations.
Expand Down
2 changes: 1 addition & 1 deletion README_ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@
* Running on http://x.x.x.x:9380
INFO:werkzeug:Press CTRL+C to quit
```
> 만약 확인 단계를 건너뛰고 바로 RAGFlow에 로그인하면, RAGFlow가 완전히 초기화되지 않았기 때문에 브라우저에서 `network anomaly` 오류가 발생할 수 있습니다.
> 만약 확인 단계를 건너뛰고 바로 RAGFlow에 로그인하면, RAGFlow가 완전히 초기화되지 않았기 때문에 브라우저에서 `network abnormal` 오류가 발생할 수 있습니다.
5. 웹 브라우저에 서버의 IP 주소를 입력하고 RAGFlow에 로그인하세요.
> 기본 설정을 사용할 경우, `http://IP_OF_YOUR_MACHINE`만 입력하면 됩니다 (포트 번호는 제외). 기본 HTTP 서비스 포트 `80`은 기본 구성으로 사용할 때 생략할 수 있습니다.
Expand Down
2 changes: 1 addition & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@
* Running on http://x.x.x.x:9380
INFO:werkzeug:Press CTRL+C to quit
```
> 如果您跳过这一步系统确认步骤就登录 RAGFlow,你的浏览器有可能会提示 `network anomaly``网络异常`,因为 RAGFlow 可能并未完全启动成功。
> 如果您跳过这一步系统确认步骤就登录 RAGFlow,你的浏览器有可能会提示 `network abnormal``网络异常`,因为 RAGFlow 可能并未完全启动成功。
5. 在你的浏览器中输入你的服务器对应的 IP 地址并登录 RAGFlow。
> 上面这个例子中,您只需输入 http://IP_OF_YOUR_MACHINE 即可:未改动过配置则无需输入端口(默认的 HTTP 服务端口 80)。
Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ main
### Actual behavior

The restricted_loads function at [api/utils/__init__.py#L215](https://github.com/infiniflow/ragflow/blob/main/api/utils/__init__.py#L215) is still vulnerable leading via code execution.
The main reson is that numpy module has a numpy.f2py.diagnose.run_command function directly execute commands, but the restricted_loads function allows users import functions in module numpy.
The main reason is that numpy module has a numpy.f2py.diagnose.run_command function directly execute commands, but the restricted_loads function allows users import functions in module numpy.


### Steps to reproduce
Expand Down
6 changes: 5 additions & 1 deletion docker/docker-compose-admin-tool.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
include:
- path: ./docker-compose.yml
env_file: ./.env

services:
kibana:
image: kibana:${STACK_VERSION}
Expand All @@ -12,7 +16,7 @@ services:
es01:
condition: service_healthy
kibana-user-init:
condition: service_completed_successfully
condition: service_completed_successfully

networks:
- ragflow
Expand Down
3 changes: 3 additions & 0 deletions docker/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
#!/bin/bash

# unset http proxy which maybe set by docker daemon
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""

/usr/sbin/nginx

export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/
Expand Down
75 changes: 56 additions & 19 deletions docker/init-kibana.sh
Original file line number Diff line number Diff line change
@@ -1,30 +1,67 @@
#!/bin/bash

# 等待 Elasticsearch 啟動
until curl -u "elastic:${ELASTIC_PASSWORD}" -s http://es01:9200 >/dev/null; do
echo "等待 Elasticsearch 啟動..."
sleep 5
done
# unset http proxy which maybe set by docker daemon
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""

echo "Elasticsearch built-in user: elastic:${ELASTIC_PASSWORD}"

echo "使用者: elastic:${ELASTIC_PASSWORD}"
# Wait Elasticsearch be healthy
while true; do
response=$(curl -s -v -w "\n%{http_code}" -u "elastic:${ELASTIC_PASSWORD}" "http://es01:9200")
exit_code=$?
status=$(echo "$response" | tail -n1)
if [ $exit_code -eq 0 ] && [ "$status" = "200" ]; then
echo "Elasticsearch is healthy"
break
else
echo "Elasticsearch is unhealthy: $exit_code $status"
echo "$response"
sleep 5
fi
done

# Create new role with all privileges to all indices
# https://www.elastic.co/guide/en/elasticsearch/reference/current/security-privileges.html#privileges-list-indices
echo "Going to create Elasticsearch role own_indices with all privileges to all indices"
while true; do
response=$(curl -s -v -w "\n%{http_code}" -u "elastic:${ELASTIC_PASSWORD}" -X POST http://es01:9200/_security/role/own_indices -H 'Content-Type: application/json' -d '{"indices": [{"names": ["*"], "privileges": ["all"]}]}')
exit_code=$?
status=$(echo "$response" | tail -n1)
if [ $exit_code -eq 0 ] && [ "$status" = "200" ]; then
echo "Elasticsearch role own_indices created"
break
else
echo "Elasticsearch role own_indices failure: $exit_code $status"
echo "$response"
sleep 5
fi
done

echo "Elasticsearch role own_indices:"
curl -u "elastic:${ELASTIC_PASSWORD}" -X GET "http://es01:9200/_security/role/own_indices"
echo ""

PAYLOAD="{
\"password\" : \"${KIBANA_PASSWORD}\",
\"roles\" : [ \"kibana_admin\",\"kibana_system\" ],
\"full_name\" : \"${KIBANA_USER}\",
\"email\" : \"${KIBANA_USER}@example.com\"
}"
echo "新用戶帳戶: $PAYLOAD"
PAYLOAD="{\"password\": \"${KIBANA_PASSWORD}\", \"roles\": [\"kibana_admin\", \"kibana_system\", \"own_indices\"], \"full_name\": \"${KIBANA_USER}\", \"email\": \"${KIBANA_USER}@example.com\"}"

# 創建新用戶帳戶
curl -X POST "http://es01:9200/_security/user/${KIBANA_USER}" \
-u "elastic:${ELASTIC_PASSWORD}" \
-H "Content-Type: application/json" \
-d "$PAYLOAD"s
echo "Going to create Elasticsearch user ${KIBANA_USER}: ${PAYLOAD}"

# Create new user
while true; do
response=$(curl -s -v -w "\n%{http_code}" -u "elastic:${ELASTIC_PASSWORD}" -X POST http://es01:9200/_security/user/${KIBANA_USER} -H "Content-Type: application/json" -d "${PAYLOAD}")
exit_code=$?
status=$(echo "$response" | tail -n1)
if [ $exit_code -eq 0 ] && [ "$status" = "200" ]; then
echo "Elasticsearch user ${KIBANA_USER} created"
break
else
echo "Elasticsearch user ${KIBANA_USER} failure: $exit_code $status"
echo "$response"
sleep 5
fi
done

echo "新用戶帳戶已創建"
echo "Elasticsearch user ${KIBANA_USER}:"
curl -u "elastic:${ELASTIC_PASSWORD}" -X GET "http://es01:9200/_security/user/${KIBANA_USER}"
echo ""

exit 0

0 comments on commit 6b23308

Please sign in to comment.