Merge pull request #199 from alimy/feature/pyroscope

add Pyroscope support
rocboss · Feb 15, 2023 · fb42e72 · fb42e72
2 parents 0103e18 + a821f47
commit fb42e72
Show file tree

Hide file tree

Showing 27 changed files with 215 additions and 26 deletions.
diff --git a/README.md b/README.md
@@ -370,6 +370,7 @@ release/paopao-ce --no-default-features --features sqlite3,localoss,loggerfile,r
 |`Alipay` | 支付 | 稳定 | 开启基于[支付宝开放平台](https://open.alipay.com/)的钱包功能 |
 |`Sms` | 短信验证 | 稳定 | 开启短信验证码功能，用于手机绑定验证手机是否注册者的；功能如果没有开启，手机绑定时任意短信验证码都可以绑定手机 |
 |`Docs:OpenAPI` | 开发文档 | 稳定 | 开启openapi文档功能，提供web api文档说明(visit http://127.0.0.1:8008/docs/openapi) |
+|[`Pyroscope`](docs/proposal/016-关于使用pyroscope用于性能调试的设计.md)| 性能优化 | 内测 | 开启Pyroscope功能用于性能调试 |   
 |`PhoneBind` | 其他 | 稳定 | 手机绑定功能 |   
 
 > 功能项状态详情参考 [features-status](features-status.md).
@@ -492,6 +493,32 @@ MinIO: # MinIO 存储配置
 ...
 ```
 
+#### [Pyroscope](https://github.com/pyroscope-io/pyroscope) 性能剖析
+* Pyroscope运行
+```sh
+mkdir -p data/minio/data
+
+# 使用Docker运行
+docker run -it -p 4040:4040 pyroscope/pyroscope:latest server
+# 使用docker compose运行， 需要删除docker-compose.yaml中关于pyroscope的注释
+docker compose up -d pyroscope
+# visit http://loclahost:4040
+```
+
+* 修改Pyroscope配置
+```yaml
+# features中加上 MinIO
+Features:
+  Default: ["Meili", "LoggerMeili", "Base", "Sqlite3", "BigCacheIndex", "Pyroscope"]
+...
+Pyroscope: # Pyroscope配置
+  AppName: "paopao-ce"
+  Endpoint: "http://localhost:4040"   # Pyroscope server address
+  AuthToken:                          # Pyroscope authentication token
+  Logger:  none                       # Pyroscope logger (standard | logrus | none)
+...
+```
+
 ### 源代码分支管理 
 **主代码库`github.com/rocboss/paopao-ce`**      
 ```bash

diff --git a/ROADMAP.md b/ROADMAP.md
@@ -13,6 +13,7 @@
 #### v0.2.0
 * [x] add `Friendship` feature
 * [x] add `Lightship` feature
+* [x] add `Pyroscope` feature
 * [ ] add extend base ORM code for implement data logic base sqlx/sqlc
 * [x] add new `Web` service
 * [x] add `Frontend:Web` feature
@@ -41,15 +42,15 @@
 * [ ] optimize backend data logic service(optimize database CRUD operate)
 
 ## paopao-ce-plus roadmap
+#### paopao-ce-plus/v0.3.0
+* [ ] adapt for paopao-ce v0.3.0
+
 #### paopao-ce-plus/v0.2.0
 * [ ] adapt for paopao-ce v0.2.0
 
-#### paopao-ce-plus/v0.3.0
+## paopao-ce-pro roadmap
+#### paopao-ce-pro/v0.3.0
 * [ ] adapt for paopao-ce v0.3.0
 
-## paopao-ce-pro roadmap
 #### paopao-ce-pro/v0.2.0
 * [ ] adapt for paopao-ce v0.2.0
-
-#### paopao-ce-pro/v0.3.0
-* [ ] adapt for paopao-ce v0.3.0
diff --git a/auto/api/m/v1/user.go b/auto/api/m/v1/user.go
diff --git a/auto/api/r/v1/user.go b/auto/api/r/v1/user.go
diff --git a/auto/api/s/v1/user.go b/auto/api/s/v1/user.go
diff --git a/auto/api/v1/admin.go b/auto/api/v1/admin.go
diff --git a/auto/api/v1/alipay_priv.go b/auto/api/v1/alipay_priv.go
diff --git a/auto/api/v1/alipay_pub.go b/auto/api/v1/alipay_pub.go
diff --git a/auto/api/v1/core.go b/auto/api/v1/core.go
diff --git a/auto/api/v1/followship.go b/auto/api/v1/followship.go
diff --git a/auto/api/v1/friendship.go b/auto/api/v1/friendship.go
diff --git a/auto/api/v1/loose.go b/auto/api/v1/loose.go
diff --git a/auto/api/v1/priv.go b/auto/api/v1/priv.go
diff --git a/auto/api/v1/pub.go b/auto/api/v1/pub.go
diff --git a/auto/api/x/v1/user.go b/auto/api/x/v1/user.go
diff --git a/docker-compose.yaml b/docker-compose.yaml
@@ -76,6 +76,16 @@ services:
   #   networks:
   #     - paopao-network
 
+  pyroscope:
+    image: pyroscope/pyroscope:latest
+    restart: always
+    ports:
+      - 4040:4040
+    command:
+      - 'server'
+    networks:
+      - paopao-network
+
   phpmyadmin:
     image: phpmyadmin:5.2
     depends_on:

diff --git a/docs/proposal/015-关于paopao-ce引入bcrypt作为用户密码加密算法的设计.md b/docs/proposal/015-关于paopao-ce引入bcrypt作为用户密码加密算法的设计.md
@@ -17,8 +17,8 @@ TODO；
 TODO；
 
 ### 参考文档
-[bcrypt](https://github.com/golang/crypto/blob/master/bcrypt/bcrypt.go)
-[bcrypt module](https://pkg.go.dev/golang.org/x/[email protected]/bcrypt)
+* [bcrypt](https://github.com/golang/crypto/blob/master/bcrypt/bcrypt.go)
+* [bcrypt module](https://pkg.go.dev/golang.org/x/[email protected]/bcrypt)
 
 ### 更新记录
 #### v0.0(2023-02-13) - 北野

diff --git a/docs/proposal/016-关于使用pyroscope用于性能调试的设计.md b/docs/proposal/016-关于使用pyroscope用于性能调试的设计.md
@@ -0,0 +1,43 @@
+| 编号 | 作者 | 发表时间 | 变更时间 | 版本 | 状态 |
+| ----- | ----- | ----- | ----- | ----- | ----- |
+| 016| 北野 | 2023-02-15 | 2023-02-15 | v1.0 | 提议 |
+
+### 概述  
+Pyroscope 是一个开源的持续性能剖析平台。它能够帮你：
+* 找出源代码中的性能问题和瓶颈
+* 解决 CPU 利用率高的问题
+* 理解应用程序的调用树（call tree）
+* 追踪随一段时间内变化的情况
+
+### 需求   
+* 开发环境下启用Pyroscope，但是部署环境下禁用。
+
+### 方案  
+* 使用//go:build pyroscope 可选编译app集成Pyroscope功能
+* config.yaml中添加`Pyroscope` 功能来启用Pyroscope
+
+### 疑问
+
+1. 为什么要引入Pyroscope？      
+用于开发环境下对paopao-ce进行性能优化。
+
+2. 如何开启这个功能？          
+在配置文件config.yaml中的`Features`中添加`Pyroscope`功能项开启该功能：
+    ```yaml
+    ...
+    # features中加上 Friendship
+    Features:
+      Default: ["Meili", "LoggerMeili", "Base", "Sqlite3", "BigCacheIndex", "MinIO", "Pyroscope"]
+      Base: ["Redis", "PhoneBind"]
+    ...
+    ```
+
+
+
+### 参考文档
+* [pyroscope](https://github.com/pyroscope-io/pyroscope)
+* [pyroscope client](https://github.com/pyroscope-io/client)
+
+### 更新记录
+#### v1.0(2023-02-15) - 北野
+* 初始文档
diff --git a/features-status.md b/features-status.md
@@ -164,10 +164,15 @@
     * [x] 业务逻辑实现 
 
 ### 开发文档:  
-* `Docs:OpenAPI` 开启openapi文档功能，提供web api文档说明(visit http://127.0.0.1:8008/docs/openapi)；  
+* `Docs:OpenAPI` 开启openapi文档功能，提供web api文档说明(visit http://127.0.0.1:8008/docs/openapi);  
     * [ ] 提按文档  
     * [x] 接口定义
-    * [x] 业务逻辑实现      
+    * [x] 业务逻辑实现   
+
+### 性能优化
+* [`Pyroscope`](docs/proposal/016-关于使用pyroscope用于性能调试的设计.md) 开启Pyroscope功能用于性能调试(目前状态: 内测);  
+    * [x] 提按文档  
+    * [x] 业务逻辑实现  
 
 ### 其他:    
 * `PhoneBind` 手机绑定功能； 

diff --git a/go.mod b/go.mod
@@ -6,7 +6,7 @@ require (
 	github.com/Masterminds/semver/v3 v3.1.1
 	github.com/afocus/captcha v0.0.0-20191010092841-4bd1f21c8868
 	github.com/alimy/cfg v0.3.0
-	github.com/alimy/mir/v3 v3.1.0
+	github.com/alimy/mir/v3 v3.1.1
 	github.com/aliyun/aliyun-oss-go-sdk v2.2.2+incompatible
 	github.com/allegro/bigcache/v3 v3.0.2
 	github.com/bytedance/sonic v1.5.0
@@ -26,6 +26,7 @@ require (
 	github.com/json-iterator/go v1.1.12
 	github.com/meilisearch/meilisearch-go v0.21.0
 	github.com/minio/minio-go/v7 v7.0.45
+	github.com/pyroscope-io/client v0.6.0
 	github.com/sirupsen/logrus v1.9.0
 	github.com/smartwalle/alipay/v3 v3.1.7
 	github.com/spf13/viper v1.14.0
@@ -96,6 +97,7 @@ require (
 	github.com/pelletier/go-toml v1.9.5 // indirect
 	github.com/pelletier/go-toml/v2 v2.0.5 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
+	github.com/pyroscope-io/godeltaprof v0.1.0 // indirect
 	github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 // indirect
 	github.com/rs/xid v1.4.0 // indirect
 	github.com/smartwalle/crypto4go v1.0.2 // indirect

diff --git a/go.sum b/go.sum
@@ -147,8 +147,8 @@ github.com/alexflint/go-filemutex v0.0.0-20171022225611-72bdc8eae2ae/go.mod h1:C
 github.com/alexflint/go-filemutex v1.1.0/go.mod h1:7P4iRhttt/nUvUOrYIhcpMzv2G6CY9UnI16Z+UJqRyk=
 github.com/alimy/cfg v0.3.0 h1:9xgA0QWVCPSq9fFNRcYahVCAX22IL9ts2wrTQPfAStY=
 github.com/alimy/cfg v0.3.0/go.mod h1:rOxbasTH2srl6StAjNF5Vyi8bfrdkl3fLGmOYtSw81c=
-github.com/alimy/mir/v3 v3.1.0 h1:zk8Afsi1rcbRpL8MdGCkKqWqqLUupeSY9NcJgnXxic8=
-github.com/alimy/mir/v3 v3.1.0/go.mod h1:ybhT2ijOiDn0lLwWzIY6vXdv+uzZrctS7VFfczcXBWU=
+github.com/alimy/mir/v3 v3.1.1 h1:3tz7uGOwuA1IKU0BysyBvGbyqKtEVMuhPBD/APk1ANw=
+github.com/alimy/mir/v3 v3.1.1/go.mod h1:ybhT2ijOiDn0lLwWzIY6vXdv+uzZrctS7VFfczcXBWU=
 github.com/aliyun/aliyun-oss-go-sdk v2.2.2+incompatible h1:9gWa46nstkJ9miBReJcN8Gq34cBFbzSpQZVVT9N09TM=
 github.com/aliyun/aliyun-oss-go-sdk v2.2.2+incompatible/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8=
 github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156/go.mod h1:Cb/ax3seSYIx7SuZdm2G2xzfwmv3TPSk2ucNfQESPXM=
@@ -1250,6 +1250,10 @@ github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4O
 github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
 github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
 github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
+github.com/pyroscope-io/client v0.6.0 h1:rcUFgcnfmuyVYDYT+4d0zfqc8YedOyruHSsUb9ImaBw=
+github.com/pyroscope-io/client v0.6.0/go.mod h1:4h21iOU4pUOq0prKyDlvYRL+SCKsBc5wKiEtV+rJGqU=
+github.com/pyroscope-io/godeltaprof v0.1.0 h1:UBqtjt0yZi4jTxqZmLAs34XG6ycS3vUTlhEUSq4NHLE=
+github.com/pyroscope-io/godeltaprof v0.1.0/go.mod h1:psMITXp90+8pFenXkKIpNhrfmI9saQnPbba27VIaiQE=
 github.com/remyoudompheng/bigfft v0.0.0-20190728182440-6a916e37a237/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
 github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 h1:OdAsTTz6OkFY5QxjkYwrChwuRruF69c169dPK26NUlk=
 github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=

diff --git a/internal/conf/conf.go b/internal/conf/conf.go
@@ -19,6 +19,7 @@ var (
 	loggerMeiliSetting *LoggerMeiliSettingS
 	redisSetting       *RedisSettingS
 
+	PyroscopeSetting        *PyroscopeSettingS
 	DatabaseSetting         *DatabaseSetingS
 	MysqlSetting            *MySQLSettingS
 	PostgresSetting         *PostgresSettingS
@@ -81,6 +82,7 @@ func setupSetting(suite []string, noDefault bool) error {
 		"BigCacheIndex":     &BigCacheIndexSetting,
 		"Alipay":            &AlipaySetting,
 		"SmsJuhe":           &SmsJuheSetting,
+		"Pyroscope":         &PyroscopeSetting,
 		"Logger":            &loggerSetting,
 		"LoggerFile":        &loggerFileSetting,
 		"LoggerZinc":        &loggerZincSetting,

diff --git a/internal/conf/config.yaml b/internal/conf/config.yaml
@@ -72,6 +72,11 @@ BigCacheIndex: # 使用BigCache缓存泡泡广场消息流
   MaxIndexPage: 1024            # 最大缓存页数，必须是2^n, 代表最大同时缓存多少页数据
   Verbose: False                # 是否打印cache操作的log
   ExpireInSecond: 300           # 多少秒(>0)后强制过期缓存
+Pyroscope: # Pyroscope配置
+  AppName: "paopao-ce"                # application name
+  Endpoint: "http://localhost:4040"   # Pyroscope server address
+  AuthToken:                          # Pyroscope authentication token
+  Logger:  none                       # Pyroscope logger (standard | logrus | none)
 Logger: # 日志通用配置
   Level: debug # 日志级别 panic|fatal|error|warn|info|debug|trace
 LoggerFile: # 使用File写日志

diff --git a/internal/conf/settting.go b/internal/conf/settting.go
@@ -10,6 +10,7 @@ import (
 	"strings"
 	"time"
 
+	"github.com/pyroscope-io/client/pyroscope"
 	"github.com/sirupsen/logrus"
 	"github.com/spf13/viper"
 	"gorm.io/gorm/logger"
@@ -22,6 +23,13 @@ type Setting struct {
 	vp *viper.Viper
 }
 
+type PyroscopeSettingS struct {
+	AppName   string
+	Endpoint  string
+	AuthToken string
+	Logger    string
+}
+
 type LoggerSettingS struct {
 	Level string
 }
@@ -381,6 +389,16 @@ func (s *MeiliSettingS) Endpoint() string {
 	return endpoint(s.Host, s.Secure)
 }
 
+func (s *PyroscopeSettingS) GetLogger() (logger pyroscope.Logger) {
+	switch strings.ToLower(s.Logger) {
+	case "standard":
+		logger = pyroscope.StandardLogger
+	case "logrus":
+		logger = logrus.StandardLogger()
+	}
+	return
+}
+
 func endpoint(host string, secure bool) string {
 	schema := "http"
 	if secure {

diff --git a/main.go b/main.go
@@ -59,10 +59,15 @@ func main() {
 		fmt.Fprintln(color.Output, "no service need start so just exit")
 		return
 	}
-	wg := &sync.WaitGroup{}
+
+	// start pyroscope if need
+	debug.StartPyroscope()
+
 	// start services
+	wg := &sync.WaitGroup{}
 	fmt.Fprintf(color.Output, "\nstarting run service...\n\n")
 	service.Start(wg)
+
 	// graceful stop services
 	wg.Add(1)
 	go func() {