intro goroutine pool to speed up engine API call #343
Conversation
jschwinger233
changed the title
|
jschwinger233
force-pushed
the
zc/conn
branch
2 times, most recently
from
d78461c to
e6587e4
Compare
jschwinger233
changed the title
jschwinger233
changed the title
|
github.com/projecteru2/core/discovery/helium 这个部分改了啥，为啥覆盖率下降了4个点 |
看来是鸡蛋引入的 |
| if e := sem.Acquire(context.Background(), c.config.MaxConcurrency); e != nil { | ||
| log.Errorf("[Calcium.doDeployWorkloadsOnNode] Failed to wait all workers done: %+v", e) | ||
| err = e | ||
| indices = utils.Range(deploy) | ||
| } |
There was a problem hiding this comment.
理论上由于使用了 Background ctx, 这里不会有 error;
如果阻塞在 Acquire(MaxConn), 那 c-c 也会继续阻塞在这里, 但是分发下去的 goroutine 去调用 docker api 的时候就会被 cancel, 最终导致部分创建容器成功部分失败, 然后在那些 goroutine defer 里 release 之后, Acquire(MaxConn) 成功 return.
There was a problem hiding this comment.
其实就和 WaitGroup.Wait() 是一样的了, wg.Wait() 直接就不让 ctx 打断, 这里使用 Background 就回到 wg.Wait()
本地运行是 88%, CI 就变成 82% 了... |
cluster/calcium/helper.go
Outdated
| @@ -191,6 +191,7 @@ func processVirtualizationOutStream( | |||
| if split != 0 { | |||
| bs = append(bs, split) | |||
| } | |||
| println(string(bs)) | |||
| networks := map[string]string{} | ||
| for name, network := range opts.Networks { | ||
| networkMode = dockercontainer.NetworkMode(name) | ||
| networks[name] = network | ||
| if networkMode.IsHost() { | ||
| opts.Networks[name] = "" | ||
| networks[name] = "" |
There was a problem hiding this comment.
新创建了局部变量 map, 而不是修改传入的指针, 因为这里变成并发调用了, 并发修改指针 map 不可以.
我本地运行还惨一些，直接 segment fault |
jschwinger233
force-pushed
the
zc/conn
branch
from
b2f10d3 to
a925b06
Compare
jschwinger233
force-pushed
the
zc/conn
branch
from
a925b06 to
224e96d
Compare
jschwinger233
force-pushed
the
zc/conn
branch
from
224e96d to
51b54ed
Compare
No description provided.