Stats.DB Index Repo[425] hangs #26064
Comments
|
I restarted my gitea to "fix" this. |
|
Thanks for the report @somera, could you confirm that repo with ID 425 in your DB is https://github.com/google/material-design-icons ? I'm cloning it now to poke at it in my instance. |
|
Just a guess, maybe it's related to this problem (I have left some comments in the "code/indexer.go") This issue's stacktrace: Previous comment:
|
Yes, this repo is in my DB. |
|
1.20.2: hangs for 2 hours now.
|
|
Still the same problem as above: stats.(*DBIndexer).Index -> GetLanguageStats -> git.ReadBatchLine .... I have tried writing some different tests but I still can't reproduce it on my side, no idea yet ..... |
|
I think this happens, when the mirror and stats tasks run at the same time. But i'm not sure, that this is the reason. |
|
And I found |
|
And I see, this is the same repo id -> 425. |
|
I removed the mirror. The issue can be closed or you fix this: #26064 (comment) |
|
I think it is a really serious bug, so it could be kept open until it gets resolved ...... ps: the repo contains a lot of small files (580937) , https://github.com/google/material-design-icons |
lunny
added
the
issue/critical
|
As a workaround: I think it could help you to avoid indexing such large repo |
|
After more trails, I figured out the problem. It looks like a Golang's framework's Demo code: func TestCommand(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
cmd := exec.CommandContext(ctx, "cat")
rStdin, wStdin := io.Pipe()
rStdout, wStdout := io.Pipe() // nio.Pipe(buffer.New(32 * 1024))
cmd.Stdin = rStdin
cmd.Stdout = wStdout
written := 0
read := 0
stopWriting := atomic.Bool{}
go func() {
for !stopWriting.Load() {
line := strings.Repeat("a", 5+rand.Intn(100)) + "\n"
for i := 0; i < len(line); i++ {
_, _ = wStdin.Write([]byte{line[i]})
}
written++
time.Sleep(50 * time.Microsecond)
}
}()
go func() {
bio := bufio.NewReader(rStdout)
for {
_, _ = bio.ReadString('\n')
read++
}
}()
go func() {
time.Sleep(1000 * time.Millisecond)
t.Logf("now written: %d, read: %d, stop writing", written, read)
stopWriting.Store(true) // with this line, if the "writing" is stopped ahead, the cmd.Run()/Wait() gets stuck forever
t.Logf("cancel context (kill process) ...")
cancel()
}()
err := cmd.Run()
assert.Error(t, err)
t.Logf("done. written: %d, read: %d", written, read)
}Update: Hmm .... OK, actually, not Golang's bug, it's a Golang's design flaw ..... the Stdin/Stdout should not be used that way. |
|
And one more thing, after #26325 gets merged, you might see your mirror tasks consume more CPU Before: the mirror's tasks like "language stats" runs with CPU=100% and fails after 10minute timeout After: the tasks won't fail by timeout, it will try to run as long as possible to finish the "language stats" task, for the "material-design-icons" repo which has more than 580k files, it will run for a pretty long time with CPU=100%. The good news is that the "languages stats" will be right then. |
…lt timeout (go-gitea#26325) Fix go-gitea#26064 Some git commands should use parent context, otherwise it would exit too early (by the default timeout, 10m), and the "cmd.Wait" waits till the pipes are closed.
…lt timeout (#26325) (#26330) Backport #26325 by @wxiaoguang Fix #26064 Some git commands should use parent context, otherwise it would exit too early (by the default timeout, 10m), and the "cmd.Wait" waits till the pipes are closed. Co-authored-by: wxiaoguang <[email protected]>
…lt timeout (go-gitea#26325) (go-gitea#26330) Backport go-gitea#26325 by @wxiaoguang Fix go-gitea#26064 Some git commands should use parent context, otherwise it would exit too early (by the default timeout, 10m), and the "cmd.Wait" waits till the pipes are closed. Co-authored-by: wxiaoguang <[email protected]> (cherry picked from commit 9451781)
Description
Just saw, that the job hangs.
gitea-diagnosis-20230722-235117.zip
Gitea Version
1.20.1
Can you reproduce the bug on the Gitea demo site?
Yes
Log Gist
No response
Screenshots
No response
Git Version
2.25.1
Operating System
Ubuntu 20.04.6
How are you running Gitea?
Precompiled gitea-1.20.1-linux-amd64
Database
PostgreSQL
The text was updated successfully, but these errors were encountered: