Skip to content

Commit

Permalink
fix(portal-web): 修复目录下文件过多时 touch -a 会导致 500 报错的问题 (#1103)
Browse files Browse the repository at this point in the history
当目录下文件过多且文件size过大时,会报500的错误:
err:{"type":"Error","message":"read ECONNRESET","stack":"Error: read
ECONNRESET\n at TCP.onStreamRead
(node:internal/stream_base_commons:217:20)","errno":-104,"code":"ECONNRESET","syscall":"read","level":"client-socket"},"msg":"Error
occurred. Return the error."

处理方式为,将 touch 改为分批异步处理
待优化:优化跨集群传输代码,在仅有需要时才 touch
  • Loading branch information
Miracle575 authored Feb 5, 2024
1 parent 3242957 commit 941340a
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 3 deletions.
5 changes: 5 additions & 0 deletions .changeset/angry-forks-promise.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@scow/portal-server": patch
---

修复目录文件过多时导致的 touch 命令报错
12 changes: 9 additions & 3 deletions apps/portal-server/src/services/file.ts
Original file line number Diff line number Diff line change
Expand Up @@ -191,9 +191,15 @@ export const fileServiceServer = plugin((server) => {
if (pureFiles.length > 0) {
const filePaths = pureFiles.map((file) => join(path, file.filename)).join(" ");

const fileSyncCmd = `touch -a ${filePaths}`;

await loggedExec(ssh, logger, false, fileSyncCmd, []);
// 避免目录下文件过多导致 touch -a 命令报错,采用分批异步执行的方式
// 一次执行 1000 个文件是根据经验设置的安全值,可修改
for (let i = 0; i < filePaths.length; i += 1000) {
const execFilePaths = filePaths.slice(i, i + 1000);
const fileSyncCmd = `touch -a ${execFilePaths}`;
loggedExec(ssh, logger, false, fileSyncCmd, []).catch((err) => {
logger.error(err, "touch -a failed", userId, execFilePaths);
});
}
}

for (const file of files) {
Expand Down

0 comments on commit 941340a

Please sign in to comment.