help request: I use enable_compression in log-rotate plugin, but apisix did not delete the original access log

[iamacnhero]

Description

I configged in config.yml like this:

plugin_attr:
  log-rotate:
      interval: 86400             # rotate interval (unit: second)
      max_kept: 30                # max number of log files will be kept
      max_size: -1                # max size of log files will be kept
      enable_compression: true    # enable log file compression(gzip) or not, default false

but the original log does not deleted:

why? how should I fix this ?

Environment

• APISIX version: 3.0.0
• Operating system: Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 GNU/Linux
• OpenResty / Nginx version: openresty/1.21.4.1
• etcd version: 3.5.9
• Plugin runner version, for issues related to plugin runners: log-rotate

[jiangfucheng]

apisix/apisix/plugins/log-rotate.lua

Lines 172 to 184 in 513c052

	local ok, stdout, stderr, reason, status = shell.run(cmd)
	if not ok then
	core.log.error("compress log file from ", new_filename, " to ", com_filename,
	" fail, stdout: ", stdout, " stderr: ", stderr, " reason: ", reason,
	" status: ", status)
	return
	end
	ok, stderr = os_remove(new_file)
	if stderr then
	core.log.error("remove uncompressed log file: ", new_file,
	" fail, err: ", stderr, " res:", ok)
	end

It seems cause by that the log file too large cause compress command timeout and then return immediately, not remove origin file. Maybe we can increase the timeout of shell.run() method, I'm not sure if this will lead other issues. Let's listen to the opinions of others.

[iamacnhero]

Indeed, the compression timed out. below is the error.log

2023/07/11 00:00:11 [error] 52#52: *45898080 [lua] log-rotate.lua:186: compression_file(): compress log file from 2023-07-11_00-00-00__access.log to 2023-07-11_00-00-00__access.log.tar.gz fail, stdout: stderr: reason: failed to wait for process: timeout status: nil, context: ngx.timer

[alptugay]

#8620
I've created a pull request for this but is now stale. It will solve your problem

[shreemaan-abhishek]

@alptugay the PR has been reopened, I have left review comments let's get this merged. 🔥

[shreemaan-abhishek]

fixed by #8620