fixed memory leak on nginx reload #815
Conversation
lneto
commented
Dec 16, 2014
lneto
commented
- created a root pool to destroy modsec pool safely among reloads
Refactoring on the nginx module, including: - Better handling larger posts; - Now using nginx echo module during the regression tests. - Better interacting with neginx chain rules - Separation of the request handling and content filters. - Better handling nginx sessions and resource counts to allow a more efficient garbage collector. - Handling both http/1.0 and 1.1, including keep-alive. - Tests are now capable to test nginx as a proxy or end-server. - Tested agains nginx 1.6 and 1.7.
If nginx segfaults it will return, warning that the test failed.
Add a check for the definition MOVE_REQUEST_CHAIN_TO_MODSEC, whenever it is set the chain will be moved into the brigade. If it was not set the chain will be only copied. Moving was causing segfaults on the following regression tests: owasp-modsecurity#15 - SecRequestBodyInMemoryLimit owasp-modsecurity#16 - SecRequestBodyInMemoryLimit (greater) owasp-modsecurity#19 - SecRequestBodyLimitAction ProcessPartial (multipart/greater - chunked) (from: regression/config/10-request-directives.t)
Otherwise nginx's installation directory could not be specified. Signed-off-by: paulyang <[email protected]>
This eliminates segfaults caused by unset (NULL) r->port_start and non-NULL r->port_end. In fact, r->port_start is always NULL, so it is useless to rely on this pointer.
* created a root pool to destroy modsec pool safely among reloads
zimmerle
force-pushed
the
nginx_refactoring
branch
from
33b8760 to
ed4a4d2
Compare
|
Please, don't merge yet. A new PR is comming ;-). |
|
@zimmerle, I can't find the new pull request from @lneto you are referring to, so asking here: We face the said memory leak with ModSec 2.9.0 compiled from code with Nginx 1.8.0. We are running Nginx+ModSec in pair with Consul-template, which reloads Nginx config anytime when backend service config changes in Consul. Our Nginx servers are running out or memory after 100-200 config reloads, which totally breaks the stability of our (pretty dynamic) infra. So I am wondering why, after one year, fix is still not present in a stable branch.. What blocks the release of this fix? I will be glad to help, e.g. to help with reproducing this issue by providing a Docker image with erroneous Nginx + Modsec with reproducer script. |
|
Oops, sorry, missed the opened #895, added my question there |
|
As you may able to see by @lneto comments, he is preparing some modifications to send another pull request. We are currently working in a new version of ModSecurity, version 3. This version 3 works in a different fashion fixing not only this issue, but other related to this nginx implementation. There are further information about libModSecurity here: http://tinyurl.com/zdyhgxk |
|
Actually, I was; but it turns out that I couldn't come out with a Regards,Lourival Vieira Neto On Tue, Feb 2, 2016 at 1:37 PM, Felipe Zimmerle
|
|
@zimmerle, @lneto, thank you for an update. @zimmerle, are you going to merge at least critical fixes (like fixes for memory and file descriptor leaks) to 2.9 version or you are going to stop 2.9 version support? So, what should current ModSec + Nginx users do until ModSec 3.0 will be ready for PROD usage: switch to Apache or wait for ModSec 2.9.1 with critical Nginx-related fixes merged into last stable code? |
|
Currently all the critical issues are merged into the "nginx_refactoring" branch. This branch won't be released. The version 2.9.1-RC1 will be release tomorrow (if everything goes right), but with no updates to nginx version. That particular fixes are not merged because they may lead to other problems, currently they are only affecting dynamic configuration such as yours. Thus, very restrict set of users. Keep tuned on the libModSecurity updates, i think it seems promising to your infrastructure. |
@zimmerle, using this branch at PROD is not acceptable for us because except for the said fix there is a lot of other changes not qualified by community.. So I forked the 2.9.0 version and applied only critical fixes from 'nginx_refactoring' branch on the top of it: https://github.com/daniilyar/ModSecurity. This fork is tested under the load at our dynamic env and satisfies our needs. We are going to merge all farther critical fixes to it until new 'clear and shiny' ModSec 3.0 (http://blog.zimmerle.org/2016/01/an-overview-of-upcoming-libmodsecurity.html) will be at least near-to-PROD-ready. |
