ModSecurity does not block request, only logs, while SecRuleEngine is set to On #4385

kwaazaar · 2019-07-31T14:16:55Z

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG

NGINX Ingress controller version: 0.25.0 (installed through helm chart nginx-ingress-1.10.1)

Kubernetes version (use kubectl version):
v1.12.7

Environment:

Cloud provider or hardware configuration: Azure Kubernetes Service (AKS)
OS (e.g. from /etc/os-release): Ubuntu 16.04.6 LTS
Kernel (e.g. uname -a): 4.15.0-1052-azure
Install tools: -
Others: -

What happened:
Request is not blocked by ModSecurity, only logged. Debug log says:
[4] Not running disruptive action: pass. SecRuleEngine is not On.

What you expected to happen:
Blocked request by returning HTTP 403, because SecRuleEngine is specified as on.

How to reproduce it (as minimally and precisely as possible):

Install nginx-ingress with this content in values.yaml (partial):
controller:
config:
enable-modsecurity: "true"
enable-owasp-modsecurity-crs: "true"
Apply ingress with this content (partial):
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecRule REQUEST_HEADERS_NAMES "^x-waf-test" "log,deny,id:48,status:403,t:lowercase,msg:WAFWAFWAF"
SecRequestBodyAccess On
SecDebugLog /tmp/modsec_debug.log
SecDebugLogLevel 9
SecRuleEngine On
Send request to ingress-host which should trigger ModSecurity rule(s):
HTTP GET to /?test=%271%20OR%201=1&x=%3Cscript%3Ealert%28%27hello%27%29;%3C/script%3E
Check mod-security debug log and see:
[156458238464.652956] [/?test=%271%20OR%201=1&x=%3Cscript%3Ealert%28%27hello%27%29;%3C/script%3E] [4] Not running disruptive action: pass. SecRuleEngine is not On.

Anything else we need to know:
I check logs with: kubectl -n kube-system exec -it internal-ingresscontroller-nginx-ingress-controller-7f8c6774w84 -- tail -f /tmp/modsec_debug.log

The text was updated successfully, but these errors were encountered:

kwaazaar · 2019-07-31T14:26:49Z

Nginx accepts the config and it not rejecting it and remaining on previous version of nginx.conf: to be sure I check the logs for the ingress controller pods and added an extra response header through a configuration snippet.

Nginx.conf: https://www.dropbox.com/s/2rj9r53ftbowl0o/nginx.conf?dl=0
Ingress definition: https://github.com/kwaazaar/nginx-ingress-adv (ingress.yml)

Aanuka · 2019-08-09T13:03:19Z

Can also verify that this is a issue.
Running the same setup with nginx 0.25.0 (Openresty 1.15.8.1), enabled modsecurity and geoip2 through values.yaml and enabling modsecurity through configuration-snippet.

Logs are being created but no block action being done.
Geoip2 filters on the same configuration snippet works.

mhauer71 · 2019-08-14T14:24:57Z

I also have the same issue with nginx 0.25.0.
Tried the example in this article:

nginx.ingress.kubernetes.io/modsecurity-snippet: |
      SecRuleEngine On
      SecRequestBodyAccess On
      SecAuditEngine RelevantOnly
      SecAuditLogParts ABIJDEFHZ
      SecAuditLog /var/log/modsec_audit.log
      SecRule REQUEST_HEADERS:User-Agent \"fern-scanner\" \"log,deny,id:107,status:403,msg:\'Fern Scanner Identified\'\"

With curl https://mysite/ -k -H "user-agent: fern-scanner" it only logged to /var/log/modsec_audit.log
Editing the ngnix deployment to version 0.24.1 then I received 403 Forbidden as expected.

Aanuka · 2019-09-16T19:48:06Z

The issue is still seen with 0.25.1
Using the following configuration:

nginx.ingress.kubernetes.io/configuration-snippet: |
  modsecurity_rules '
    SecRuleEngine On
    SecAuditLog /var/log/modsec_audit.log
    SecAuditLogParts ABIJDEFHZ
    SecAuditEngine RelevantOnly
    SecAction \
      "id:900000,\
      phase:1,\
      nolog,\
      pass,\
      t:none,\
      setvar:tx.paranoia_level=1"

Warnings are being logged in modsec_audit.log but no blocking actions are being made.

kwaazaar · 2019-09-17T14:47:46Z

Please clarify if ModSecurity integration will get attention. If not, discontinue the feature, because currently it feels like we're waisting our time with it.

aledbf · 2019-09-17T14:58:13Z

Please clarify if ModSecurity integration will get attention

What do you mean?

We updated the configuration in the last update of the nginx image here efc6645#diff-4a5113d28634e29f2ab0bca34da7a3e5R333 (not released yet).

Please use quay.io/kubernetes-ingress-controller/nginx-ingress-controller:dev to test this issue

Aanuka · 2019-09-17T15:28:09Z

Please clarify if ModSecurity integration will get attention

What do you mean?

We updated the configuration in the last update of the nginx image here efc6645#diff-4a5113d28634e29f2ab0bca34da7a3e5R333 (not released yet).

Please use quay.io/kubernetes-ingress-controller/nginx-ingress-controller:dev to test this issue

Just tried with :dev , this time it didn't even log anything to the /var/log/modsec_audit.log file when trying an command and just passes an 404

Snips from the nginx.conf

modsecurity on;
modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;
modsecurity_rules_file /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf;

modsecurity_rules '
SecRuleEngine On
SecAuditLog /var/log/modsec_audit.log
SecAuditLogParts ABIJDEFHZ
SecAuditEngine RelevantOnly
SecAction
"id:900000,
phase:1,
nolog,
pass,
t:none,
setvar:tx.paranoia_level=1"
';

nginx version: openresty/1.15.8.2

aledbf · 2019-09-18T17:59:28Z

To those affected by this issue, please check the gist test-modsecurity.sh

Use it:

wget https://gist.githubusercontent.com/aledbf/f6ccf154d4db8a3046b1067db80bd889/raw/6939cbc61b4035e8ceddddf7e8351fbe5b03bd6a/test-modsecurity.sh
chmod +x test-modsecurity.sh
./test-modsecurity.sh

This script:

starts a new kubernetes cluster using minikube.
installs the ingress controller.
updates the image to use the dev tag.
creates an ingress that contains the modsecurity-snippet annotation
makes a request with curl that returns 403 (expected behavior)
dumps the ingress controller logs

Aanuka · 2019-09-19T11:22:18Z

To those affected by this issue, please check the gist test-modsecurity.sh

Use it:

wget https://gist.githubusercontent.com/aledbf/f6ccf154d4db8a3046b1067db80bd889/raw/6939cbc61b4035e8ceddddf7e8351fbe5b03bd6a/test-modsecurity.sh

chmod +x test-modsecurity.sh

./test-modsecurity.sh

This script:

starts a new kubernetes cluster using minikube.

installs the ingress controller.

updates the image to use the dev tag.

creates an ingress that contains the modsecurity-snippet annotation

makes a request with curl that returns 403 (expected behavior)

dumps the ingress controller logs

Checked the code and reapplied the ingress on our test env, with the same issues i.e no blocking is being made. In fact, with the :dev release no detection is seen in the modsec_audit.log

celamb4 · 2019-09-21T10:07:16Z

I ran into this issue. Even with SecRuleEngine ON I only get logs and no blocks (403) when testing. I see the logs correctly catching the attacks but not blocking them.

---
kind: ConfigMap
metadata:
  name: nginx-ingress-controller
apiVersion: v1
data:  
  enable-modsecurity: "true"
  modsecurity-transaction-id: "$request_id"
  modsecurity-snippet: |
    SecRuleEngine On
    SecRequestBodyAccess On
    SecAuditEngine RelevantOnly
    SecAuditLogParts ABIJDEFHZ
    SecAuditLog /var/log/modsec_audit.log
    Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
    Include /etc/nginx/modsecurity/modsecurity.conf

Upon further review, I noticed the /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf still had detection mode only. I created my own file for both this and also crs-setup.conf files. To ensure (a) Engine is ON and (b) secAction is set to block. I then mounted them using extra volumes.

SecRuleEngine On #nginx-modsecurity.conf file
SecDefaultAction "phase:1,log,auditlog,deny,status:403" #crs-setup.conf file
SecDefaultAction "phase:2,log,auditlog,deny,status:403" #crs-setup.conf file

Deploy nginx with the following additional configurations:

    "--set controller.extraVolumes[0].configMap.name=modsecurity-config "
    "--set controller.extraVolumes[0].name=modsecurity-config "
    "--set controller.extraVolumeMounts[0].name=modsecurity-config "
    "--set controller.extraVolumeMounts[0].mountPath=/etc/nginx/modsecurity/modsecurity.conf "
    "--set controller.extraVolumeMounts[0].subPath=modsecurity.conf "
    "--set controller.extraVolumes[1].configMap.name=crs-setup-config "
    "--set controller.extraVolumes[1].name=crs-setup-config "
    "--set controller.extraVolumeMounts[1].name=crs-setup-config "
    "--set controller.extraVolumeMounts[1].mountPath=/etc/nginx/owasp-modsecurity-crs/crs-setup.conf "
    "--set controller.extraVolumeMounts[1].subPath=crs-setup.conf "

Modsecurity is now successfully blocking the requests

$ curl "https://example.com/login?q='1 OR 1=1"
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>openresty</center>
</body>
</html>

aledbf · 2019-09-28T00:07:06Z

Closing. Please update to 0.26.0. The release contains fixes for modsecurity. Please reopen if the issue persists.

Aanuka · 2019-09-28T16:38:54Z

This issue still seems to persist in 0.26.0 with even more strange behaviour.

I've tried with:

Annotations in the configmap for the controller
Annotations in the ingress for a specific URL

Without any success.
With 0.26.0 modsecurity dosen't even log any errors into /var/log/modsec_audit.log and just passes through the request like normal (I.e 404 for something that dosen't exist).

What's even more strange is that controller now seems to log everything to /var/log/modsec_audit.log, we can for an example see:
127.0.0.1:10256 127.0.0.1 - [28/Sep/2019:16:27:16 +0000] "GET /nginx_status HTTP/1.1" 200 106 - "Go-http-client/1.1" 15696880369.409824 - /var/log/audit//20190928/20190928-1627/20190928-162716-15696880369.409824 0 1315.000000 md5:2127b3a365aeddf5ae366618a2356bf3

reddare · 2019-10-09T15:49:17Z

Same with @Aanuka on 0.26.1 release.
@kwaazaar @aledbf please reopen.

Aanuka · 2019-11-11T10:00:13Z

This still seems to be an issue in 0.26.1
@kwaazaar @aledbf Can anyone of you reopen so we can try and solve this?

Keralin · 2019-11-25T10:16:33Z

I was getting crazy because of this bug, was in 0.25.1 with this problem, then updated to 0.26.1 and still the same behavior, looked at @mhauer71 post, downgraded to 0.24.1 and finally after 1 day and a half messing with this, now works as expected! Please reopen this issue, it's annoying until you realize that could be a bug @kwaazaar @aledbf

kwaazaar · 2019-11-25T14:21:21Z

I cannot reopen the issue, since a repo collaborator closed it.

It's sad that ModSecurity support seems to be ignored by the team.

zetaab · 2020-01-29T08:32:21Z

I can confirm that it does not work for 0.28.0 either like @celamb4 said the files are not correct. Files are only saying "pass" if it will detect something.

I have following in ingress configuration:

      nginx.ingress.kubernetes.io/configuration-snippet: |
        more_set_headers "server: hide";
      nginx.ingress.kubernetes.io/enable-modsecurity: "true"
      nginx.ingress.kubernetes.io/modsecurity-snippet: |
        SecRuleEngine On
        SecRequestBodyAccess On
        SecAuditEngine RelevantOnly
        SecAuditLogParts ABIJDEFHZ
        SecAuditLog /var/log/modsec_audit.log
        Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
      nginx.ingress.kubernetes.io/modsecurity-transaction-id: $request_id

and when I do request for instance curl -s -o /dev/null "https://foobar.com/?username='%20or%20'1'%20=%20'" it will end up to /var/log/modsec_audit.log. But the request itself is passed as 200 HTTP OK.

zetaab · 2020-01-29T08:51:19Z

When I check generated nginx.conf (version 0.28.0) it looks following:

                        modsecurity on;

                        modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;

                        modsecurity_rules '
                        SecRuleEngine On
                        SecRequestBodyAccess On
                        SecAuditEngine RelevantOnly
                        SecAuditLogParts ABIJDEFHZ
                        SecAuditLog /var/log/modsec_audit.log
                        Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf

                        ';

                        modsecurity_transaction_id "$request_id";

And when I check /etc/nginx/modsecurity/modsecurity.conf it says SecRuleEngine DetectionOnly

When I compare this to 0.24.1 which is working (returning 403).

			modsecurity on;

			modsecurity_rules '
			SecRuleEngine On
			SecRequestBodyAccess On
			SecAuditEngine RelevantOnly
			SecAuditLogParts ABIJDEFHZ
			SecAuditLog /var/log/modsec_audit.log
			Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf

			';

So the difference in newer versions is that modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf; is loaded.

zetaab · 2020-01-29T09:05:17Z

@MRoci as you have contributed this change in #4080 can you please confirm that it is not breaking this whole thing

MRoci · 2020-01-31T11:32:24Z

@MRoci as you have contributed this change in #4080 can you please confirm that it is not breaking this whole thing

@zetaab reverting this could reintroduce #3585
Because owasp-csr alone doesn't do all the required cofiguration work to have a working setup.

Maybe we could enable modsecurity.conf when using default config and with owasp-csr and ignore it when using a custom config snippet

pcallewaert · 2020-02-06T16:13:58Z

I've got it working like this (with 0.27.1):

ConfigMap:

enable-modsecurity: "true"

Ingress Annotation:

    nginx.ingress.kubernetes.io/modsecurity-snippet: |
      SecRuleEngine On
      Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf

Now it blocks requests (403 Forbidden) like expected... Hope this helps

Aanuka · 2020-02-06T16:37:11Z

Also got it working in 0.27.1

ConfigMap:

enable-modsecurity: "true"
modsecurity-snippet: |
SecRuleEngine On
SecRequestBodyAccess On
SecAuditEngine RelevantOnly
SecRuleRemoveById 920350
Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf

I had to add the SecRuleRemoveById 920350 to the ConfigMap to not have the /var/log/modsec_audit.log filled with entries for internal request.

zetaab · 2020-02-18T09:11:08Z

I can confirm that @pcallewaert works. However, my use-case is that I would like to have following:

configmap setting that ALL do have this:

  modsecurity-snippet: |
    SecRequestBodyAccess On
    SecAuditEngine RelevantOnly
    SecAuditLogParts ABHKZ
    SecAuditLog http://modsec-audit-deployment.kube-system.svc.cluster.local:9998/audits
    SecAuditLogType HTTPS
    SecAuditLogFormat JSON
    SecRuleRemoveById 920350
    Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf

only selected ingresses could have SecRuleEngine On I do not want turn that on cluster wide, because it will break things.

I have not found way to do this. It is only possible to turn on in global way currently? tl;dr all ingresses are reporting secaudit things, but selected should have forced policy

edit: Cool I got it working:

configmap:

  enable-modsecurity: "true"
  modsecurity-snippet: |
    SecRequestBodyAccess On
    SecAuditEngine RelevantOnly
    SecAuditLogParts ABHKZ
    SecAuditLog http://modsec-audit-deployment.kube-system.svc.cluster.local:9998/audits
    SecAuditLogType HTTPS
    SecAuditLogFormat JSON
    SecRuleRemoveById 920350
    Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf

and then in ingress only this:

    nginx.ingress.kubernetes.io/modsecurity-snippet: |
      SecRuleEngine On

JorgeGuerreroTech · 2020-02-28T14:20:47Z

I've got it working like this (with 0.27.1):

ConfigMap:
enable-modsecurity: "true"
Ingress Annotation:
    nginx.ingress.kubernetes.io/modsecurity-snippet: |
      SecRuleEngine On
      Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
Now it blocks requests (403 Forbidden) like expected... Hope this helps

This work for me!
Thanks!

Hello-Linux · 2020-04-09T10:42:45Z

I'am not working like this (with 0.30): #5343

JorgeGuerreroTech · 2020-04-09T11:16:26Z

I'am not working like this (with 0.30): #5343

Yes, I have same problem when I updated to 0.30 not working in 0.27.1 working fine, this is a problem because I can't update to latest version.
Thanks!

fennellgb23 · 2020-06-11T03:38:01Z

Can confirm as well, not working in v 0.30, is anyone looking into this?

Update: the file /etc/nginx/modsecurity/modsecurity.conf has SecRuleEngine set to DetectionOnly. I edited the file to On and it now works. Is there an easy way to just override this setting since it does not seem us setting this in the Ingress is making any difference.

Update 2: I can confirm that this "DOES" work. After more investigations I found my team changed the configmap for my ingress so I was updating the wrong file. For anyone that lands here, if you 'kubectl edit configmap {name}' the correct configmap and save the changes you will see it working properly.

junaid18183 · 2020-07-09T13:48:53Z

I can confirm Its working on "0.32.0", here is my config -

  config:
    enable-modsecurity: "true"
    enable-owasp-modsecurity-crs: "true"

Ingress Annotation:

  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/modsecurity-snippet: |
      SecRuleEngine On

JorgeGuerreroTech · 2020-07-11T10:02:40Z

Yes, I also confirm Its working on "0.32.0", problem solved! Thanks!!!!
The @junaid18183 works for me!

aledbf closed this as completed Sep 28, 2019

This was referenced Jan 29, 2020

Load modsecurity config with OWASP core rules #4080

Merged

fix modsecurity SecRuleEngine On #4982

Closed

zetaab mentioned this issue Feb 18, 2020

Modsecurity can never be enabled in enforced mode #5099

Closed

besha100 mentioned this issue Dec 18, 2021

Disable default modsecurity_rules_file if modsecurity-snippet is specified #8021

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ModSecurity does not block request, only logs, while SecRuleEngine is set to On #4385

ModSecurity does not block request, only logs, while SecRuleEngine is set to On #4385

kwaazaar commented Jul 31, 2019

kwaazaar commented Jul 31, 2019

Aanuka commented Aug 9, 2019

mhauer71 commented Aug 14, 2019

Aanuka commented Sep 16, 2019

kwaazaar commented Sep 17, 2019

aledbf commented Sep 17, 2019

Aanuka commented Sep 17, 2019

aledbf commented Sep 18, 2019

Aanuka commented Sep 19, 2019

celamb4 commented Sep 21, 2019 •

edited

Loading

aledbf commented Sep 28, 2019

Aanuka commented Sep 28, 2019

reddare commented Oct 9, 2019 •

edited

Loading

Aanuka commented Nov 11, 2019

Keralin commented Nov 25, 2019

kwaazaar commented Nov 25, 2019

zetaab commented Jan 29, 2020 •

edited

Loading

zetaab commented Jan 29, 2020 •

edited

Loading

zetaab commented Jan 29, 2020

MRoci commented Jan 31, 2020 •

edited

Loading

pcallewaert commented Feb 6, 2020

Aanuka commented Feb 6, 2020

zetaab commented Feb 18, 2020 •

edited

Loading

JorgeGuerreroTech commented Feb 28, 2020

Hello-Linux commented Apr 9, 2020

JorgeGuerreroTech commented Apr 9, 2020

fennellgb23 commented Jun 11, 2020 •

edited

Loading

junaid18183 commented Jul 9, 2020 •

edited

Loading

JorgeGuerreroTech commented Jul 11, 2020

ModSecurity does not block request, only logs, while SecRuleEngine is set to On #4385

ModSecurity does not block request, only logs, while SecRuleEngine is set to On #4385

Comments

kwaazaar commented Jul 31, 2019

kwaazaar commented Jul 31, 2019

Aanuka commented Aug 9, 2019

mhauer71 commented Aug 14, 2019

Aanuka commented Sep 16, 2019

kwaazaar commented Sep 17, 2019

aledbf commented Sep 17, 2019

Aanuka commented Sep 17, 2019

aledbf commented Sep 18, 2019

Aanuka commented Sep 19, 2019

celamb4 commented Sep 21, 2019 • edited Loading

aledbf commented Sep 28, 2019

Aanuka commented Sep 28, 2019

reddare commented Oct 9, 2019 • edited Loading

Aanuka commented Nov 11, 2019

Keralin commented Nov 25, 2019

kwaazaar commented Nov 25, 2019

zetaab commented Jan 29, 2020 • edited Loading

zetaab commented Jan 29, 2020 • edited Loading

zetaab commented Jan 29, 2020

MRoci commented Jan 31, 2020 • edited Loading

pcallewaert commented Feb 6, 2020

Aanuka commented Feb 6, 2020

zetaab commented Feb 18, 2020 • edited Loading

JorgeGuerreroTech commented Feb 28, 2020

Hello-Linux commented Apr 9, 2020

JorgeGuerreroTech commented Apr 9, 2020

fennellgb23 commented Jun 11, 2020 • edited Loading

junaid18183 commented Jul 9, 2020 • edited Loading

JorgeGuerreroTech commented Jul 11, 2020

celamb4 commented Sep 21, 2019 •

edited

Loading

reddare commented Oct 9, 2019 •

edited

Loading

zetaab commented Jan 29, 2020 •

edited

Loading

zetaab commented Jan 29, 2020 •

edited

Loading

MRoci commented Jan 31, 2020 •

edited

Loading

zetaab commented Feb 18, 2020 •

edited

Loading

fennellgb23 commented Jun 11, 2020 •

edited

Loading

junaid18183 commented Jul 9, 2020 •

edited

Loading