Openresty throwing error 413: Request entity too large

[bagaderohit]

What happened:

I am running Nginx-ingress helmchart(Version: 4.10.1) on My OKE Kubernetes cluster(V1.29). I am trying to migrate one object having size 10 MB into the cluster but I am getting 413: Request entity too large error from Openresty.

<title>413 Request Entity Too Large</title>

413 Request Entity Too Large

---

openresty

I already have passed the proxy-body-size = "100M" in custom values.yaml while deploying the helmchart and it is getting reflected in nginx.conf successfully. Still I am seeing this error. Any suggesting on any openresty config I need to modify.

What you expected to happen:

The payload should have successfully migrated.

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):

NGINX Ingress controller
Release: v1.10.1
Build: 4fb5aac
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.25.3

---

Kubernetes version (use kubectl version): v1.29

Environment:

• Cloud provider or hardware configuration:

• OS (e.g. from /etc/os-release): Oracle Linux 7.9

• Kernel (e.g. uname -a): 5.15.0-203.146.5.1.el8uek.x86_64 Figure out documentation layout #2 SMP Thu Feb 8 17:14:39 PST 2024 x86_64 x86_64 x86_64 GNU/Linux

• Install tools:

  • Please mention how/where was the cluster created like kubeadm/kops/minikube/kind etc.

• Basic cluster related info:

  • kubectl version
  • kubectl get nodes -o wide

• How was the ingress-nginx-controller installed:

  • If helm was used then please show output of helm ls -A | grep -i ingress
  • If helm was used then please show output of helm -n <ingresscontrollernamespace> get values <helmreleasename>
  • If helm was not used, then copy/paste the complete precise command used to install the controller, along with the flags and options used
  • if you have more than one instance of the ingress-nginx-controller installed in the same cluster, please provide details for all the instances

• Current State of the controller:

  • kubectl describe ingressclasses
  • kubectl -n <ingresscontrollernamespace> get all -A -o wide
  • kubectl -n <ingresscontrollernamespace> describe po <ingresscontrollerpodname>
  • kubectl -n <ingresscontrollernamespace> describe svc <ingresscontrollerservicename>

• Current state of ingress object, if applicable:

  • kubectl -n <appnamespace> get all,ing -o wide
  • kubectl -n <appnamespace> describe ing <ingressname>
  • If applicable, then, your complete and exact curl/grpcurl command (redacted if required) and the reponse to the curl/grpcurl command with the -v flag

• Others:

  • Any other related information like ;
    • copy/paste of the snippet (if applicable)
    • kubectl describe ... of any custom configmap(s) created and in use
    • Any other related information that may help

How to reproduce this issue:

Anything else we need to know:

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[longwuyuan]

/remove-kind bug
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-max-body-size

[longwuyuan]

/kind support

[longwuyuan]

• I deployed this https://artifacthub.io/packages/helm/http-folder/http-folder
• I configured this ingress

% k describe ing my-http-folder 
Name:             my-http-folder
Labels:           <none>
Namespace:        default
Address:          192.168.49.2
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host                        Path  Backends
  ----                        ----  --------
  upload.dev.enjoydevops.com  
                              /   my-http-folder:80 (10.244.0.38:80)
Annotations:                  <none>
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  Sync    10s (x4 over 14m)  nginx-ingress-controller  Scheduled for sync
[~/Downloads] 

• I got this file to try upload

% ls -lth file 
-rw-r----- 1 me me 16M Sep  7 01:09 file
[~/Downloads] 

• Upload failed initially

% curl --data-binary "@file" upload.dev.enjoydevops.com                                                                                                                                  
<html>                                                                                      
<head><title>413 Request Entity Too Large</title></head>                                    
<body>                                                                                      
<center><h1>413 Request Entity Too Large</h1></center>                                      
<hr><center>nginx</center>                                                                  
</body>                                                                                     
</html>                        

• I added the annotation

% k describe ing my-http-folder 
Name:             my-http-folder
Labels:           <none>
Namespace:        default
Address:          192.168.49.2
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host                        Path  Backends
  ----                        ----  --------
  upload.dev.enjoydevops.com  
                              /   my-http-folder:80 (10.244.0.38:80)
Annotations:                  nginx.ingress.kubernetes.io/proxy-body-size: 20m
Events:
  Type    Reason  Age               From                      Message
  ----    ------  ----              ----                      -------
  Normal  Sync    6s (x5 over 20m)  nginx-ingress-controller  Scheduled for sync

• Tried upload again and it worked

% curl --data-binary "@file" upload.dev.enjoydevops.com -i
HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Date: Fri, 06 Sep 2024 19:53:32 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive

[]%                                                                                         
[~/Downloads] 
% curl --data-binary "@file" upload.dev.enjoydevops.com/file-uploaded 
Uploaded succesfully%                                                                                                                                                                    
[~/Downloads] 
% curl upload.dev.enjoydevops.com/                                    
["file-uploaded"]%                                                                                                                                                                       
[~/Downloads] 

So I think the annotation is working and there may be some config error on your ingress

[longwuyuan]

@bagaderohit I will close this issue for now as there is not much to analyze your problem.

Please post a detailed and real test from your cluster along with all details like kubectl describe of related resources, the actual curl command used with -v and response, the logs etc etc, if you want to reopen this issue. That way there will be data to analyze and debug.

/close

[k8s-ci-robot]

@longwuyuan: Closing this issue.

In response to this:

@bagaderohit I will close this issue for now as there is not much to analyze your problem.

Please post a detailed and real test from your cluster along with all details like kubectl describe of related resources, the actual curl command used with -v and response, the logs etc etc, if you want to reopen this issue. That way there will be data to analyze and debug.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[bagaderohit]

HI @longwuyuan! Thanks for taking your time troubleshooting this. However, I have already tested this and the issue I am observing is with the Openresty, please see below:

<title>413 Request Entity Too Large</title>

413 Request Entity Too Large

---

openresty

I am not able to find any reference documentation on how we can configure parameters for openresty. Anyone has came across similar issue, kindly suggest.

[bagaderohit]

@longwuyuan kindly reopen this issue as it still persists.

[longwuyuan]

@bagaderohit what is not clear is how or why that "openresty" aspect relates to this controller. There is no distinct & direct exposure to users for components like Openresty or Nginx (excepting the snippets feature). Hence I closed because normal way to assume reproduce steps did not show problem.

If you posted here to seek comments from others, then I guess there is no action-item on the project.