Error: Failed to create new OS thread #1114
Comments
|
This is weird, I have never seen a panic like this before. Are you sure systemd captured the full and complete output? Usually panics have a message of some sort at the top (a header if you will). It's really important to get that message so we know what went wrong, otherwise there's almost no hope of fixing it. Edit: Your logs say they begin on March 24... is that really the whole thing? |
|
Turns out i used |
coderobe
changed the title
coderobe
changed the title
|
Thanks - that's what we needed! It's bizarre though, never seen an error like this. Looks like a syscall failed epically. |
|
|
|
Actually try raising your |
|
Weird. |
|
Okay. Well I am going to close this since it doesn't look like a bug in Caddy (rather, a system misconfiguration), but I hope you are able to figure it out! (Try running without systemd if you aren't confident it is tuning the environment properly, maybe?) If you find out, post the answer here. Further discussion welcome. |
|
Just had the same error |
|
I had this exact same issue last night. I've been using recently caddy to run as a frontend proxy to my docker containers (works/has been working great). It would refuse to restart even when I upped my ulimit. Only a reboot would resolve this. It seems the culprit was a rogue domain pointing proxying to an invalid backend. Running this entirely through a KVM container with Proxmox. VPS Running Caddy: Any help would be greatly appreciated. I have not started using caddy for production but would hate to have this issue happen once I have live production sites running. Thanks again! |
What do you mean by this exactly? |
Interesting, so we should be able to reproduce this with any health_check path defined wrong? |
|
Actually it for sure has to do with the ulimit open files. I increased and it hasn't had an issue yet. I didn't think I was doing too much on the system (8 docker containers & caddy) but it seems to have eaten up all 50,000 openfiles. |
|
That is too many I think. @doughnet Can you give me more details? What do you mean by "invalid backend" -- connection failed entirely or the backend returned something like 404? |
|
@mholt This would be a docker container that is running php-fpm & apache for example. It runs smoothly if I only have 7 docker containers running + caddy but once I add an 8th it will give me a 502 error and caddy will fail to start due to the same error of: Restarting the system is about the only way I am able to revive it or stopping docker, and then starting caddy first then starting docker and then removing the extra container. I'm not a huge KVM/virtualization guru so it could be an issue with virtualization(s) but I figured I'd provide the resources I am using (my initial reply) in case it flagged someone. |
|
Thanks @doughnet, for the details! I wonder if it happens without virtualization/containerization. |
|
I'm not using docker but the host is a virtual machine. My setup was a proxy to 10 endpoints with health check. Just tried to replicate the error on my local dev machine but no luck. |
|
@zwack |
|
I run as www-data and didn't have the issue anymore after using the correct endpoint path. Will use your config if it ever happens again though, thanks! |
1. What version of Caddy are you running (
caddy -version)?Caddy 0.9.1 (+e8e5595 Mon Sep 05 15:47:42 UTC 2016)
2. What are you trying to do?
Run Caddy
3. What is your entire Caddyfile?
4. How did you run Caddy (give the full command and describe the execution environment)?
/usr/local/bin/caddy -log stdout -agree=true -conf=/srv/www/Caddyfilestarted via systemd
5. What did you expect to see?
Caddy running normally
6. What did you see instead (give full error messages and/or log)?
7. How can someone who is starting from scratch reproduce this behavior as minimally as possible?
I wasn't able to reproduce the error yet, because it only occurred once. A restart didn't trigger it again.
It was running fine & then suddenly the above happened. Not whilst starting up.
The text was updated successfully, but these errors were encountered: