Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PiHole on docker won't start anymore with the latest version #794

Closed
devedse opened this issue Feb 16, 2021 · 36 comments
Closed

PiHole on docker won't start anymore with the latest version #794

devedse opened this issue Feb 16, 2021 · 36 comments
Assignees
Labels

Comments

@devedse
Copy link

devedse commented Feb 16, 2021

Versions

Pi-hole version is v5.2.4 (Latest: v5.2.4)
AdminLTE version is v5.4 (Latest: v5.4)
FTL version is v5.7 (Latest: v5.7)

My watchtower just automatically updated my PiHole running on a raspberry pi:

time="2021-02-16T20:41:57Z" level=info msg="Found new pihole/pihole:latest image (sha256:a2eef2ddff91c7117eacfcfb6927ea56d5dd51291c2282e66d1fca4d7b2ba5ce)"

time="2021-02-16T20:42:40Z" level=info msg="Stopping /Pihole (3e1fae648dfb6d3bbadc5aa28017cfd77248012a5fa08191900662c07bfdb9ed) with SIGTERM"

time="2021-02-16T20:42:46Z" level=info msg="Creating /Pihole"

However after the PiHole containers shows up as Unhealthy. Here's the container log:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.

[s6-init] ensuring user provided files have correct perms...exited 0.

[fix-attrs.d] applying ownership & permissions fixes...

[fix-attrs.d] 01-resolver-resolv: applying... 

[fix-attrs.d] 01-resolver-resolv: exited 0.

[fix-attrs.d] done.

[cont-init.d] executing container initialization scripts...

[cont-init.d] 20-start.sh: executing... 

 ::: Starting docker specific checks & setup for docker pihole/pihole


  [i] Installing configs from /etc/.pihole...

  [i] Existing dnsmasq.conf found... it is not a Pi-hole file, leaving alone!

  [i] Copying 01-pihole.conf to /etc/dnsmasq.d/01-pihole.conf...
  [✓] Copying 01-pihole.conf to /etc/dnsmasq.d/01-pihole.conf

Converting DNS1 to PIHOLE_DNS_

Setting DNS servers based on PIHOLE_DNS_ variable

::: Pre existing WEBPASSWORD found

DNSMasq binding to default interface: eth0

Added ENV to php:

			"PHP_ERROR_LOG" => "/var/log/lighttpd/error.log",

			"ServerIP" => "0.0.0.0",

			"VIRTUAL_HOST" => "0.0.0.0",

Using IPv4 and IPv6

::: Preexisting ad list /etc/pihole/adlists.list detected ((exiting setup_blocklists early))

https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

https://mirror1.malwaredomains.com/files/justdomains

::: Testing pihole-FTL DNS: FTL started!

::: Testing lighttpd config: Syntax OK

::: All config checks passed, cleared for startup ...

::: Enabling Query Logging

  [i] Enabling logging...


  [✓] Logging has been enabled!

 ::: Docker start setup complete

  [i] Neutrino emissions detected...


  [✓] Pulling blocklist source list into range


  [i] Preparing new gravity database...
  [✓] Preparing new gravity database

  [i] Using libz compression


  [i] Target: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

  [i] Status: Pending...
  [✓] Status: Retrieval successful

  [i] Received 60887 domains


  [i] Target: https://mirror1.malwaredomains.com/files/justdomains

  [i] Status: Pending...
  [✗] Status: Not found

  [✗] List download failed: using previously cached list

  [i] Received 26854 domains


  [i] Storing downloaded domains in new gravity database...
  [✓] Storing downloaded domains in new gravity database

  [i] Building tree...
  [✓] Building tree

  [i] Swapping databases...
  [✓] Swapping databases

  [i] Number of gravity domains: 87741 (87713 unique domains)

  [i] Number of exact blacklisted domains: 0

  [i] Number of regex blacklist filters: 0

  [i] Number of exact whitelisted domains: 1

  [i] Number of regex whitelist filters: 0

  [i] Cleaning up stray matter...
  [✓] Cleaning up stray matter


  [✓] DNS service is listening

     [✓] UDP (IPv4)

     [✓] TCP (IPv4)

     [✓] UDP (IPv6)

     [✓] TCP (IPv6)


  [✓] Pi-hole blocking is enabled

  Pi-hole version is v5.2.4 (Latest: v5.2.4)

  AdminLTE version is v5.4 (Latest: v5.4)

  FTL version is v5.7 (Latest: v5.7)

[cont-init.d] 20-start.sh: exited 0.

[cont-init.d] done.

[services.d] starting services

Starting pihole-FTL (no-daemon) as root

Starting lighttpd

Starting crond

[services.d] done.

Stopping pihole-FTL

kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

Starting pihole-FTL (no-daemon) as root

Stopping pihole-FTL

kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

Starting pihole-FTL (no-daemon) as root

Stopping pihole-FTL

kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

Starting pihole-FTL (no-daemon) as root

Stopping pihole-FTL

kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

Starting pihole-FTL (no-daemon) as root

Stopping pihole-FTL

kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

Starting pihole-FTL (no-daemon) as root

Stopping pihole-FTL

Platform

  • OS and version: Linux dockerpi 5.4.83-v7+
  • Platform: Raspberry PI

Expected behavior

PiHole should start correctly

Actual behavior / bug

PiHole FTL service won't start

Steps to reproduce

Steps to reproduce the behavior:

  1. Update to the latest version

Edit

I've also checked the FTL log found by executing cat /var/log/pihole-FTL.log inside the docker container. This is the output:

[2021-02-16 22:31:06.202 7444M] ########## FTL started! ##########
[2021-02-16 22:31:06.203 7444M] FTL branch: master
[2021-02-16 22:31:06.209 7444M] FTL version: v5.7
[2021-02-16 22:31:06.210 7444M] FTL commit: 2999e2b5
[2021-02-16 22:31:06.210 7444M] FTL date: 2021-02-16 19:36:43 +0000
[2021-02-16 22:31:06.210 7444M] FTL user: root
[2021-02-16 22:31:06.210 7444M] Compiled for armv7hf (compiled on CI) using arm-linux-gnueabihf-gcc (Debian 6.3.0-18) 6.3.0 20170516
[2021-02-16 22:31:06.211 7444M] FATAL: create_shm(): Failed to create shared memory object "FTL-lock": File exists
[2021-02-16 22:31:06.211 7444M] Initialization of shared memory failed.
@opicron
Copy link

opicron commented Feb 16, 2021

Unfortunately it happens here too. Definately something is wrong with latest release.

Running on Synology Docker in DSM 6.2

@Beanow
Copy link

Beanow commented Feb 16, 2021

Can confirm for amd64 and arm64 platforms.

Same loop of

Starting pihole-FTL (no-daemon) as root
Stopping pihole-FTL
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

@dschaper
Copy link
Member

"Me too" "Same Here" and 👍🏻 comments don't help at all so please just use the icons for the original post.

@TobenderZephyr
Copy link

TobenderZephyr commented Feb 16, 2021

Can confirm. I researched how to load an old image in Docker and couldn't find any help.
If anyone else needs a helping hand here (as a workaround):
sudo docker image ls, look out for your old pihole where the tag is unset. Note down the id and insert it into your docker run command or docker-compose.yml.

As @jgeusebroek mentioned, you could use pihole/pihole:v5.6 instead. Since my Pi uses pihole itself, I was unable to "download" the target image.

Sorry if this isn't helpful for the devs

@jsuelwald
Copy link

Can confirm, same here.
Docker-Container was updated by watchtower - didn't start afterwards.

docker exec -it pihole2 bash
root@d289e0149195:/# /usr/bin/pihole-FTL
bash: /usr/bin/pihole-FTL: Operation not permitted

@jgeusebroek
Copy link

pihole/pihole:v5.6 is the previous build. FYI.

@jsuelwald
Copy link

pihole/pihole:v5.6 is the previous build. FYI.
Yeah, this works perfectly - will keep using this until this issue is fixed

@Beanow
Copy link

Beanow commented Feb 16, 2021

Could this be caused by pi-hole/FTL@49ba60e?
Could we be running multiple instance during our init?

@gergnz
Copy link

gergnz commented Feb 16, 2021

This workaround worked for me (assuming your docker container name is pihole):

docker exec -it pihole /bin/bash
rm /dev/shm/FTL*

@DL6ER
Copy link
Member

DL6ER commented Feb 16, 2021

@PromoFaux The workaround by @gergnz suggests that /dev/shm isn't clean in the docker image. Could you verify this?

@Beanow
Copy link

Beanow commented Feb 16, 2021

I believe that as of that commit: pi-hole/FTL@49ba60e
It will crash when those files mentioned by @gergnz are present (/dev/shm/FTL*)

Also we're using kill -9 <pid> to shut down FTL. So it won't have the opportunity to clean up these files normally.
The main culprit being in /etc/cont-init.d/20-start.sh because it always runs before the service tries to start.

# Kill dnsmasq because s6 won't like it if it's running when s6 services start
kill -9 $(pgrep pihole-FTL) || true

TL;DR we never did "clean" shutdowns. But since FLT v5.7 it's become necessary to do so.

@Beanow
Copy link

Beanow commented Feb 16, 2021

@DL6ER I've just checked that the image doesn't have /dev/shm/FTL* in the container image.
It's populated due to the /20-start.sh init script not shutting down cleanly.

@DL6ER
Copy link
Member

DL6ER commented Feb 16, 2021

Thanks. I'm not at all in how this container works, so I was just assuming what may be going wrong. Thanks for the research!

Whatever the solution will be, kill -9 shouldn't be part of it (anywhere!).

@Beanow
Copy link

Beanow commented Feb 16, 2021

To confirm, the image doesn't contain lockfiles.

docker run --rm -ti --entrypoint="" pihole/pihole:v5.7 ls -la /dev/shm
total 0
drwxrwxrwt 2 root root  40 Feb 16 22:15 .
drwxr-xr-x 5 root root 360 Feb 16 22:15 ..

But if we look at this dir at the end of the init script it's apparently unclean.

# Dockerfile
FROM pihole/pihole:v5.7
RUN echo "ls -la /dev/shm" >> /etc/cont-init.d/20-start.sh
docker build -t pihole-locking .
docker run --rm -ti pihole-locking exit 0
...
  [✓] Flushing DNS cache
  [✓] Pi-hole Enabled
  Pi-hole version is v5.2.4 (Latest: v5.2.4)
  AdminLTE version is v5.4 (Latest: v5.4)
  FTL version is v5.7 (Latest: v5.7)
total 768
drwxrwxrwt 2 root root    260 Feb 16 22:13 .
drwxr-xr-x 5 root root    360 Feb 16 22:13 ..
-rw------- 1 root root 356352 Feb 16 22:13 FTL-clients
-rw------- 1 root root    152 Feb 16 22:13 FTL-counters
-rw------- 1 root root   4096 Feb 16 22:13 FTL-dns-cache
-rw------- 1 root root  98304 Feb 16 22:13 FTL-domains
-rw------- 1 root root     48 Feb 16 22:13 FTL-lock
-rw------- 1 root root  24576 Feb 16 22:13 FTL-overTime
-rw------- 1 root root   4096 Feb 16 22:13 FTL-per-client-regex
-rw------- 1 root root 262144 Feb 16 22:13 FTL-queries
-rw------- 1 root root     12 Feb 16 22:13 FTL-settings
-rw------- 1 root root   4096 Feb 16 22:13 FTL-strings
-rw------- 1 root root  20480 Feb 16 22:13 FTL-upstreams
[cont-init.d] 20-start.sh: exited 0.

@DL6ER
Copy link
Member

DL6ER commented Feb 16, 2021

I should add: Cleaning up behind you is somewhat good practive and killing a process hard (without giving it even the chance to clean up properly behind itself) is never.

FTL tries to open the shared memory objects and only fails with File exists when this is not possible (in exclusive mode). FTL refuses to start as there may someone/-thing else using these files which would lead to serious issues if we'd start using them as well now.

This is actually a good thing, just the logic in the docker container has to be improved. I'm not familiar at all, but maybe the logic can be changed from

  1. Starting
  2. Killing
  3. Staring again

To only start when we actually want it.

@PromoFaux
Copy link
Member

OK, what appears to work (as a quickfix here) is adding rm /dev/shm/FTL* before the kill -9 call in both finish and 20-start.sh

@binhton
Copy link

binhton commented Feb 16, 2021

This workaround worked for me (assuming your docker container name is pihole):

docker exec -it pihole /bin/bash
rm /dev/shm/FTL*

This works for me

@Beanow
Copy link

Beanow commented Feb 16, 2021

I've been trying out a non-kill-9 approach, but it appears to be flaky.

# Kill dnsmasq because s6 won't like it if it's running when s6 services start
FTLPID=$(pgrep pihole-FTL)
kill ${FTLPID} || true

while kill -0 ${FTLPID}; do
    echo "FTL ${FTLPID} still running..."
    sleep 1
done

while [ -e "/dev/shm/FTL-lock" ]; do
    echo "Lock file still exists..."
    sleep 1
done

pihole -v

Usually this works as expected.

  [i] Pi-hole blocking will be enabled
  [i] Enabling blocking
  [✓] Flushing DNS cache
  [✓] Pi-hole Enabled
FTL 348 still running...
/var/run/s6/etc/cont-init.d/20-start.sh: line 27: kill: (348) - No such process
  Pi-hole version is v5.2.4 (Latest: v5.2.4)
  AdminLTE version is v5.4 (Latest: v5.4)
  FTL version is v5.7 (Latest: v5.7)

But occasionally it will not clean up.

  [i] Pi-hole blocking will be enabled
  [i] Enabling blocking
  [✓] Flushing DNS cache
  [✓] Pi-hole Enabled
FTL 348 still running...
/var/run/s6/etc/cont-init.d/20-start.sh: line 27: kill: (348) - No such process
Lock file still exists...
Lock file still exists...
Lock file still exists...
Lock file still exists...
Lock file still exists...
Lock file still exists...
Lock file still exists...
Lock file still exists...
Lock file still exists...
Lock file still exists...

So possibly a normal terminate signal still leaks the files?

@PromoFaux
Copy link
Member

PromoFaux commented Feb 16, 2021

Also adding rm /dev/shm/FTL* before s6-setuidgid in the run script works.

Edit, in fact that's more or less what happens on a bare metal instance

https://github.com/pi-hole/pi-hole/blob/master/advanced/Templates/pihole-FTL.service#L25-L41

@DL6ER
Copy link
Member

DL6ER commented Feb 16, 2021

So possibly a normal terminate signal still leaks the files?

Shouldn't

https://github.com/pi-hole/FTL/blob/2999e2b57c62b4455187ee9b77840d49df0a8e2e/src/main.c#L127

https://github.com/pi-hole/FTL/blob/2999e2b57c62b4455187ee9b77840d49df0a8e2e/src/shmem.c#L515-L531

https://github.com/pi-hole/FTL/blob/2999e2b57c62b4455187ee9b77840d49df0a8e2e/src/shmem.c#L743-L755

And man shm_unlink says:

  The  operation  of shm_unlink() is analogous to unlink(2): it removes a
  shared memory object name, and, once all processes  have  unmapped  the
  object, de-allocates and destroys the contents of the associated memory
  region.  After a successful shm_unlink(), attempts to shm_open() an ob‐
  ject  with  the  same name fail (unless O_CREAT was specified, in which
  case a new, distinct object is created).

@pralor-bot
Copy link

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/docker-update-to-v5-7-causes-ftl-to-crash-at-launch/44464/3

@Beanow
Copy link

Beanow commented Feb 16, 2021

So possibly a normal terminate signal still leaks the files?

Shouldn't

https://github.com/pi-hole/FTL/blob/2999e2b57c62b4455187ee9b77840d49df0a8e2e/src/main.c#L127

https://github.com/pi-hole/FTL/blob/2999e2b57c62b4455187ee9b77840d49df0a8e2e/src/shmem.c#L515-L531

https://github.com/pi-hole/FTL/blob/2999e2b57c62b4455187ee9b77840d49df0a8e2e/src/shmem.c#L743-L755

And man shm_unlink says:

  The  operation  of shm_unlink() is analogous to unlink(2): it removes a
  shared memory object name, and, once all processes  have  unmapped  the
  object, de-allocates and destroys the contents of the associated memory
  region.  After a successful shm_unlink(), attempts to shm_open() an ob‐
  ject  with  the  same name fail (unless O_CREAT was specified, in which
  case a new, distinct object is created).

Nope @DL6ER I'm pretty sure it does, but seems unrelated to the kill -9 problem.
When I do have the situation when the normal kill still leaves files, this is in /var/log/pihole-FTL.log

[2021-02-16 23:07:34.161 349M] Received signal: Segmentation fault
[2021-02-16 23:07:34.161 349M]      at address: 0x7f8831f0c9d0
[2021-02-16 23:07:34.161 349M]      with code:  SEGV_MAPERR (Address not mapped to object)
Full log
[2021-02-16 23:07:33.191 347M] ########## FTL started! ##########
[2021-02-16 23:07:33.191 347M] FTL branch: master
[2021-02-16 23:07:33.191 347M] FTL version: v5.7
[2021-02-16 23:07:33.191 347M] FTL commit: 2999e2b5
[2021-02-16 23:07:33.191 347M] FTL date: 2021-02-16 19:36:43 +0000
[2021-02-16 23:07:33.191 347M] FTL user: root
[2021-02-16 23:07:33.192 347M] Compiled for x86_64 (compiled on CI) using gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
[2021-02-16 23:07:33.192 347M] Creating mutex
[2021-02-16 23:07:33.192 347M] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2021-02-16 23:07:33.192 347M]    SOCKET_LISTENING: only local
[2021-02-16 23:07:33.192 347M]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2021-02-16 23:07:33.192 347M]    MAXDBDAYS: max age for stored queries is 365 days
[2021-02-16 23:07:33.192 347M]    RESOLVE_IPV6: Resolve IPv6 addresses
[2021-02-16 23:07:33.192 347M]    RESOLVE_IPV4: Resolve IPv4 addresses
[2021-02-16 23:07:33.192 347M]    DBINTERVAL: saving to DB file every minute
[2021-02-16 23:07:33.192 347M]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2021-02-16 23:07:33.192 347M]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2021-02-16 23:07:33.192 347M]    PRIVACYLEVEL: Set to 0
[2021-02-16 23:07:33.192 347M]    IGNORE_LOCALHOST: Show queries from localhost
[2021-02-16 23:07:33.192 347M]    BLOCKINGMODE: Null IPs for blocked domains
[2021-02-16 23:07:33.192 347M]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2021-02-16 23:07:33.192 347M]    DBIMPORT: Importing history from database
[2021-02-16 23:07:33.192 347M]    PIDFILE: Using /run/pihole-FTL.pid
[2021-02-16 23:07:33.192 347M]    PORTFILE: Using /run/pihole-FTL.port
[2021-02-16 23:07:33.192 347M]    SOCKETFILE: Using /run/pihole/FTL.sock
[2021-02-16 23:07:33.192 347M]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2021-02-16 23:07:33.192 347M]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2021-02-16 23:07:33.192 347M]    GRAVITYDB: Using /etc/pihole/gravity.db
[2021-02-16 23:07:33.192 347M]    PARSE_ARP_CACHE: Active
[2021-02-16 23:07:33.192 347M]    CNAME_DEEP_INSPECT: Active
[2021-02-16 23:07:33.192 347M]    DELAY_STARTUP: No delay requested.
[2021-02-16 23:07:33.192 347M]    BLOCK_ESNI: Enabled, blocking _esni.{blocked domain}
[2021-02-16 23:07:33.192 347M]    NICE: Cannot change niceness to -10 (permission denied)
[2021-02-16 23:07:33.192 347M]    MAXNETAGE: Removing IP addresses and host names from network table after 365 days
[2021-02-16 23:07:33.192 347M]    NAMES_FROM_NETDB: Enabled, trying to get names from network database
[2021-02-16 23:07:33.192 347M]    EDNS0_ECS: Overwrite client from ECS information
[2021-02-16 23:07:33.192 347M]    REFRESH_HOSTNAMES: Periodically refreshing IPv4 names
[2021-02-16 23:07:33.192 347M]    RATE_LIMIT: Rate-limiting client making more than 1000 queries in 60 seconds
[2021-02-16 23:07:33.192 347M] Finished config file parsing
[2021-02-16 23:07:33.192 347M] WARNING: Starting pihole-FTL as user root is not recommended
[2021-02-16 23:07:33.192 347M] No database file found, creating new (empty) database
[2021-02-16 23:07:33.202 347M] Database version is 1
[2021-02-16 23:07:33.202 347M] Updating long-term database to version 2
[2021-02-16 23:07:33.211 347M] Updating long-term database to version 3
[2021-02-16 23:07:33.214 347M] Updating long-term database to version 4
[2021-02-16 23:07:33.216 347M] Updating long-term database to version 5
[2021-02-16 23:07:33.218 347M] Updating long-term database to version 6
[2021-02-16 23:07:33.224 347M] Updating long-term database to version 7
[2021-02-16 23:07:33.232 347M] Updating long-term database to version 8
[2021-02-16 23:07:33.235 347M] Updating long-term database to version 9
[2021-02-16 23:07:33.240 347M] Imported 0 alias-clients
[2021-02-16 23:07:33.240 347M] Database successfully initialized
[2021-02-16 23:07:33.240 347M] Imported 0 queries from the long-term database
[2021-02-16 23:07:33.240 347M]  -> Total DNS queries: 0
[2021-02-16 23:07:33.240 347M]  -> Cached DNS queries: 0
[2021-02-16 23:07:33.240 347M]  -> Forwarded DNS queries: 0
[2021-02-16 23:07:33.240 347M]  -> Blocked DNS queries: 0
[2021-02-16 23:07:33.240 347M]  -> Unknown DNS queries: 0
[2021-02-16 23:07:33.240 347M]  -> Unique domains: 0
[2021-02-16 23:07:33.240 347M]  -> Unique clients: 0
[2021-02-16 23:07:33.240 347M]  -> Known forward destinations: 0
[2021-02-16 23:07:33.240 347M] Successfully accessed setupVars.conf
[2021-02-16 23:07:33.240 347M] *************************************************************************
[2021-02-16 23:07:33.240 347M] * WARNING: Required Linux capability CAP_NET_ADMIN not available        *
[2021-02-16 23:07:33.240 347M] *************************************************************************
[2021-02-16 23:07:33.240 347M] *************************************************************************
[2021-02-16 23:07:33.240 347M] * WARNING: Required Linux capability CAP_SYS_NICE not available         *
[2021-02-16 23:07:33.240 347M] *************************************************************************
[2021-02-16 23:07:33.243 349M] PID of FTL process: 349
[2021-02-16 23:07:33.243 349/T350] Listening on port 4711 for incoming IPv4 telnet connections
[2021-02-16 23:07:33.243 349M] INFO: FTL is running as root
[2021-02-16 23:07:33.243 349/T352] Listening on Unix socket
[2021-02-16 23:07:33.245 349M] Reloading DNS cache
[2021-02-16 23:07:33.245 349M] Blocking status is enabled
[2021-02-16 23:07:33.722 349M] Reloading DNS cache
[2021-02-16 23:07:33.722 349M] Blocking status is enabled
[2021-02-16 23:07:34.158 349M] Reloading DNS cache
[2021-02-16 23:07:34.158 349M] Blocking status is enabled
[2021-02-16 23:07:34.161 349M] Shutting down...
[2021-02-16 23:07:34.161 349M] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2021-02-16 23:07:34.161 349M] ---------------------------->  FTL crashed!  <----------------------------
[2021-02-16 23:07:34.161 349M] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2021-02-16 23:07:34.161 349M] Please report a bug at https://github.com/pi-hole/FTL/issues
[2021-02-16 23:07:34.161 349M] and include in your report already the following details:
[2021-02-16 23:07:34.161 349M] FTL has been running for 1 seconds
[2021-02-16 23:07:34.161 349M] FTL branch: master
[2021-02-16 23:07:34.161 349M] FTL version: v5.7
[2021-02-16 23:07:34.161 349M] FTL commit: 2999e2b5
[2021-02-16 23:07:34.161 349M] FTL date: 2021-02-16 19:36:43 +0000
[2021-02-16 23:07:34.161 349M] FTL user: started as root, ended as root
[2021-02-16 23:07:34.161 349M] Compiled for x86_64 (compiled on CI) using gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
[2021-02-16 23:07:34.161 349M] Process details: MID: 349
[2021-02-16 23:07:34.161 349M]                  PID: 349
[2021-02-16 23:07:34.161 349M]                  TID: 349
[2021-02-16 23:07:34.161 349M]                  Name: pihole-FTL
[2021-02-16 23:07:34.161 349M] Received signal: Segmentation fault
[2021-02-16 23:07:34.161 349M]      at address: 0x7f8831f0c9d0
[2021-02-16 23:07:34.161 349M]      with code:  SEGV_MAPERR (Address not mapped to object)
[2021-02-16 23:07:34.161 349M] Backtrace:
[2021-02-16 23:07:34.161 349M] B[0000]: pihole-FTL(+0x595a5) [0x55665e0ec5a5]
[2021-02-16 23:07:34.162 349M] L[0000]: N/A (0x595a5)
[2021-02-16 23:07:34.162 349M] B[0001]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730) [0x7f88322cc730]
[2021-02-16 23:07:34.162 349M] B[0002]: /lib/x86_64-linux-gnu/libpthread.so.0(pthread_cancel+0x9) [0x7f88322c96c9]
[2021-02-16 23:07:34.162 349M] B[0003]: pihole-FTL(main+0x22a) [0x55665e0d7aaa]
[2021-02-16 23:07:34.162 349M] L[0003]: N/A (0x44aaa)
[2021-02-16 23:07:34.162 349M] B[0004]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f883211d09b]
[2021-02-16 23:07:34.162 349M] B[0005]: pihole-FTL(_start+0x2a) [0x55665e0d7b2a]
[2021-02-16 23:07:34.163 349M] L[0005]: N/A (0x44b2a)
[2021-02-16 23:07:34.163 349M] ------ Listing content of directory /dev/shm ------
[2021-02-16 23:07:34.163 349M] File Mode User:Group  Filesize Filename
[2021-02-16 23:07:34.163 349M] rwxrwxrwx root:root 260 .
[2021-02-16 23:07:34.163 349M] rwxr-xr-x root:root 360 ..
[2021-02-16 23:07:34.163 349M] rw------- root:root 4K FTL-per-client-regex
[2021-02-16 23:07:34.163 349M] rw------- root:root 4K FTL-dns-cache
[2021-02-16 23:07:34.163 349M] rw------- root:root 25K FTL-overTime
[2021-02-16 23:07:34.163 349M] rw------- root:root 262K FTL-queries
[2021-02-16 23:07:34.163 349M] rw------- root:root 20K FTL-upstreams
[2021-02-16 23:07:34.163 349M] rw------- root:root 356K FTL-clients
[2021-02-16 23:07:34.163 349M] rw------- root:root 98K FTL-domains
[2021-02-16 23:07:34.163 349M] rw------- root:root 4K FTL-strings
[2021-02-16 23:07:34.163 349M] rw------- root:root 12 FTL-settings
[2021-02-16 23:07:34.163 349M] rw------- root:root 152 FTL-counters
[2021-02-16 23:07:34.163 349M] rw------- root:root 48 FTL-lock
[2021-02-16 23:07:34.163 349M] ---------------------------------------------------
[2021-02-16 23:07:34.163 349M] Please also include some lines from above the !!!!!!!!! header.
[2021-02-16 23:07:34.163 349M] Thank you for helping us to improve our FTL engine!
[2021-02-16 23:07:34.163 349M] FTL terminated!

@apveening

This comment has been minimized.

@cikeZ00

This comment has been minimized.

@dschaper

This comment has been minimized.

@dschaper

This comment has been minimized.

@thedanbob
Copy link

I can confirm that #797 fixes this issue.

@PromoFaux
Copy link
Member

Thanks @thedanbob, I've just pulled :v5.7 locally and also working

Everyone here should now be able to re-pull :latest or :v5.7 and be up and going. Apologies all for making things fall over!
(this is also why I don't personally let anything auto update - us dev types can cause havok somtimes! 😉)

@devedse
Copy link
Author

devedse commented Feb 16, 2021

My watchtower just updated PiHole to the latest version and I can confirm this issue is now resolved 😄

@pralor-bot
Copy link

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/docker-update-to-v5-7-causes-ftl-to-crash-at-launch/44464/4

@apveening
Copy link

My watchtower just updated PiHole to the latest version and I can confirm this issue is now resolved 😄

I can confirm that, manual update of one (the problematic one) worked, automatic update with WatchTower of the other one worked as well (after resuming WatchTower from pause).

@pralor-bot
Copy link

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/pi-hole-wont-start-after-docker-update/44454/8

@trolli-ch
Copy link

Thanks for fixing the problem. Works now in my Docker with latest version!

@4n3tcqCkckaRFn
Copy link

Hi Guys,

I have the same issue with the lastes release. Can't find how to solve it. Pleas help.

Lg

@xeion
Copy link

xeion commented Jan 13, 2022

I am new to pihole and was installing it on an odroid with docker-compose, I also had this issue with pihole-FTL

[services.d] done.
Stopping pihole-FTL
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
Starting pihole-FTL (no-daemon) as pihole
Stopping pihole-FTL
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

My problem was that I had enabled the log volume
- './var-log/pihole.log:/var/log/pihole.log'
but I had missed to create the 'var-log' directory and touch './var-log/pihole.log'

So if anyone made the same mistake,
sudo mkdir ./var-log
sudo touch ./var-log/pihole.log

then start the with docker-compose and it works fine :) I also moved all mounts to '/opt/pihole'...

@PromoFaux
Copy link
Member

My problem was that I had enabled the log volume

  • './var-log/pihole.log:/var/log/pihole.log'
    but I had missed to create the 'var-log' directory and touch './var-log/pihole.log'

I've actually removed this from the example file - because it's probably an unnecessary mount, and people often miss the part to create it first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests