DietPi-Software | PiHole webstats - Pi-hole stats with DietPi-Cloudshell (v113 milestone)

[k-plan]

@Fourdee

have noticed and tested Update PiHole webstats to current #232

Really nice work, but run into several problems.

• Problem with pi-hole AdminLTE "Query Log" page.
  

Also can be seen in logfile: /error.log

root@oDroide-C1:~# tail -f  /error.log

[Sun Mar 13 12:41:11.049918 2016] [:error] [pid 17238] [client 192.168.100.2:49687] PHP Notice:  Undefined offset: 8 in /var/www/pihole/data.php on line 133, referer: http://192.168.100.55/pihole/queries.php
(... and so on)

(May be have a look at: pi-hole/web#61 )

Fix: https://www.reddit.com/r/pihole/comments/49t4hp/query_log_not_being_parsed_correctly/

• Create a backup copy of your data.php file before editing it.
  :~# cp /var/www/pihole/data.php /var/www/pihole/data.php.bak
• To edit it from the command line, enter:
  :~# nano /var/www/pihole/data.php
  The lines to edit look like this before the change:

substr($exploded[5], 6, -1),
$exploded[6],
$exploded[8],

After the change, they should look like this:

substr($exploded[4], 6, -1),
$exploded[5],
$exploded[7],

Exit and save the file. Restart apache2
:~# service apache2 restart

---

• Problem with pi-hole logfiles, then dietpi Option: DietPi-Ramlog #1 or DietPi-Ramlog #2 is used, because "/var/log/pihole.log" is gone, after 1h.

root@oDroide-C1:~# tail -f  /error.log

[Sun Mar 13 14:17:45.666031 2016] [:error] [pid 6284] [client 192.168.10.2:51319] PHP Warning:  Division by zero in /var/www/pihole/data.php on line 17, referer: http://192.168.100.55/pihole/index.php
(... and so on)

edit: - fix for the log message

But didn't solve the fundamental issue using Ramlog with Pi-hole.
Better to have /var/log/pihole.log synced in Ram or on disk for 1 day.

---

• Error message while refresh and update the block list.

root@oDroide-C1:~# gravity.sh
:::
::: You are root.
cp: der Aufruf von stat für „/etc/.pihole/adlists.default“ ist nicht möglich: Datei oder Verzeichnis nicht gefunden
::: Neutrino emissions detected...

---

• Pi-hole stats with DietPi-Cloudshell: value will not been updated.
  [Regardless of used: Full (Rsyslog & Logrotate) or DietPi-Ramlog #2]
  

[Fourdee]

@k-plan

Fix: https://www.reddit.com/r/pihole/comments/49t4hp/query_log_not_being_parsed_correctly/

👍
I've applied this to the DietPi fork of PiHole webstats. All systems will be patched in v113 update:
https://github.com/Fourdee/AdminLTE/commit/825b39697a645084d12a140133c4f99db7b736cb

[Fourdee]

@k-plan

Error message while refresh and update the block list.
cp: der Aufruf von stat für „/etc/.pihole/adlists.default“ ist nicht möglich: Datei oder Verzeichnis nicht gefunden

/etc/.pihole is generated when using the PiHole installation script (not DietPi).
It appears they use this hidden folder as a tempoary directory to store PiHole files before being applied to the system: https://github.com/pi-hole/pi-hole/blob/master/automated%20install/basic-install.sh#L552-L560.

I had to search for this as they don't use the global variable they generated, that holds the path in their code piholeFilesDir
https://github.com/pi-hole/pi-hole/blob/master/automated%20install/basic-install.sh#L470-L482

Theres no reference to /etc/.pihole/adlists.default in their basic-install.sh code either. So i'am not sure why its really in gravity.sh
https://github.com/pi-hole/pi-hole/blob/master/gravity.sh#L354

Looks like some code left over from the install script. They dont even parse or use /etc/.pihole/adlists.default
adListDefault=/etc/pihole/adlists.default
https://github.com/pi-hole/pi-hole/blob/master/gravity.sh#L122

mmm, needs more investigation.

[Fourdee]

@k-plan

Problem with pi-hole logfiles, then dietpi Option: DietPi-Ramlog #1 or DietPi-Ramlog #2 is used, because "/var/log/pihole.log" is gone, after 1h.

I was unable to replicate this with DietPi-Ramlog 2 and Full logging modes on a fresh install. It may be language/locale related.

I need some more information.

Can you paste results of:

ls -lha /var/log

date +'%b %e'

Reboot system to generate pihole.log. Allow for a few minutes of queries, then:

cat /var/log/pihole.log

stat -c%s /var/log/pihole.log

Note to self:
https://github.com/Fourdee/DietPi/blob/master/dietpi/dietpi-logclear#L117

[k-plan]

@Fourdee

Currently using Pi-hole with DietPi full logging. AdminLTE looks now:

root@oDroide-C1:~# ls -lha /var/log
insgesamt 2,6M
drwxrwxr-x  9 root     root     4,0K Mär 13 13:40 .
drwxr-xr-x 12 root     root     4,0K Mär 12 21:13 ..
-rwxrwxr-x  1 root     root        0 Mär 12 19:45 alternatives.log
drwxrwxr-x  2 root     adm      4,0K Mär 12 19:42 apache2
drwxrwxr-x  2 root     root     4,0K Dez 27 18:00 apt
-rwxrwxr-x  1 root     adm       18K Mär 14 16:39 auth.log
-rwxrwxr-x  1 root     utmp        0 Dez  3 14:44 btmp
-rwxrwxr-x  1 root     adm       21K Mär 14 16:17 daemon.log
-rwxrwxr-x  1 root     adm      2,2K Mär 13 15:07 debug
-rwxrwxr-x  1 root     root        0 Mär 12 18:30 dietpi-apt-get_update
-rwxrwxr-x  1 root     root        1 Mär 12 18:30 dietpi-backup.log
-rwxrwxr-x  1 root     root        1 Mär 13 15:07 dpkg.log
drwxrwxr-x  2 root     root     4,0K Dez  3 14:44 fsck
-rwxrwxr-x  1 root     adm       54K Mär 13 15:07 kern.log
-rwxrwxr-x  1 root     root      292 Mär 14 14:30 lastlog
-rwxrwxr-x  1 root     adm       66K Mär 14 16:39 messages
drwxrwsr-x  2 mysql    adm      4,0K Mär 12 18:27 mysql
-rwxrwxr-x  1 mysql    adm         0 Mär 12 19:43 mysql.err
-rwxrwxr-x  1 mysql    adm         0 Mär 12 19:43 mysql.log
drwxrwxr-x  2 root     root     4,0K Dez  3 14:44 news
-rwxrwxr-x  1 root     root      24K Mär 14 16:17 ntpd.log
drwxrwxr-x  2 root     root     4,0K Dez  3 14:44 ntpstats
-rwxrwxr-x  1 www-data www-data 2,3M Mär 14 16:56 pihole.log
drwxrwxr-x  2 root     root     4,0K Dez  3 14:44 samba
-rwxrwxr-x  1 root     adm       98K Mär 14 16:39 syslog
-rwxrwxr-x  1 root     utmp     6,4K Mär 14 14:30 wtmp

root@oDroide-C1:~# date +'%b %e'
Mär 14

root@oDroide-C1:~# reboot

root@oDroide-C1:~# cat /var/log/pihole.log
Mar 13 15:07:48 dnsmasq[1302]: gestartet, Version 2.72, Cachegröße 10000
Mar 13 15:07:48 dnsmasq[1302]: Übersetzungsoptionen: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect
Mar 13 15:07:48 dnsmasq[1302]: Warnung: Ignoriere "resolv-file", weil "no-resolv" aktiv ist
Mar 13 15:07:48 dnsmasq[1302]: Benutze Namensserver 8.8.8.8#53
Mar 13 15:07:48 dnsmasq[1302]: Benutze Namensserver 8.8.8.4#53
Mar 13 15:07:48 dnsmasq[1302]: /etc/hosts gelesen - 2 Adressen
Mar 13 15:08:05 dnsmasq[1302]: /etc/pihole/gravity.list gelesen - 84590 Adressen
....
....
....
Mar 14 17:00:32 dnsmasq[1292]: reply z-m.c10r.facebook.com is 2a03:2880:f01a:20:face:b00c:0:14c9
Mar 14 17:00:32 dnsmasq[1292]: query[A] b-api.facebook.com from 192.168.100.2
Mar 14 17:00:32 dnsmasq[1292]: cached b-api.facebook.com is <CNAME>
Mar 14 17:00:32 dnsmasq[1292]: cached z-m.facebook.com is <CNAME>
Mar 14 17:00:32 dnsmasq[1292]: forwarded b-api.facebook.com to 8.8.8.8
Mar 14 17:00:32 dnsmasq[1292]: reply b-api.facebook.com is <CNAME>
Mar 14 17:00:32 dnsmasq[1292]: reply z-m.facebook.com is <CNAME>
Mar 14 17:00:32 dnsmasq[1292]: reply z-m.c10r.facebook.com is 31.13.90.37
Mar 14 17:00:49 dnsmasq[1292]: query[A] win10.ipv6.microsoft.com from 192.168.100.4
Mar 14 17:00:49 dnsmasq[1292]: /etc/pihole/gravity.list win10.ipv6.microsoft.com is 192.168.100.55

root@oDroide-C1:~# stat -c%s /var/log/pihole.log
2398092

AdminLTE after reboot looks :

---

Now I will change from Log System : Full to Log System : DietPi-Ramlog #2 (with auto reboot)

root@oDroide-C1:~# ls -lha /var/log
insgesamt 28K
drwxrwxr-x  9 root     root      520 Mär 14 17:18 .
drwxr-xr-x 12 root     root     4,0K Mär 12 21:13 ..
-rwxrwxr-x  1 root     root        0 Mär 12 19:45 alternatives.log
drwxrwxr-x  2 root     adm       100 Mär 12 19:42 apache2
drwxrwxr-x  2 root     root       80 Dez 27 18:00 apt
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 auth.log
-rwxrwxr-x  1 root     utmp        0 Dez  3 14:44 btmp
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 daemon.log
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 debug
-rwxrwxr-x  1 root     root        0 Mär 12 18:30 dietpi-apt-get_update
-rwxrwxr-x  1 root     root        0 Mär 12 18:30 dietpi-backup.log
-rwxrwxr-x  1 root     root        0 Mär 14 17:18 dpkg.log
drwxrwxr-x  2 root     root       40 Dez  3 14:44 fsck
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 kern.log
-rwxrwxr-x  1 root     root      292 Mär 14 17:18 lastlog
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 messages
drwxrwsr-x  2 mysql    adm        60 Mär 12 18:27 mysql
-rwxrwxr-x  1 mysql    adm         0 Mär 12 19:43 mysql.err
-rwxrwxr-x  1 mysql    adm         0 Mär 12 19:43 mysql.log
drwxrwxr-x  2 root     root       40 Dez  3 14:44 news
-rwxrwxr-x  1 root     root      874 Mär 14 17:18 ntpd.log
drwxrwxr-x  2 root     root       40 Dez  3 14:44 ntpstats
-rwxrwxr-x  1 www-data www-data  11K Mär 14 17:21 pihole.log
drwxrwxr-x  2 root     root       40 Dez  3 14:44 samba
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 syslog
-rwxrwxr-x  1 root     utmp     1,9K Mär 14 17:18 wtmp

root@oDroide-C1:~# date +'%b %e'
Mär 14

root@oDroide-C1:~# cat /var/log/pihole.log
Mar 14 17:18:36 dnsmasq[1293]: gestartet, Version 2.72, Cachegröße 10000
Mar 14 17:18:36 dnsmasq[1293]: Übersetzungsoptionen: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect
Mar 14 17:18:36 dnsmasq[1293]: Warnung: Ignoriere "resolv-file", weil "no-resolv" aktiv ist
Mar 14 17:18:36 dnsmasq[1293]: Benutze Namensserver 8.8.8.8#53
Mar 14 17:18:36 dnsmasq[1293]: Benutze Namensserver 8.8.8.4#53
Mar 14 17:18:36 dnsmasq[1293]: /etc/hosts gelesen - 2 Adressen
Mar 14 17:18:53 dnsmasq[1293]: /etc/pihole/gravity.list gelesen - 84627 Adressen
Mar 14 17:18:56 dnsmasq[1293]: query[PTR] 55.100.168.192.in-addr.arpa from 127.0.0.1
Mar 14 17:18:56 dnsmasq[1293]: config 192.168.100.55 is NXDOMAIN
Mar 14 17:19:13 dnsmasq[1293]: query[AAAA] e.crashlytics.com from 192.168.100.9
Mar 14 17:19:13 dnsmasq[1293]: forwarded e.crashlytics.com to 8.8.8.8
Mar 14 17:19:13 dnsmasq[1293]: forwarded e.crashlytics.com to 8.8.8.4
Mar 14 17:19:13 dnsmasq[1293]: reply e.crashlytics.com is <CNAME>
Mar 14 17:19:13 dnsmasq[1293]: query[A] e.crashlytics.com from 192.168.100.9
Mar 14 17:19:13 dnsmasq[1293]: /etc/pihole/gravity.list e.crashlytics.com is 192.168.100.55

root@oDroide-C1:~# stat -c%s /var/log/pihole.log
26754

AdminLTE:

Now I have to wait a hour ..... to give you paste results again.

so long - kplan

[Fourdee]

@k-plan
Many thanks for the tests and results.

It appears DNSMASQ is logging dates in english:

root@oDroide-C1:~# cat /var/log/pihole.log
Mar 14 17:18:36 dnsmasq[1293]: gestartet, Version 2.72, Cachegröße 10000

Where as, your system date is pulling de:

root@oDroide-C1:~# date +'%b %e'
Mär 14

So it looks like I was right.
Because no Mäy 14 entries exist in your /var/log/pihole.log, all lines in the log file are being cleared when dietpi-logclear is run. DietPi trims all pihole log entries for all days, excluding today. This is to free up ramlog, so you get a day of usage stats with a max logfile size of 5MB.

I'll try a few things and get back to you, might need you to test them, but i'll let you know.

[Fourdee]

@k-plan

Looks like we can use the following to force english months from date:

root@DietPi:~# date +'%b %e'
Mär 14
root@DietPi:~# LANG=en_GB.UTF-8 date +'%b %e'
Mar 14

If you can confirm the following returns Mar i'd be greatful:

LANG=en_GB.UTF-8 date +'%b %e'

[k-plan]

@Fourdee

root@oDroide-C1:~# LANG=en_GB.UTF-8 date +'%b %e'
Mar 14

If you can confirm the following returns Mar i'd be greatful:

YES! 👍 :0))

[Fourdee]

@k-plan
Excellent, thanks again 👍

Ok so all the main issues have been resolved.

The only outstanding item is:

root@oDroide-C1:~# gravity.sh
:::
::: You are root.
cp: der Aufruf von stat für „/etc/.pihole/adlists.default“ ist nicht möglich: Datei oder Verzeichnis nicht gefunden
::: Neutrino emissions detected...

Although this file is only related to the basic-install.sh script from PiHole (which DietPi doesnt use), I could just add &> /dev/null to the command or remove the line entirly, but I'll have a think about this one.
Either way, this does no harm to your system and the "no file found" error is "semi-acceptable".

[k-plan]

@Fourdee

So it looks like I was right.
Because no Mäy 14 entries exist in your /var/log/pihole.log, all lines in the log file are being cleared when dietpi-logclear is run.

Correct! Happend this moment.

root@oDroide-C1:~# cat /var/log/pihole.log
Mar 14 18:17:01 dnsmasq[2493]: gestartet, Version 2.72, Cachegröße 10000
Mar 14 18:17:01 dnsmasq[2493]: Übersetzungsoptionen: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect
Mar 14 18:17:01 dnsmasq[2493]: Warnung: Ignoriere "resolv-file", weil "no-resolv" aktiv ist
Mar 14 18:17:01 dnsmasq[2493]: Benutze Namensserver ....

But no more "divide-by-zero error" in cat /error.log because I have change Line 17 in /var/www/pihole/data.php as describedi Fix here: /pi-hole/web@8d02575

AdminLTE before dieitpi-logclear is run:

And after:

To change Language and Regional Settings is not good thought, because you now have a quantity of work.
Sorry for that and many thanks for your assistance

[Fourdee]

@k-plan

But no more "divide-by-zero error" in cat /error.log because I have change Line 17 in /var/www/pihole/data.php as describedi Fix here: /pi-hole/AdminLTE@8d02575

It appears this hasn't reached the PiHole master branch yet (which DietPi forks).
I've updated the DietPi fork for now to resolve the issue. Again, will be patched in v113 https://github.com/Fourdee/AdminLTE/commit/e1cd3556c55d982d8548e207c1f8d35a09d78f7d

To change Language and Regional Settings is not good thought, because you now have a quantity of work.
Sorry for that and many thanks for your assistance

Hehe, no worries, its not much work (this time :) ). Just a change in the following line is all it needed:

https://github.com/Fourdee/DietPi/blob/master/dietpi/dietpi-logclear#L117

[k-plan]

@Fourdee

The only outstanding item is:
cp: der Aufruf von stat für „/etc/.pihole/adlists.default“ ist nicht möglich: Datei oder Verzeichnis nicht gefunden

not fussed about this. Will only be notice, if gravity.sh is started from command line by user.

BTW: - will gravity.sh started automatically via cron by default after pi-hole installation?

And it will be verry nice, you have a look on this:

[Fourdee]

@k-plan

Ah, missed that one. Same fix as per https://github.com/Fourdee/DietPi/blob/master/dietpi/dietpi-logclear#L117:
https://github.com/Fourdee/DietPi/blob/testing/dietpi/dietpi-cloudshell#L590

You can apply this now, just download the new script and restart dietpi-cloudshell:

wget https://raw.githubusercontent.com/Fourdee/DietPi/testing/dietpi/dietpi-cloudshell -O /DietPi/dietpi/dietpi-cloudshell

Patched for v113

[k-plan]

@Fourdee

💯 👍

[k-plan]

@Fourdee

do some testing with modified/DietPi/dietpi/dietpi-logclear

Results:

- with DietPi-Ramlog #1 (default) and DietPi-Ramlog #2:

(+) Display graph and counters match with "... Today" bottom in AdminLTE and fits the specified date. Same in Cloudshell
(pihole.log is cleared at 0:00 h)
(+) Increases the life of your SDcard by reducing file system IO (read/writes), because pihole.log is written in RAM disk (tmpfs)
(+) pihole.log size stay small (max. 5 MB)
(-) no long-time history in log

It will be my preferred choice. Will be a good default choice for Pi-hole installation.

- with Full (Rsyslog & Logrotate):

~ Display graph and counters is the past 24 h in the same record.
(confused me a little bit in the beginning)
(-) pihole.log size can be rise up after long run-time (keep a eye on it)
(-) Reduces the life of your SDcard due to increased file system IO (read/write) and decreases overall system performance
(+) long-time history in log, to see, what is gone on in your LAN

**_Suggestion:_**

• gravity.sh run automatically once after Pi-hole installation. It will be useful, to add this automatically to /etc/cron.daily/ (with a update log file /var/log/pihole-update.log) or prompt user for this.
• Will be a nice add-on, installation prompt user for his preferred DNS server (not use google DNS by default)

My upshot: it's now run better as the current default automated install of pi-hole itself.
Nice job!

[Fourdee]

@k-plan
DietPi-Logclear will only clear the following data from pihole.log on an hourly basis, or when dietpi-logclear is run:

• Other days of log data, excluding today
• If the file has reached 5MB in size

https://github.com/Fourdee/DietPi/blob/testing/dietpi/dietpi-logclear#L109-L151

When you reboot the system using DietPi-Ramlog mode 1/2, all log file contents is cleared. We dont store the log file data over reboots, unless you're using full logging mode.

The 2nd DietPi-Ramlog mode backs up the logfile contents, but as we (and pihole) dont parse the backups, in theory both DietPi-Ramlog modes will give the same results. You should always have todays pihole logs and stats, unless the pihole.log file exceeds 5MB in size, or you reboot system.

My upshot: it's now run better as the current default automated install of pi-hole itself.
Nice job!

Excellent 👍 The core DietPi optimizations including ramlog and quicker ondemand upscaling will improve PiHole (dns query) response times.
This is a bit outdated, but lists optimizations and their benefits over the Jessie Lite install with PiHole: https://docs.google.com/spreadsheets/d/1mDHGZC-H6tU6_O8kuLTG8d4A8Nt7lV1Q7MXTR_6qw30/edit?usp=sharing

gravity.sh run automatically once after Pi-hole installation. It will be useful, to add this automatically to /etc/cron.daily/ or prompt user for this.
Will be a nice add-on, installation prompt user for his preferred DNS server (not use google DNS by default)

Great suggestions 👍 . I'll leave them here for reference and take a look when things settle down :)

[k-plan]

@Fourdee
Thank you for explanation. 👍

DietPi-Logclear will only clear the following data from pihole.log on an hourly basis, or when dietpi-logclear is run:

• Other days of log data, excluding today
• If the file has reached 5MB in size

How you described, it will be as we desired.
Hmm, it's quite possible, that I have screwed up something by editing /DietPi/dietpi/dietpi-logclear.
Will do some more testing and then will updating my previous text, so its don't puzzle anyone. Promised .

When you reboot the system using DietPi-Ramlog mode 1/2, all log file contents is cleared. We don't store the log file data over reboots, unless you're using full logging mode.

Make sense and my device will do as described above.

The 2nd DietPi-Ramlog mode backs up the logfile contents, ...

Yes, I have read in the describtion of DietPi-Ramlog:

DietPi-Ramlog # 2:
...
+ DietPi will automatically save/update log file data to disk every hour/root/logfile_storage.
+ DietPi will then clear log files in /var/log (to free memory used by DietPi-Ramlog).
─ Potential of losing upto 1 hour of log file data if power is interrupted.

But I can't find pihole.log in backup logfile directory:

root@oDroide-C1:~# cat /DietPi/dietpi/.installed

  #DietPi Choice System: Logging
  INDEX_LOGGING_CURRENT -2
  INDEX_LOGGING_TARGET -2

root@oDroide-C1:~# ls -lha /var/log/
insgesamt 1,2M
drwxrwxr-x  9 root     root      520 Mär 17 00:17 .
drwxr-xr-x 12 root     root     4,0K Mär 12 21:13 ..
-rwxrwxr-x  1 root     root        0 Mär 12 19:45 alternatives.log
drwxrwxr-x  2 root     adm       100 Mär 12 19:42 apache2
drwxrwxr-x  2 root     root       80 Dez 27 18:00 apt
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 auth.log
-rwxrwxr-x  1 root     utmp        0 Dez  3 14:44 btmp
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 daemon.log
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 debug
-rwxrwxr-x  1 root     root        1 Mär 16 02:17 dietpi-apt-get_update
-rwxrwxr-x  1 root     root        1 Mär 16 11:17 dietpi-backup.log
-rwxrwxr-x  1 root     root        1 Mär 16 11:17 dpkg.log
drwxrwxr-x  2 root     root       40 Dez  3 14:44 fsck
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 kern.log
-rwxrwxr-x  1 root     root      292 Mär 17 17:19 lastlog
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 messages
drwxrwsr-x  2 mysql    adm        60 Mär 12 18:27 mysql
-rwxrwxr-x  1 mysql    adm         0 Mär 12 19:43 mysql.err
-rwxrwxr-x  1 mysql    adm         0 Mär 12 19:43 mysql.log
drwxrwxr-x  2 root     root       40 Dez  3 14:44 news
-rwxrwxr-x  1 root     root       58 Mär 17 17:17 ntpd.log
drwxrwxr-x  2 root     root       40 Dez  3 14:44 ntpstats
-rwxrwxr-x  1 www-data www-data 1,1M Mär 17 17:20 pihole.log
drwxrwxr-x  2 root     root       40 Dez  3 14:44 samba
-rwxrwxr-x  1 root     adm         0 Mär 14 17:18 syslog
-rwxrwxr-x  1 root     utmp      384 Mär 17 17:19 wtmp

root@oDroide-C1:~# ls -lha /root/logfile_storage/
insgesamt 7,8M
drwxr-xr-x  5 root root 4,0K Mär 16 02:17 .
drwx------ 11 root root 4,0K Mär 14 18:22 ..
drwxr-xr-x  2 root root 4,0K Mär 12 19:17 apache2
drwxr-xr-x  2 root root 4,0K Mär 14 20:17 apt
-rw-r--r--  1 root root 4,6K Mär 16 02:17 dietpi-apt-get_update
-rw-r--r--  1 root root 7,6M Mär 16 11:17 dietpi-backup.log
-rw-r--r--  1 root root 3,3K Mär 16 11:17 dpkg.log
-rw-r--r--  1 root root 3,2K Mär 17 18:17 lastlog
drwxr-xr-x  2 root root 4,0K Mär 12 19:17 mysql
-rw-r--r--  1 root root  86K Mär 17 18:17 ntpd.log
-rw-r--r--  1 root root  17K Mär 17 18:17 wtmp

No matter, confused me a little bit. Can be as desired.
No hard feelings. For someone like me, how can't type a single line of working code or script, it will be very difficult to follow program line of the top coder. But I will put one's best foot forward.

[Fourdee]

@k-plan

But I can't find pihole.log in backup logfile directory:

This is because we are using if / elif when processing the filetype.
https://github.com/Fourdee/DietPi/blob/master/dietpi/dietpi-logclear#L109-L156
All the code highlighted will not get backed up.

Also, when DietPi-Logclear backs up the logfiles, it actually reads the contents, then adds that data to the existing backups, before the original logfile in /var/log is cleared.
https://github.com/Fourdee/DietPi/blob/master/dietpi/dietpi-logclear#L169

So in theory, you could change the code to backup the pihole logs aswell, but you would need to do it a different way. Because /var/log/pihole.log isnt cleared when the script runs, you will end up with duplicate entries in the backed up pihole.log file.

[PromoFaux]

Hey,

Theres no reference to /etc/.pihole/adlists.default in their basic-install.sh code either. So i'am not sure why its really in gravity.sh
https://github.com/pi-hole/pi-hole/blob/master/gravity.sh#L354

Looks like some code left over from the install script. They dont even parse or use /etc/.pihole/adlists.default
adListDefault=/etc/pihole/adlists.default

adlists.default That is indeed used. It's copied from the temp install directory each time gravity.sh is run, so there is no need to copy it on install. This ensures that the default list is always the same (provided the user has not changed /etc/.pihole/adlists.default)

If user wants to use their own list, instead of the default, then they need to manually create adlists.list and make their changes there. If gravity.sh does not detect this file, then it refers to adlists.default

You could just ask.

[Fourdee]

@PromoFaux

It's copied from the temp install directory each time gravity.sh is run, so there is no need to copy it on install.

Seems strange to copy a file from a tempoary directory each time gravity.sh is run.
Would it not make more sense to move adlists.default into /etc/pihole as its clearly required for gravity.sh to function, or at the very least, check the temporary file exists in the temporary install directory before copying it?

You could even &> /dev/null or &> /var/log/pihole.log for a "dirty quicky"

You could just ask.

Clearly unnecessary :)

[PromoFaux]

One day, we might be friends, and we will look back and laugh. Until then, I think you're a pompous sod... ;)

I use the term "temp install directory" loosely. It is, in fact, permanent. The way the install process works is we clone the git repository into /etc/.pihole/ and copy the files to their final destinations.

The reason I made gravity copy the file each time was so that we could be sure that the user has absolutely not made changes to adlists.default (that way, if they mess up with adlists.list, they just need to delete it and they'll have the defaults, without having to re-download it...)

If you think there are better ways we could be doing things that would mean more compatibility with DietPi, whilst still maintaining compatibility with other systems, then go ahead and submit a PR with your fixes so that you don't have to keep tweaking them each time we update! :)

[Fourdee]

@PromoFaux

One day, we might be friends, and we will look back and laugh. Until then, I think you're a pompous sod... ;)

Lol, I really couldnt care less what your opinion of me is.
I'am here to code and get things done, not talk trash because someone has the inability to keep their childish opinions to themselves.

I use the term "temp install directory" loosely. It is, in fact, permanent. The way the install process works is we clone the git repository into /etc/.pihole/ and copy the files to their final destinations.

So what your saying is /etc/.pihole/* is actually a hidden permanent directory for your PiHole curl -L install.pi-hole.net | bash installation?

If you think there are better ways we could be doing things that would mean more compatibility with DietPi, whilst still maintaining compatibility with other systems, then go ahead and submit a PR with your fixes so that you don't have to keep tweaking them each time we update! :)

Will do.

[PromoFaux]

Meh, not trash talk, just an assessment of how your general attitude has been. But that's probably just me, not a reflection on the rest of the Pi-hole team.

Anyway, all that aside.

So what your saying is /etc/.pihole/* is actually a hidden permanent directory for your PiHole curl -L install.pi-hole.net | bash installation

Correct, before we did this, we were individually curling each file into the required locations, which was causing some issues if connections were dropped mid-install. Made sense to clone the whole repo, and then there isn't as much to download/overwrite when there are updates.

[Fourdee]

Meh, not trash talk, just an assessment of how your general attitude has been. But that's probably just me, not a reflection on the rest of the Pi-hole team.

If I need an assessment of my attitude, I'll ask for one. You really need to learn a way of controlling your emotions and keeping your personal opinions of people, to yourself.

Correct

Thanks for the confirmation.