DietPi-Globals | G_CHECK_URL fails as DNS resolving takes very long

[musicfan145]

Details:

• Date | Sat 20 Apr 15:48:10 EDT 2019
• Bug report | N/A
• DietPi version | v6.21.1 (Fourdee/master)
• Img creator | n/a
• Pre-image | n/a
• SBC device | RPi 3 Model B (armv7l) (index=3)
• Kernel version | Request | RT Kernel option #1200 SMP Tue Feb 12 20:27:48 GMT 2019
• Distro | stretch (index=4)
• Command | Connection test: http://raspbian.raspberrypi.org/raspbian
• Exit code | 4
• Software title | DietPi-Software

Steps to reproduce:

1. Log in through SSH (iOS Termius App)
2. Run dietpi-software

Expected behaviour:

• dietpi-software opens and allows software selection & installation.

Actual behaviour:

root@IQaudIO:~# dietpi-software
[ OK ] DietPi-Software | Root access verified.
[ OK ] DietPi-Software | RootFS R/W access verified.

[ OK ] DietPi-Software | Initialized database
[ OK ] DietPi-Software | Reading database
[ .... ] DietPi-Software | Failed connection attempt (1/3), retrying[ .... ] DietPi-Software | Failed connection attempt (2/3), retrying[ .... ] DietPi-Software | Failed connection attempt (3/3), retrying[FAILED] DietPi-Software | Connection test: http://raspbian.raspberrypi.org/raspbian

Extra details:

This is happening on all of my DietPi machines, Raspberry Pi 3 and Sparky, versions 6.21.1 to 6.22.3. It also fails the connection test in DietPi-config network settings. However, DietPi updates still work if connection failures are ignored.

Additional logs:

Log file contents:
Spider mode enabled. Check if remote file exists.
--2019-04-20 15:47:58--  http://raspbian.raspberrypi.org/raspbian
Resolving raspbian.raspberrypi.org (raspbian.raspberrypi.org)... failed: Connection timed out.
wget: unable to resolve host address ‘raspbian.raspberrypi.org’

---

[FAILED] DietPi-Software | Unable to continue, DietPi-Software will now terminate.

root@IQaudIO:~#

[MichaIng]

@musicfan145
Many thanks for your report.

Could you please go trough: #2717
According to your error message as well name resolving seems to take unexpected long and our connection test function waits for 5 seconds by default.

[musicfan145]

Looks like 6+ seconds for me too:

root@Sparky:~# time wget --spider https://deb.debian.org/debian
Spider mode enabled. Check if remote file exists.
--2019-04-21 09:07:18-- https://deb.debian.org/debian
Resolving deb.debian.org (deb.debian.org)... 128.31.0.62, 149.20.4.15, 5.153.231.4, ...
Connecting to deb.debian.org (deb.debian.org)|128.31.0.62|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cdn-aws.deb.debian.org/debian [following]
Spider mode enabled. Check if remote file exists.
--2019-04-21 09:07:24-- https://cdn-aws.deb.debian.org/debian
Resolving cdn-aws.deb.debian.org (cdn-aws.deb.debian.org)... 13.249.109.43, 13.249.109.54, 13.249.109.3, ...
Connecting to cdn-aws.deb.debian.org (cdn-aws.deb.debian.org)|13.249.109.43|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://cdn-fastly.deb.debian.org/debian [following]
Spider mode enabled. Check if remote file exists.
--2019-04-21 09:07:24-- http://cdn-fastly.deb.debian.org/debian
Resolving cdn-fastly.deb.debian.org (cdn-fastly.deb.debian.org)... 151.101.56.204, 2a04:4e42:45::204
Connecting to cdn-fastly.deb.debian.org (cdn-fastly.deb.debian.org)|151.101.56.204|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://ftp.debian.org/debian/ [following]
Spider mode enabled. Check if remote file exists.
--2019-04-21 09:07:24-- http://ftp.debian.org/debian/
Resolving ftp.debian.org (ftp.debian.org)... 130.89.148.12, 2001:67c:2564:a119::148:12
Connecting to ftp.debian.org (ftp.debian.org)|130.89.148.12|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Remote file exists and could contain further links,
but recursion is disabled -- not retrieving.

real 0m6.200s
user 0m0.160s
sys 0m0.060s
root@Sparky:~# time wget --spider https://130.89.148.12
Spider mode enabled. Check if remote file exists.
--2019-04-21 09:08:01-- https://130.89.148.12/
Connecting to 130.89.148.12:443... connected.
ERROR: The certificate of ‘130.89.148.12’ is not trusted.
ERROR: The certificate of ‘130.89.148.12’ hasn't got a known issuer.The certificate's owner does not match hostname ‘130.89.148.12’

real 0m0.465s
user 0m0.090s
sys 0m0.030s
root@Sparky:~#

[musicfan145]

Based on /etc/resolv.conf DNS is apparently set to my router. I will try some of the other troubleshooting steps next.

[MichaIng]

@musicfan145
Usually the router should provide a local ISP DNS which should be fine. But yeah worth to try it with e.g. 8.8.8.8 which is a good standard comparison.

You are already the third reporting this. In the other two cases the internet otherwise is fast 50 - 100 Mbits and in one case all other clients (e.g. Windows desktops etc) using the same DNS and resolving fractions of a second.

In your case it is a bid different then in the linked issue as only first resolving is taking long (09:07:18 => 09:07:24) which the others finish in the same second.

---

So to test:

• Comparing different connection methods:
  • time wget --spider https://dietpi.com
  • time curl -LI https://dietpi.com
  • apt update # time does not make sense here, but perhaps while running this one can see if it takes long as well to resolve the hostname (https://deb.debian.org/debian)
  • curl -LI https://dietpi.com can be as well run from Windows cmd just to compare.
• Different DNS entry (e.g. 8.8.8.8)
• Switching from DHCP to static IP (using the copy from current option)

Not sure what can be the reason here. The DNS resolving itself should only take fractions of a second, as long as you didn't manually hack inside a DNS server from the other side of the globe. I will dig a bid into it and try to find methods to debug the DNS resolving itself at best calling the bare library behind it and skipping the wget/curl/etc wrappers.

[musicfan145]

I tried switching to static IP using the copy command in dietpi-config. (I already had it set to static on the router.) With the DNS setting unchanged (my router as DNS) the connection time was the same. I switched the DNS to 8.8.8.8 and this sped things up a bit, but it was still over 6 seconds for time wget --spider https://deb.debian.org/debian.

I edited the timeout in dietpi-globals, and this allowed me to run dietpi-software. However, it is still a mystery as to why DNS is so slow.

I tried running curl -sSL https://dietpi.com in Windows cmd for comparison, but it just gave me the dietpi.com web page HTML. Not sure how to get the response time.

[MichaIng]

@musicfan145
Just found the curl command that is equivalent to wget spider is the following:
time curl -LI https://deb.debian.org/debian

Nice about it is as well that it lists all the redirects done in a better overview than wget does. But the timestamps each redirect are missing, instead you see the time reported from the servers, which are not always 100% in sync.

[MichaIng]

We added the possibility to raise the timout and retry count for G_CHECK_URL which should resolve this issue. This could be further tweaked, e.g. we scrape the output for "timeout" and in case give more specific info and allow to change these settings directly (or open the submenu directly): #2717 (comment)