Updating signature and other assets fails #200

waja · 2017-02-02T20:34:17Z

maldet(6576): {sigup} local signature set is version 2017012825142
maldet(6576): {sigup} could not download signature data from server, please try again later.

With c55e7d3 you introduced https based URLs. Anyway ... there might some issues with this.

Debian 7.x:

$ LANG=C; wget https://cdn.rfxn.com/downloads/maldet.current.ver.beta
--2017-02-02 21:29:09--  https://cdn.rfxn.com/downloads/maldet.current.ver.beta
Resolving cdn.rfxn.com (cdn.rfxn.com)... 104.28.23.81, 104.28.22.81, 2400:cb00:2048:1::681c:1751, ...
Connecting to cdn.rfxn.com (cdn.rfxn.com)|104.28.23.81|:443... connected.
GnuTLS: A TLS fatal alert has been received.
Unable to establish SSL connection.
$ LANG=C; wget http://cdn.rfxn.com/downloads/maldet.current.ver.beta
--2017-02-02 21:29:31--  http://cdn.rfxn.com/downloads/maldet.current.ver.beta
Resolving cdn.rfxn.com (cdn.rfxn.com)... 104.28.23.81, 104.28.22.81, 2400:cb00:2048:1::681c:1751, ...
Connecting to cdn.rfxn.com (cdn.rfxn.com)|104.28.23.81|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4
Saving to: `maldet.current.ver.beta'

100%[===================================================================================================>] 4           --.-K/s   in 0s

2017-02-02 21:29:31 (247 KB/s) - `maldet.current.ver.beta' saved [4/4]

Debian 6.x:

$ LANG=C; wget https://cdn.rfxn.com/downloads/maldet.current.ver.beta
--2017-02-02 21:28:37--  https://cdn.rfxn.com/downloads/maldet.current.ver.beta
Resolving cdn.rfxn.com... 104.28.23.81, 104.28.22.81, 2400:cb00:2048:1::681c:1751, ...
Connecting to cdn.rfxn.com|104.28.23.81|:443... connected.
OpenSSL: error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error
Unable to establish SSL connection.

From what I've found on the search engines there might be some issue in redirected https connections, but nothing specific. Other distributions and (Debian) derivates might be affected. On Debian 8.x this works as expected.

Many thanks, Jan.

The text was updated successfully, but these errors were encountered:

rfxn · 2017-02-02T21:01:03Z

I will investigate this, I suspect the issue is the SSL/TLS version on connection to Cloudflare CDN.

Thanks

waja · 2017-02-02T21:03:12Z

Looking into https://www.ssllabs.com/ssltest/analyze.html?d=cdn.rfxn.com&hideResults=on seems to have TLS1.0-TLS1.2.

rfxn · 2017-02-02T21:47:48Z

I got deb6/7/8 virtualbox instances I'll kick off tonight and poke at this further. As always, thanks for the contribution in pointing this out!

rfxn · 2017-02-03T18:32:57Z

There doesn't appear to be an easy fix with wget itself on these older distro releases, the issue is with wget itself and the cipher suites enforced on the CDN simply not being available on older versions of wget.

An option might be to preference curl which works fine on deb7, then fallback to wget as a last resort. I am liking this better than the alternative of turning off the CDN.

rfxn · 2017-03-18T00:14:34Z

In the interest of closing out v1.6, I've for now switched CDN's to something with a little looser cipher suites that works on deb 6/7 TLSv1.

I will revisit this as a TODO item in a later iterative update to switch to CURL.

waja · 2017-03-19T15:37:05Z

@rfxn Unfortunately it seems not to work right now for Debian 6 and 7

~$ grep PRETTY_NAME /etc/os-release
PRETTY_NAME="Debian GNU/Linux 7 (wheezy)"
~$ LANG=C /usr/bin/wget --referer=LMD:1.6 -e http_proxy= -e https_proxy= -T5 -t3 https://cdn.rfxn.com/downloads/maldet.sigs.ver -O /tmp/.hver19815
--2017-03-19 16:31:31--  https://cdn.rfxn.com/downloads/maldet.sigs.ver
Resolving cdn.rfxn.com (cdn.rfxn.com)... 104.28.22.81, 104.28.23.81, 2400:cb00:2048:1::681c:1651, ...
Connecting to cdn.rfxn.com (cdn.rfxn.com)|104.28.22.81|:443... connected.
GnuTLS: A TLS fatal alert has been received.
Unable to establish SSL connection.

~$ cat /etc/debian_version
6.0.10
~$ LANG=C /usr/bin/wget --referer=LMD:1.6 -e http_proxy= -e https_proxy= -T5 -t3 https://cdn.rfxn.com/downloads/maldet.sigs.ver -O /tmp/.hver19815
--2017-03-19 16:35:20--  https://cdn.rfxn.com/downloads/maldet.sigs.ver
Resolving cdn.rfxn.com... 104.28.23.81, 104.28.22.81, 2400:cb00:2048:1::681c:1751, ...
Connecting to cdn.rfxn.com|104.28.23.81|:443... connected.
OpenSSL: error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error
Unable to establish SSL connection.

waja · 2017-03-19T15:57:25Z

While downloading with curl on Debian 7 works:

~$ grep PRETTY_NAME /etc/os-release
PRETTY_NAME="Debian GNU/Linux 7 (wheezy)"
~$ LANG=C curl https://cdn.rfxn.com/downloads/maldet.sigs.ver --tlsv1 -o /tmp/.hver19815
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    14  100    14    0     0     24      0 --:--:-- --:--:-- --:--:--    24
~$ cat /tmp/.hver19815
2017022510908

On Debian 6 it doesn't:

~$ cat /etc/debian_version
6.0.10
~$ LANG=C curl https://cdn.rfxn.com/downloads/maldet.sigs.ver --tlsv1 -o /tmp/.hver19815
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (35) error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure

MichelClement · 2017-03-27T10:21:27Z

Info: on Debian GNU/Linux 8 (jessie) is works if you install the package 'ca-certificates'.

waja · 2017-03-27T10:29:04Z

Info: on Debian GNU/Linux 8 (jessie) is works if you install the package 'ca-certificates'.

But that issue has another source:

~$ LANG=C /usr/bin/wget --referer=LMD:1.6 -e http_proxy= -e https_proxy= -T5 -t3 https://cdn.rfxn.com/downloads/maldet.sigs.ver -O /tmp/.hver19815
converted 'https://cdn.rfxn.com/downloads/maldet.sigs.ver' (ANSI_X3.4-1968) -> 'https://cdn.rfxn.com/downloads/maldet.sigs.ver' (UTF-8)
--2017-03-27 12:28:01--  https://cdn.rfxn.com/downloads/maldet.sigs.ver
Resolving cdn.rfxn.com (cdn.rfxn.com)... 45.33.122.151
Connecting to cdn.rfxn.com (cdn.rfxn.com)|45.33.122.151|:443... connected.
ERROR: The certificate of 'cdn.rfxn.com' is not trusted.
ERROR: The certificate of 'cdn.rfxn.com' hasn't got a known issuer.

kero242 · 2017-03-31T10:15:55Z

I installed ca-certificates but same error with Debian 7. So i removed temporarily the s from https in internals.conf.

PSiAU · 2017-05-05T04:58:17Z

Thanks for the workaround @kero242 . I got bit by this too.

psvec · 2017-05-07T06:28:40Z

I replaced
in /usr/local/maldetect/internals/internals.conf
wget=which wget 2> /dev/null to wget=which curl 2> /dev/null
and than in /usr/local/maldetect/internals/functions in all wget calls you should delete options related to wget and leave just option to curl. You should delete "--referer="$lmd_referer" $wgetop" and change "-O to -o"

PSiAU · 2017-05-08T00:32:38Z

The https to http change in internals.conf got reverted back, presumably due to maldet detecting a modified file :(
Short of making the file immutable, could we just have https reverted back to http until this is resolved pretty please?

There's an issue with Debian's version of wget, preventing us from downloading signatures. rfxn#200

rfxn#200

rfxn · 2017-05-15T14:12:14Z

As an aside @PSiAU , @taylorskalyo @psvec and @kero242, the following sysconfig files are sourced into LMD after internals.conf. As such, you can redefine ANY variable within these files and they will be preserved across updates/upgrades.

maldet:

if [ -f "/etc/sysconfig/maldet" ]; then
        syscnf=/etc/sysconfig/maldet
elif [ -f "/etc/default/maldet" ]; then
        syscnf=/etc/default/maldet
fi
.....
if [ -f "$syscnf" ]; then
        source $syscnf
fi

rfxn · 2017-05-15T17:35:13Z

As noted previously, this issue inherently is the TLS versioning that Cloudflare is enforcing on the 'cdn.rfxn.com' domain, which looks to be at free tier v1.2 or newer. That is an incredibly strict standard to enforce where even semi-modern servers would not meet it.

https://support.cloudflare.com/hc/en-us/articles/200933580-What-cipher-suites-does-Cloudflare-use-for-SSL-

As of commit d1ae534 , there is now support for curl as default with wget as secondary. This, combined with disabling cloudflare as the CDN seems to allow Debian 6 and 7 to operate without issue out-of-the-box.

waja · 2017-05-15T20:06:17Z

As of commit d1ae534 , there is now support for curl as default with wget as secondary. This, combined with disabling cloudflare as the CDN seems to allow Debian 6 and 7 to operate without issue out-of-the-box.

Do you think it would be a good idea to release this soon as minor release of 1.6? Or maybe at least a RC version of it for easier testing.

rfxn · 2017-05-16T05:10:21Z

@waja @PSiAU 1.6-1.rc1 RC is up, testing would be a huge help!
https://github.com/rfxn/linux-malware-detect/releases/tag/1.6-1.rc1

Thanks

waja · 2017-05-17T10:37:36Z

1.6-1.rc1 RC is up, testing would be a huge help!

This is why I asked for a new tarball. :)

Looks fine on Debian 6 with curl installed:

maldet(19509): {sigup} performing signature update check...
maldet(19509): {sigup} could not determine signature version
maldet(19509): {sigup} signature files missing or corrupted, forcing update...
maldet(19509): {sigup} new signature set (2017051530038) available
maldet(19509): {sigup} downloading https://cdn.rfxn.com/downloads/maldet-sigpack.tgz
maldet(19509): {sigup} downloading https://cdn.rfxn.com/downloads/maldet-cleanv2.tgz
maldet(19509): {sigup} verified md5sum of maldet-sigpack.tgz
maldet(19509): {sigup} unpacked and installed maldet-sigpack.tgz
maldet(19509): {sigup} verified md5sum of maldet-clean.tgz
maldet(19509): {sigup} unpacked and installed maldet-clean.tgz
maldet(19509): {sigup} signature set update completed
maldet(19509): {sigup} 16561 signatures (13831 MD5 | 1951 HEX | 779 YARA | 0 USER)

Many thanks for fixing this. Very appreciated!

rfxn · 2017-05-28T13:44:16Z

1.6.1 is now up live, think that takes care of this issue. Closing but anyone feel free to reopen if issues persist.

Thanks!

philippetrov · 2017-06-16T21:33:46Z

`root@srv2:/ # maldet -u
Linux Malware Detect v1.6.1
(C) 2002-2017, R-fx Networks [email protected]
(C) 2017, Ryan MacDonald [email protected]
This program may be freely redistributed under the terms of the GNU GPL v2

maldet(1303): {sigup} performing signature update check...
maldet(1303): {sigup} could not determine signature version

root@srv2:/ # uname -v
FreeBSD 11.0-RELEASE-p2 #0: Mon Oct 24 06:55:27 UTC 2016 [email protected]:/usr/obj/usr/src/sys/GENERIC`

byatis · 2018-03-26T10:37:21Z

On CentOS release 5.11 (Final):
Linux Malware Detect v1.6.2
(C) 2002-2017, R-fx Networks [email protected]
(C) 2017, Ryan MacDonald [email protected]
This program may be freely redistributed under the terms of the GNU GPL v2

maldet(7174): {sigup} performing signature update check...
maldet(7174): {sigup} local signature set is version 2017070716978
maldet(7174): {sigup} could not download https://cdn.rfxn.com/downloads/maldet.sigs.ver, please try again later.
maldet(7174): {sigup} could not download signature data from server, please try again later.

if I replace https to http from internals.conf it works.

aworstell mentioned this issue Apr 24, 2017

Maldet losing ability to talk to clamscan, quarantining valid files #231

Closed

waja mentioned this issue May 12, 2017

Problem update version and database #190

Closed

taylorskalyo added a commit to codeguard/linux-malware-detect that referenced this issue May 12, 2017

Use Centos

550c0bf

There's an issue with Debian's version of wget, preventing us from downloading signatures. rfxn#200

taylorskalyo added a commit to codeguard/linux-malware-detect that referenced this issue May 12, 2017

Use Centos

c1c61b3

There's an issue with Debian's version of wget, preventing us from downloading signatures. rfxn#200

taylorskalyo added a commit to codeguard/linux-malware-detect that referenced this issue May 12, 2017

Use Centos

139ba51

There's an issue with Debian's version of wget, preventing us from downloading signatures. rfxn#200

taylorskalyo added a commit to codeguard/linux-malware-detect that referenced this issue May 12, 2017

Use Centos

c11cd8d

There's an issue with Debian's version of wget, preventing us from downloading signatures. rfxn#200

taylorskalyo added a commit to codeguard/linux-malware-detect that referenced this issue May 12, 2017

Ugly hack to fix wget issue

6ed0b04

rfxn#200

rfxn closed this as completed May 28, 2017

philippetrov mentioned this issue Jun 16, 2017

Wrong path for curl/wget on FreeBSD 11 #237

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating signature and other assets fails #200

Updating signature and other assets fails #200

waja commented Feb 2, 2017

rfxn commented Feb 2, 2017

waja commented Feb 2, 2017

rfxn commented Feb 2, 2017

rfxn commented Feb 3, 2017

rfxn commented Mar 18, 2017

waja commented Mar 19, 2017

waja commented Mar 19, 2017

MichelClement commented Mar 27, 2017

waja commented Mar 27, 2017

kero242 commented Mar 31, 2017

PSiAU commented May 5, 2017

psvec commented May 7, 2017

PSiAU commented May 8, 2017 •

edited

Loading

rfxn commented May 15, 2017 •

edited

Loading

rfxn commented May 15, 2017

waja commented May 15, 2017 •

edited

Loading

rfxn commented May 16, 2017

waja commented May 17, 2017

rfxn commented May 28, 2017

philippetrov commented Jun 16, 2017

byatis commented Mar 26, 2018

Updating signature and other assets fails #200

Updating signature and other assets fails #200

Comments

waja commented Feb 2, 2017

rfxn commented Feb 2, 2017

waja commented Feb 2, 2017

rfxn commented Feb 2, 2017

rfxn commented Feb 3, 2017

rfxn commented Mar 18, 2017

waja commented Mar 19, 2017

waja commented Mar 19, 2017

MichelClement commented Mar 27, 2017

waja commented Mar 27, 2017

kero242 commented Mar 31, 2017

PSiAU commented May 5, 2017

psvec commented May 7, 2017

PSiAU commented May 8, 2017 • edited Loading

rfxn commented May 15, 2017 • edited Loading

rfxn commented May 15, 2017

waja commented May 15, 2017 • edited Loading

rfxn commented May 16, 2017

waja commented May 17, 2017

rfxn commented May 28, 2017

philippetrov commented Jun 16, 2017

byatis commented Mar 26, 2018

PSiAU commented May 8, 2017 •

edited

Loading

rfxn commented May 15, 2017 •

edited

Loading

waja commented May 15, 2017 •

edited

Loading