Possibility of Blocking Requests to localhost and Reserved IP Addresses by websockets? #1070

ilikenwf · 2020-05-26T15:56:38Z

I'm currently using a rules list from here, which should do this, however in theory someone suggested a hostname could be assigned to an internal IP to bypass it (not sure how true this is):

https://github.com/gwarser/filter-lists/blob/master/lan-block.txt

Are there any solutions to restricting websocket access to local network IP addresses?

For more specific targeting, it appears ebay uses check.js from some threat assessment company:

http://ghacks.net/2020/05/25/ebay-is-port-scanning-your-system-when-you-load-the-webpage

gorhill · 2020-05-26T16:16:46Z

What are the possible issues adding https://github.com/gwarser/filter-lists/blob/master/lan-block.txt to uBO's default lists? (maybe even importing it in the project as a uBlock list).

||::1^$third-party needs to become ||[::1]^$third-party though.

I imported the list and I see no impact on performance as per built-in benchmark.

I did see a request from the benchmark which is being blocked by that list and which was not blocked:

url=http://127.0.0.1:3033/__webpack_hmr
docOrigin=https://hltv.gainskins.com

ilikenwf · 2020-05-26T16:28:18Z

What are the possible issues adding https://github.com/gwarser/filter-lists/blob/master/lan-block.txt to uBO's default lists? (maybe even importing it in the project as a uBlock list).

||::1^$third-party needs to become ||[::1]^$third-party though.

I imported the list and I see no impact on performance as per built-in benchmark.

I did see a request from the benchmark which is being blocked by that list and which was not blocked:
url=http://127.0.0.1:3033/__webpack_hmr
docOrigin=https://hltv.gainskins.com

@gwarser could probably comment...

But what if a site assigns a hostname to one of these IP addresses? In such a case, would these IP blocks by regex still catch those? If not we may need to think about that too...

gorhill · 2020-05-26T16:46:17Z

Well it's possible to block at onHeadersReceived() time and the ip address information is available at this point, so theoretically uBO could ensure the ip address is not a local one. In that case that would not be a matter of filter list, but maybe a per-site switch.

ilikenwf · 2020-05-26T17:13:42Z

The amazing thing is that this is apparently a feature and not a bug in most browsers...

So I guess the question is whether it would be worthwhile to just use the list, or as your suggested per-site switch...

The author of this article, which discusses and shows some of the more malicious possibilities with a stock Firefox:

https://medium.com/@stestagg/stealing-secrets-from-developers-using-websockets-254f98d577a0

Has provided a websocket portscan test here, but it uses IP and not hostnames...still, useful for peace of mind on that front...minus domains:

http://frontend-overflowstack.com/

@gorhill

as pointed out by @gorhill in uBlockOrigin/uBlock-issues#1070

ganego · 2020-05-27T02:08:26Z

NoScript had "ABE (Application Boundaries Enforcer") enabled by default which basically blocked access from non-local domains to local domains (so opening 127.0.0.1 could still access 127.0.0.1, but internet websites could not). This helped against attackes like https://chaoswebs.net/blog/timebleed-breaking-privacy-with-a-simple-timing-attack.html (using pictures that are by default never blocked in uMatrix).

Since NS removed this feature I had no use for it anymore and switched to uMatrix with a blocklist like this: https://www.privacy-handbuch.de/handbuch_21g.htm
For a long time I wondered why uMatrix (or uBlock) does not come with this feature by default & enabled since it's just a few rules that need to be added.

Are those giant regex even necessary @gwarser ? The 127.xxx one for example is written in this guide only as ||127.0.*^$important,third-party whereas you have some giant regex that is probably much slower? Same with 172 where it might be faster just adding those 16 prefixes separately?
I once coded some script that used similar giant regex - it was painfully slow. I speed it up immensely by breaking the regex apart in smaller ones (not possible here of course) and then even more by replacing some regex with simpler string matchers.

As what does a WSS register in uMatrix? Fetch/XHR?

ilikenwf · 2020-05-27T02:15:30Z

I think the regex is there to protect all of the IANA reserved IP range...

…

On 5/26/20 9:08 PM, ganego wrote: NoScript had "ABE (Application Boundaries Enforcer" enabled by default which basically blocked access from non-local domains to local domains (so opening 127.0.0.1 could still access 127.0.0.1, but internet websites could not). This helped against attackes like https://chaoswebs.net/blog/timebleed-breaking-privacy-with-a-simple-timing-attack.html (using pictures that are by default never blocked in uMatrix). Since NS removed this feature I had no use for it anymore and switched to uMatrix with a blocklist like this: https://www.privacy-handbuch.de/handbuch_21g.htm For a long time I wondered why uMatrix (or uBlock) does not come with this feature by default & enabled since it's just a few rules that need to be added. Are those giant regex even necessary @gwarser <https://github.com/gwarser> ? The 127.xxx one for example is written in this guide only as |||127.0.*^$important,third-party| whereas you have some giant regex that is probably much slower? Same with 172 where it might be faster just adding those 16 prefixes separately? As what does a WSS register in uMatrix? Fetch/XHR? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1070 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADFIOWPSK4FBUZNAUC7GWLRTRY2PANCNFSM4NKQJFZA>.

liamengland1 · 2020-05-27T02:27:51Z

Web devs could get mad if this is implemented. Third-party requests to localhost are often used to detect the presence of an application on a user's computer, which can be malicious, but also serves a purpose. For example, discord uses websockets on localhost to determine if a user already has the discord application installed, and communicate between the website and application. This happens on invite pages like this: https://discord.com/invite/valorant

Another example - website-application communication on localhost also occurs on https://screencast-o-matic.com/screen-recorder. (click the Launch Free Recorder button)

ilikenwf · 2020-05-27T02:33:54Z

Quite frankly it's none of their damned business because this behavior is in and of itself malicious in the case of checking multiple ports. If we can't address this fully here it should be a permissions system built into Firefox, similar to canvas permissions IMO...as should the ability to embed chrome:// images and CSS fingerprint.

…

On 5/26/20 9:28 PM, LE wrote: Web devs could get mad if this is implemented. Third-party requests to localhost are often used to detect the presence of an application on a user's computer, which can be malicious, but also serves a purpose. For example, discord uses websockets on localhost to determine if a user already has the discord application installed, and communicate between the website and application. This happens on invite pages like this: |https://discord.com/invite/valorant| Another example - website-application communication on localhost also occurs on |https://screencast-o-matic.com/screen-recorder|. (click the |Launch Free Recorder| button) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1070 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADFIOUPTBJRCUD7WIMKTNLRTR3DHANCNFSM4NKQJFZA>.

ganego · 2020-05-27T07:50:32Z

https://screencast-o-matic.com/screen-recorder

Interesting, they seem to actually use the TimeBleed attack vector whereas discord uses some XHR according to uMatrix.
Firefox network inspector for the site does not show anything. That really sucks.

gwarser · 2020-05-27T14:39:36Z

Are those giant regex even necessary @gwarser ?

Yes. You can have local server hosted on any IP. Does not matter if Ebay is scanning only 127.0.0.1. Other pages can scan for something else.

The 127.xxx one for example is written in this guide only as ||127.0.*^$important,third-party whereas you have some giant regex

This filter is not valid - IP is not equal domain. It will match foo.127.0.com for example.

Replacing || by // will reduce false positives a bit, but can still match in query parameters. [even using start anchor with protocol (|http://127.0.*) will not prevent matching http://127.0.com for example]. Similarly filters like this ||*.local (wildcard next to domain anchor) will match any address with .local in query. ||*.fritz.box^? Why not ||fritz.box^?

that is probably much slower? Same with 172 where it might be faster just adding those 16 prefixes separately?

In uBO these Regular Expressions are tokenized - plain strings are extracted from them. In these filters it will be first octet (172 for example). uBO will use these regexes only when URL contains this token.

Same with 172 where it might be faster just adding those 16 prefixes separately?

@gorhill exposing second octet will help with tokenization?

I'm pretty sure ipv6 address filters from https://www.privacy-handbuch.de/handbuch_21g.htm are also wrong. For example fe80::/10 is not [fe80::*] because it ranges from fe80:0:0:0:0:0:0:0 to febf:ffff:ffff:ffff:ffff:ffff:ffff:ffff.

gorhill · 2020-05-27T15:07:21Z

exposing second octet will help with tokenization?

172 is a good token, it's not a common occurrence in URLs, and if ever it occurs, best to execute a single regex than 16 separate ones.

ilikenwf · 2020-05-28T15:41:55Z

This has now even made it to Schneier:
https://www.schneier.com/blog/archives/2020/05/websites_conduc.html

ilikenwf · 2020-05-28T17:06:52Z

After doing some research, I've found that these fingerprint scripts are all hosted via CNAME on the same domain, which uBO already blocks when CNAME checking is on:

online-metrix.net

Beyond that, it usually is served up as one ore more of these:

clear.png
check.js
fp.swf

It's used by a lot of sites:

https://src.ebay-us.com/fp/check.js?org_id=usllpic0&session_id=1
https://src.ebay-us.com/fp/fp.swf?org_id=usllpic0&session_id=1

https://content.mercadopago.com/fp/clear.png

https://b177.kabbage.com/fp/check.js?org_id=rxemj1ou&pageid=1&session_id=1

https://content3.risk.lexisnexis.com/tags.js?org_id=baa3flhp&allow_reprofile=1&session_id=1

And even more:
https://www.google.com/search?hl=en&q=inurl%3Acheck.js%3Forg_id%3D

Someone also has built a payload decryption tool:

https://gist.github.com/nemec/ea6b21bcd027b81ac1e3fbcfeb01db3e

liamengland1 · 2020-05-30T17:28:51Z

https://news.ycombinator.com/item?id=23361823

ilikenwf · 2020-05-30T19:51:46Z

https://news.ycombinator.com/item?id=23361823

The linked spreadsheet in that article...those domains should probably make it into the blocklists.

https://docs.google.com/spreadsheets/u/1/d/1Nu4lpyZ5PQUIpiLJBddXnr67t5-1y0u40dzyzSYj1gc/preview

gorhill · 2020-05-30T20:04:58Z

No need to add any of those hostnames, online-metrix.net is already blocked by both EasyPrivacy and Peter Lowe's.

ilikenwf · 2020-05-30T20:05:54Z

Yes, I just discovered this fact... [opens mouth and inserts foot]

…

On 5/30/20 3:05 PM, Raymond Hill wrote: No need to add any of those hostnames, |online-metrix.net| is already blocked by both EasyPrivacy and Peter Lowe's. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1070 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADFIOX4FBEDI3OBGJQNR23RUFRHLANCNFSM4NKQJFZA>.

gwarser · 2020-05-31T15:03:14Z

What is still missing in my list is ipv6 handling https://en.wikipedia.org/wiki/Private_network#Private_IPv6_addresses. No one will scan these from a to z, but some software can still be listening on one [address, so specialized fingerprinting scripts may look only for these specific addresses].

Calculator https://www.vultr.com/resources/subnet-calculator-ipv6/

fc00::/7 rages from fc00:0000:0000:0000:0000:0000:0000:0000 to fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff

||[fc*]^
||[fd*]^
fd00::/8 should be included in above - fd00:0000:0000:0000:0000:0000:0000:0000 to fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff
legacy "site-local" fec0::/10 - fec0:0000:0000:0000:0000:0000:0000:0000 to feff:ffff:ffff:ffff:ffff:ffff:ffff:ffff

||[fec*]^
||[fed*]^
||[fee*]^
||[fef*]^
autoconfiguration? fe80::/10 - fe80:0000:0000:0000:0000:0000:0000:0000 to febf:ffff:ffff:ffff:ffff:ffff:ffff:ffff

||[fe8*]^
||[fe9*]^
||[fea*]^
||[feb*]^

I'm not sure my filters above are actually valid or if these addresses can be so easily matched by simple text comparison with wildcard. Is [ and ] invalid in normal domain name? Someone with better knowledge about ipv6 address formatting will need to say. It may be good based on this: https://en.wikipedia.org/wiki/IPv6_address#Representation

And dang :/ - http://[::ffff:127.0.0.1]:8080/ is also valid https://en.wikipedia.org/wiki/IPv6_address#Transition_from_IPv4. Browser converts this to http://[::ffff:7f00:1]:8080/. It will be ::ffff:0000:0000 to ::ffff:ffff:ffff. Another regex?

~~It seem unique (has unique application), so maybe~~ ||[::ffff:*]^ ~~will be ok~~. Maybe not, but this is simple dec to hex translation - should not be hard to create good filter.

10.*
||[::ffff:0a*]^ this is bad, with short syntax leading 0 is ommited
172.16-31.*
||[::ffff:ac1*]^
192.168.*
||[::ffff:c0a8:*]^
127.*
||[::ffff:7f*]^

I will add these filters if they look good for you (anyone?).

How to efficiently add ws: protocol regex filters?
Copy-paste, replace https? by wss?, append ,websocket to options? (http|ws)s? will not be tokenized.

Testing tokenization: https://regex101.com/r/gNYvlE/2 [rev. 2 start and end of string anchors must be removed]

Hmmm, /^\w+://... seem to pass test? And even ^\w{2,5}://.... I can also use two versions with http...$~websocket and ws...$websocket.

Yuki2718 · 2020-06-02T09:36:07Z

I think @gwarser 's list is worth being added to stock lists but with default-disabled until/unless we gather enough evidences that it doesn't cause FP as warned by @llacb47 . I have been blocking these local accesses, though with problematic rules until I see it, and want to think there was no serious FP but IIRC some videos made requests to these local addresses which I haven't investigated.

Security-conscious people have long been trying to block these accesses but unfortunately rules shared on Wilders or Malwaretips are wrong and don't work. I know nothing can stop those people from importing wrong or poorly written filters but making a proper list included in stock lists will mitigate this.

(Off-topic) @ilikenwf
online-metrix.net is nothing new, they've been around for years and included in Firefox' Tracking Protection and all the major hosts files except for Dan Pollock's. Unlike their early days now they're only on payment sites because their supposed objective is preventing XSS/fraud, this appears to be the reason Malwarebytes removed it from their blacklist. Blocking them apparently caused an error on a shopping site but payment was succeeded so I didn't care much, but EasyPrivacy has related whitelists such as
@@||online-metrix.net/fp/tags.js$domain=donorschoose.org
so it and its CNAME counterparts may cause FP for some sites.
Note even without CNAME uncloaking EasyPrivacy blocks many of them by generic filters like /fp/clear.png?.

gwarser · 2020-06-02T11:05:32Z

There's a problem.

I'm not sure my filters above are actually valid or if these addresses can be so easily matched by simple text comparison with wildcard. Is [ and ] invalid in normal domain name?

||[fc*]^ will match [fc00:..., [fc0:... (false positive), [fc:... (false positive).

Regex filters will not be tokenized because of [.

And if someone will like to use IP scanning maliciously, it's very easy to convert ipv4 to ipv6 gwarser/filter-lists#6

I think special syntax for IP addresses is needed.

ilikenwf · 2020-06-18T20:58:02Z

You all realize we're the first ones to actually even address this issue? Not even mozilla has commented..."it's a feature, not a bug..."

ganego · 2020-06-19T06:53:41Z

@ilikenwf No this has been known for years now, especially since NS lost ABE, with several "exploits" using this "bug". Unfortunately nobody really cares.
See for example arkenfox/user.js#319

I still think uB and uM should come with a default enabled blocklist that works like ABE.

gorhill · 2020-10-11T17:12:50Z

||[fc*]^ will match [fc00:..., [fc0:... (false positive), [fc:... (false positive).

Regex filters will not be tokenized because of [.

Well ||[fc*]^ is already untokenizable so using a regex instead won't make the filter worst.

gwarser · 2020-10-11T17:23:48Z

@gorhill gwarser/filter-lists@eebda2a

gorhill · 2020-10-11T17:30:40Z

Given I see no impact on benchmarking results, should I offer this list as an option in stock filter lists? Under "Privacy"? (Not enabled by default)

gorhill · 2020-10-11T18:31:23Z

You can halve the number of regex filters to maintain by simply replacing ^https?:... with ^\w+:... and dropping the ~websocket option and removing the ^wss?:... version of the filter -- the resulting filters will still be tokenizable.

Yuki2718 · 2020-10-12T08:49:56Z

should I offer this list as an option in stock filter lists? Under "Privacy"? (Not enabled by default)

+1 but another option will be to put under Malware domains (better to rename Security if possible). These rules will prevent CSRF (and DNS binding, if anyone care about) to routers, essentially the superior alternative of now-gone Noscript's ABE. You can see how many home routers have suffered CSRF vulnerability nowadays by just searching in CVE database.

cleanup

gorhill · 2020-10-12T17:00:53Z

and DNS binding

This wouldn't take care of DNS rebinding, for this uBO would need to filter according to IP address information, not request URL. IP address information become available at webRequest.onHeadersReceived() time in Firefox, later at webRequest.onResponseStarted() time with Chromium.

Yuki2718 · 2020-10-13T08:37:06Z

@gorhill Ah, you're right, as it makes the victim to access local IP as evil.com. Thx for correction.

reallyuniquename · 2020-10-13T19:40:53Z

@gorhill

IP address information become available at webRequest.onHeadersReceived() time in Firefox

A bit off-topic but uBlock tries to uncloak CNAME before actual HTTP request is fired, right?

gorhill · 2020-11-22T12:52:36Z

uBlock tries to uncloak CNAME before actual HTTP request is fired, right?

Yes.

@gorhill

as pointed out by @gorhill in uBlockOrigin/uBlock-issues#1070

gorhill · 2024-04-15T00:06:28Z

Enabling Block Outsider Intrusion into LAN list is the solution for this issue.

Related issue: uBlockOrigin/uBlock-issues#1070

uBlock-user added the something to address something to address label May 26, 2020

uBlock-user added enhancement New feature or request and removed something to address something to address labels May 26, 2020

gwarser added a commit to gwarser/filter-lists that referenced this issue May 26, 2020

Fix ipv6 loopback address syntax

62fff6a

as pointed out by @gorhill in uBlockOrigin/uBlock-issues#1070

ilikenwf mentioned this issue May 28, 2020

general discussion on FPing: have at it arkenfox/user.js#942

Closed

This was referenced May 31, 2020

Can we have an option to block access to 127.0.0.1 and/or other internal only IP addresses? netblue30/firejail#3427

Closed

Add ipv6 local network filters gwarser/filter-lists#3

Closed

gwarser referenced this issue in gwarser/filter-lists Oct 12, 2020

LAN: reduce duplication

69a2392

cleanup

0xRustlang mentioned this issue Mar 27, 2021

please add an option to completely block access of any third party website to LAN CookieJarApps/SmartCookieWeb#244

Open

gwarser added a commit to uBlockOrigin/uAssets that referenced this issue Nov 18, 2021

Fix ipv6 loopback address syntax

18547cf

as pointed out by @gorhill in uBlockOrigin/uBlock-issues#1070

gwarser mentioned this issue Aug 31, 2023

Add AdGuard's $network support on Firefox #2792

Closed

8 tasks

gorhill closed this as completed Apr 15, 2024

garry-ut99 mentioned this issue May 6, 2024

Why is 0.0.0.1 blocked? easylist/easylist#19041

Closed

gorhill added a commit to gorhill/uBlock that referenced this issue Sep 10, 2024

Add support for lan/loopback values to ipaddress= option

030d733

Related issue: uBlockOrigin/uBlock-issues#1070

stephenhawk8054 added a commit to uBlockOrigin/uAssets that referenced this issue Sep 20, 2024

address uBlockOrigin/uBlock-issues#1070 (comment)

88e70ca

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Possibility of Blocking Requests to localhost and Reserved IP Addresses by websockets? #1070

Possibility of Blocking Requests to localhost and Reserved IP Addresses by websockets? #1070

ilikenwf commented May 26, 2020

gorhill commented May 26, 2020

ilikenwf commented May 26, 2020

gorhill commented May 26, 2020

ilikenwf commented May 26, 2020 •

edited

Loading

ganego commented May 27, 2020 •

edited

Loading

ilikenwf commented May 27, 2020 via email

liamengland1 commented May 27, 2020

ilikenwf commented May 27, 2020 via email

ganego commented May 27, 2020

gwarser commented May 27, 2020 •

edited

Loading

gorhill commented May 27, 2020

ilikenwf commented May 28, 2020 •

edited

Loading

ilikenwf commented May 28, 2020 •

edited

Loading

liamengland1 commented May 30, 2020

ilikenwf commented May 30, 2020 •

edited

Loading

gorhill commented May 30, 2020

ilikenwf commented May 30, 2020 via email

gwarser commented May 31, 2020 •

edited

Loading

Yuki2718 commented Jun 2, 2020

gwarser commented Jun 2, 2020 •

edited

Loading

ilikenwf commented Jun 18, 2020

ganego commented Jun 19, 2020

gorhill commented Oct 11, 2020

gwarser commented Oct 11, 2020

gorhill commented Oct 11, 2020 •

edited

Loading

gorhill commented Oct 11, 2020

Yuki2718 commented Oct 12, 2020 •

edited

Loading

gorhill commented Oct 12, 2020 •

edited

Loading

Yuki2718 commented Oct 13, 2020

reallyuniquename commented Oct 13, 2020

gorhill commented Nov 22, 2020

gorhill commented Apr 15, 2024

Possibility of Blocking Requests to localhost and Reserved IP Addresses by websockets? #1070

Possibility of Blocking Requests to localhost and Reserved IP Addresses by websockets? #1070

Comments

ilikenwf commented May 26, 2020

gorhill commented May 26, 2020

ilikenwf commented May 26, 2020

gorhill commented May 26, 2020

ilikenwf commented May 26, 2020 • edited Loading

ganego commented May 27, 2020 • edited Loading

ilikenwf commented May 27, 2020 via email

liamengland1 commented May 27, 2020

ilikenwf commented May 27, 2020 via email

ganego commented May 27, 2020

gwarser commented May 27, 2020 • edited Loading

gorhill commented May 27, 2020

ilikenwf commented May 28, 2020 • edited Loading

ilikenwf commented May 28, 2020 • edited Loading

liamengland1 commented May 30, 2020

ilikenwf commented May 30, 2020 • edited Loading

gorhill commented May 30, 2020

ilikenwf commented May 30, 2020 via email

gwarser commented May 31, 2020 • edited Loading

Yuki2718 commented Jun 2, 2020

gwarser commented Jun 2, 2020 • edited Loading

ilikenwf commented Jun 18, 2020

ganego commented Jun 19, 2020

gorhill commented Oct 11, 2020

gwarser commented Oct 11, 2020

gorhill commented Oct 11, 2020 • edited Loading

gorhill commented Oct 11, 2020

Yuki2718 commented Oct 12, 2020 • edited Loading

gorhill commented Oct 12, 2020 • edited Loading

Yuki2718 commented Oct 13, 2020

reallyuniquename commented Oct 13, 2020

gorhill commented Nov 22, 2020

gorhill commented Apr 15, 2024

ilikenwf commented May 26, 2020 •

edited

Loading

ganego commented May 27, 2020 •

edited

Loading

gwarser commented May 27, 2020 •

edited

Loading

ilikenwf commented May 28, 2020 •

edited

Loading

ilikenwf commented May 28, 2020 •

edited

Loading

ilikenwf commented May 30, 2020 •

edited

Loading

gwarser commented May 31, 2020 •

edited

Loading

gwarser commented Jun 2, 2020 •

edited

Loading

gorhill commented Oct 11, 2020 •

edited

Loading

Yuki2718 commented Oct 12, 2020 •

edited

Loading

gorhill commented Oct 12, 2020 •

edited

Loading