-
Notifications
You must be signed in to change notification settings - Fork 548
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SIP008] Online config #89
Comments
Where's authentication? This proposal will get GFW-ed within days. You just don't publish your server on the Internet. |
只要使用了 HTTPS 并且保证 URL 的安全,GFW 就无从得知包括 URL 在内的信息。 我提出这个提案的目的主要是为了解决朋友之间共享服务器时能便捷的更新代理信息。 朋友之间没有必要把 URL 公布到网上,代理站需要 URL 中的信息标识用户,更加不会公布 URL。 |
So you're using a secret in URL to reveal the secret key. The second secret is authentication. And also by doing this you have effectively introduced a handshake for the protocol. One could work around the handshake and design a totally profitable protocol out of this:
|
I suggest to host a QR code on a secret HTTPS website. It should do the same trick. |
@Mygod I still didn't get the point about add authentication over HTTPS, and a complicated way at that. |
Your URL can't be a well-known fixed string. And therefore there will be a redundant second secret. |
The point is that one can design a better protocol with a secret URL as a single entry point than forcing it into the current framework (i.e. as an SIP), not that per-user authentication is necessary. |
@madeye So, I have to notify everyone about config changed. Users have to scan QR code many times to add servers, find out which one is the new server, which one should be removed. Yeah, We need more manual action to live in the modern digital world. I think my servers is not good and stable as you have. And some of my friends don't meet the tech requirements to across the wall. Deeply hope similar feature implemented in the future. |
@CzBiX I agree that this proposal would be useful in many scenarios. In fact, it's also easy to implement. I believe @Mygod can implement it in one line with Scala. 😄 However, to become a new feature, we need to consider more, for example, the authentication requirement here. I think you can keep thinking about this proposal, polish your design and collect more feedback from the community. If so, I guess we would finally have a new SIP. |
Reopen the issue to allow @CzBiX refine his proposal further. |
@madeye Thanks. Sorry, maybe I don't know much about Internet security, I still can't understand why we need more authentications. In my mind, the client can use HTTPS cert to verify server's identity, and server identifies client use auth info exists in URL, the provider can choose which security level and the method they want:
Both of them easy to implement on server side, and easy to use because of just single URL on client side. |
Here's my proposed change if you really want this kind of thing. MotivationCurrent shadowsocks configuration involves many parameters. To add insult to injury, the introduction of plugins (#28) introduces even more configurations. As a result, new ss URLs are usually very bloated (#27). All of this would be fine if the server never changes its configuration. However this is not likely to happen due to following reasons:
Whenever the server updates its configuration, due to the handshake-less nature of the protocol, every client has to update their configuration accordingly. This can be a tedious process if the number of clients is large. Ideally we want to use only one single pre-shared key to authenticate valid users. ProtocolConfigurations via HTTPSFirst, the config file is published on the Internet following one of these two URL schemes with an optional front domain parameter as the fragment: (TODO: which one?)
The web server hosting this ssconf page could be:
It's recommended to put the web server behind a trusted CDN, preferably the ones supporting domain fronting. Upon GET request, the web server validates whether the request is performed using HTTPS and returns an ordinary shadowsocks json configuration in that case. Example: {
"server": "example.com",
"server_port": 1234,
"password": "example",
"method": "chacha20-ietf-poly1305",
"plugin": "xxx",
"plugin_opts": "xxxxx"
} The Client Behavior
Note that in order to guard against MitM attacks, the client now should be stateful to prevent country-scale certificate forgeries. Here's what a console
Remark
|
I think it's better to use a separated program / script to handle this logic (like ss-manager, but for client side), and keep ss-local simple. |
It's just a demo of usage, and also how complicated it is to implement securely. (it's nothing you can do with one line of scala) |
|
I like the idea of an online config. It can be blocked the same way the Shadowsocks server is. However, it's something you access rarely, so it will draw a lot less attention and it will be a lot harder to find and block. I have some thoughts about practicality of the idea. ColocationQuite often the most convenient is to have the config live where the server is, but that means the config will be blocked if the server is. But there are solutions:
One issue we ran into in Outline is that users often don't have a domain name. Which means they can't get a TLS certificate from Let's Encrypt. Because the certificates are self-signed, we need a certificate fingerprint to validate them. The configuration URL should provide a way to specify the TLS certificate fingerprint, so clients can validate certificates when content is served from an IP address. PublishingAlternatively, you may publish the config somewhere else. You'll want to automate that, which means generating some API key and wiring your code properly, which is often not quite straightforward if you've never done it before. Notice that you don't really need the fetch URL to be private. The config page can be encrypted and published publicly anywhere. It may even use HTTP, but you will want an AEAD cipher to validate authenticity. The config URL can have the secret key used to decrypt the content by the client. Ideally in the fragment, to make sure it's never sent to the host server. Publishing requires you to trust the host, which is not true for many people, unless you use the encrypted approach above. Some options for publishing:
URL ShortenerAn alternative to publishing a config is to use an editable URL Shortener. In that case it could redirect directly to the ss:// link. As in the case of publishing, you need to trust the URL Shortener. Backward compatibilityWhen you update the config, make sure the old configuration keeps running until all your clients have migrated to the new config. Otherwise you will break them. |
A lots of people open issues for implement of this feature in
To be simple to edit, I show the example in yaml format: # Identify the group of servers.
# Can be used by client to track the servers from same source.
# When update, the client can use the id to deleted the old servers
# which has been removed from the server list.
id: com.xxx.foo/bar
# Use the SIP002 format URL instead of a new spec. Keep it simple.
servers:
- ss://[email protected]:8888#Example1
- ss://[email protected]:8888/?plugin=obfs-local%3Bobfs%3Dhttp#Example2 |
The client could leverage the same crypto that the Shadowsocks protocol uses, so it's just a matter of reusing libraries. In Outline, we could use our ShadowsocksReader. |
For the config URL, we should consider not only a front domain, but also a domain to resolve and get the IP address from. For example, we can resolve These are the things we may need in what we give the client:
If we want to support self-signed certificates for users without a domain name:
|
I'd like see it become standard. Shadowsocks-windows has another experiment few years ago: My concern:
|
We're finalizing the online config support with an extendable JSON format. Here's an example: https://gist.github.com/madeye/8451355b1baba312dae78a6c8ca4d1a1 Generally, we don't limit any custom fields in the JSON config file. The only required fields are: {
"server": "198.199.101.152",
"server_port": 8388,
"password": "u1rRWTssNv0p",
"method": "aes-256-cfb",
"remarks": "Example 1"
} |
More about authentication
Option 1 is easy, but require POST |
GET is good enough, as we're forcing HTTPS URL for any online config. You can put any restful token in your URL for authentication. |
The latest shadowsocks-android has already supported this SIP: https://github.com/shadowsocks/shadowsocks-android/releases/tag/v5.0.3 |
@madeye maybe it’s time to formalize the format by putting it on our website. |
@database64128 PR is welcome. 😄 |
@madeye I wish I could, but the |
Okay, I forgot you need to add pages here: https://github.com/shadowsocks/shadowsocks-org/wiki |
Thanks y'all for all the brilliant ideas! I just published the draft document of SIP008 Online Configuration Delivery to the wiki. Please review and share your thoughts. Once everything is cleared up, I'll publish the standard document to our website. 😄 https://github.com/shadowsocks/shadowsocks-org/wiki/SIP008-Online-Configuration-Delivery |
Why do these two fields have different naming style? |
@zonyitoo You got me! 😅 Changed to follow the naming convention of |
We have started prototyping support for online config in the Outline clients. I agree with @database64128 in that standarizing this proposal will strengthen the Shadowsocks ecosytem. We would like for Outline clients to remain compatible with standard Shadowsocks servers. Here are some thoughts and needs that we have found so far: URL protocolI believe that registering a custom protocol provides a better user experience than manually adding the URL to the app. Additionally, Shadowsocks users are already familiar with the I realize that URL interception is not possible in every platform, so clients should support manual import. The URL formatWe are exploring how to colocate an access service (online config server) to Outline servers. The current URL format works well for servers that have domain names and the corresponding (CA signed) certificates. Most Outline servers don't have domain names and rely on self-signed certificates for accessing the management API. In order to prevent MITM attacks, we communicate the certificate fingerprint out-of-band to the client, pin it, and validate upong connecting. This solution requires access to native networking APIs, since browsers (webviews) cannot easily pin custom certificates, but it has worked well for Outline servers, so we would like to extend it to online config. In order communicate the certificate fingerprint to the client, we may want to define a more flexible URL format. We propose to encode arbitrary options in the URL tag, much like SIP002 plugin options:
The domain front can be encoded in the tag, along with the remark: Self-signed certificatesWith the new URL format we may avoid displaying a user prompt to confirm untrusted cetificates, as long as the certificate fingerprint is present. The client should terminate untrusted connections that don't provide a valid certificate fingerprint. HTTP redirectsThe proposal states that, "Clients should treat redirects (HTTP 3xx) as errors due to possible certificate issues. Web servers should handle URL changes internally." HTTP redirects can be a way to migrate online configs. The server may redirect clients to an File formatI don't feel strongly about the JSON file formats proposed on this thread. As long as we are able to support mulitple servers, are backwards compatible with Shadowsocks naming conventions, and can arbitarily extend individual configurations. The more important aspect is that the format is standardized. I'm interested in hearing your thoughts and in finding common ground to move this proposal forward. Thanks! |
URL RedirectsI would like to emphasize @alalamav 's case to support HTTP redirects. We ought to support that because it's easy and brings multiple benefits:
This is how it can work:
If the first redirect from the saved URL is a permanent redirect, we should replace the saved URL with the new one, to allow for migrations. We should only do this once we confirm the new config works. A HTTPS url should be able to redirect to a ssconf:// so we can redirect to a location with a self-signed certificate. Config RefreshWe should specify when the client should fetch the config, so the server can migrate a user (for example, to rotate IPs) to a new config seamlessly, without interrupting the existing connections. Note that seamless transitions require supporting an old config until a client migrates to a new config. A client app should fetch the config every time a client hits "Connect", and then again before the config expires. That means we need a way to communicate a config TTL or expiration time to the client somehow. For example, if you have a TTL of 2 hours, that means the server manager can stop supporting an old config 2 hours after a new config is made available. I recommend a TTL instead of an expiration time, because you can keep using the same value without updating. A TTL can be used to stop serving idle configs and save resources. A TTL can be used to force the user to refresh their config in a short period and the server can generate a new one whenever the TTL expires. This provides forward secrecy with Shadowsocks! This TTL can be in the config somewhere or in a HTTP header. I'm still unsure where. File formatThe proposed file format does not differentiate fields that are required from fields that are optional. This will prevent future extensibility. I need to know how to handle field I don't know of. Should I ignore, or should I abort? ID and remarks are optional. method and password are not. Plugin is also required. Outline doesn't support plugins, so we need to know if there's a plugin field we should reject the config. But Outline needs to know that the plugin field exists, and is required. Perhaps we have a structure like: {
"version": 1,
"servers": [
{
"id": "27b8a625-4f4b-4428-9f0f-8a2317db7c79",
"remarks": "Name of the server",
"proxy_config": {
"server": "example.com",
"server_port": 8388,
"password": "example",
"method": "chacha20-ietf-poly1305",
"plugin": "xxx",
"plugin_opts": "xxxxx"
}
},
{
"remarks": "Name of the server",
"proxy_config": {
"server": "example.com",
"server_port": 8388,
"password": "example",
"method": "chacha20-ietf-poly1305",
"plugin": "xxx",
"plugin_opts": "xxxxx"
}
}
]
} And we define that every field under |
|
@database64128 We'll probably support the standard specified here, but we'll go further and extend it to support the advanced cases we need (self-signed certs, redirects, config rotation). If we can demonstrate it works well, then we can present back to the community for standardization. On this proposed standard, It would be great to follow the caching directives whenever possible. In particular, The |
@studentmain @database64128 @madeye What's the retrieval logic for shadowsocks-android and shadowsocks-windows?
|
In the upcoming shadowsocks-windows v5, users will be able to specify an interval for automatic retrieval. Redirects are followed. |
|
The |
This comment has been minimized.
This comment has been minimized.
@database64128, @madeye , I'm trying to understand what what ss-windows and ss-android clients currently support to see if Outline can generate links that will work with your clients. Would you mind answering for your client:
|
It currently doesn't. But we do have plans to add support in an upcoming version.
The certificate gets rejected and the fetch fails.
Yes.
No. It shouldn't be difficult to implement, but I'm not sure why we would need that. It doesn't sound like a reliable way to communicate the online config URL change.
No. Again, it's easy to implement, but wouldn't it be better to just stick to the spec and return an online config JSON?
Fetch on app startup. If you are looking for a reference implementation of SIP008-compliant online config provider, take a look at database64128/shadowsocks-uri-generator. It's a set of tools that manage federated Shadowsocks servers, with support for Outline API. Server config is delivered per the SIP008 spec. Online config URL delivery and user interactions are enabled by the Telegram bot. |
Since I no longer have control over the SIP008 spec, I'm starting Open Online Config for online config protocols and API specs. Everyone is welcome to review the draft of Open Online Config 1 that supersedes SIP008. |
SIP008 standardizes unspecified SIP008 format MIME types I suggest, use If needed, suggest application for a new media type, see https://www.iana.org/form/media-types HTTP ExampleRequest: GET /to/path HTTP/1.1
Accept: text/sip008+json, text/uri-list Response: HTTP/1.1 200 OK
Content-Type: text/sip008+json References |
Motivation
When the server changes SS config(such as address, port, method, password), the client can auto-update config without user interaction.
Overview
Basic file format
Basically, we follows the original JSON config format of shadowsocks. The required fields are:
To support multiple servers, put those server objects into one JSON array.
Extended file format
The JSON config can be extended with any client specific field. Any unsupported filed can be just ignored.
Remark
id
,server
,server_port
,password
,method
inservers
.TODO
The text was updated successfully, but these errors were encountered: