don't expose a localhost https endpoint when there is no developer certificate #32361
Comments
|
cc: @javiercn
Can you say more about the failures you hit?
This should already be the default behavior. However, there's a gap between providing a dev cert and having anything trust it. Can you confirm there is no default https endpoint if you remove the dev cert?
The main goal is to make the dev environment closely match the production environments where HTTPS should be enabled. Many features behave differently between http and https. You can suppress the default https endpoint by providing an http endpoint when running them app. |
Triage: Maybe we should avoid installing the dev cert by default on Linux. If you want the dev cert it is easy to install (although harder to get it actually working). |
I've been clicking the ignore certificate in Firefox as long as I remember. I'm now playing around with tye and there are some manual steps in the repo that are based on some stackoverflow: https://github.com/dotnet/tye/blob/main/docs/tutorials/hello-tye/00_run_locally.md#certificate-is-invalid-exception-on-linux. Like with the ASP.NET apps, this breaks UX. The steps here are distro specific. Changes to the config files can't be checked into source control. And
If the dev cert is removed, the server fails to start:
I'm looking for a way to opt out for all my apps at once, rather than have to change it per app.
Yes, I think that makes sense. |
Ok, that's a trust issue. We know that story needs work. AddressBinder.AddressesStrategy means that an https address is being passed in manually, it's not using the defaults. Is it in the launchsettings.json? |
Yes: |
|
Yes it's in the template, and only includes https if that checkbox is marked. VS and dotnet run read this and set environment variables when starting the app. You only get the server default behavior if you remove this line. |
|
@Tratcher can we special case this in Kestrel? Something like: ignore the "https" binding for "localhost" when the default development certificate is missing and the environment is "Development"? |
|
@BrennanConroy we are not changing the HTTPS experience for Linux to make it different from other platforms. We already provide options to people who don't want to install the https dev cert or use https within their templates. The certificate installation can be skipped on first run by setting the following environment variable |
You can still browse to the site, and you can add a permanent exception if Firefox for |
The development experience for Linux for HTTPS is not the same as for other platforms: on Windows and macOS it works out-of-the box, on Linux it does not.
I created this issue when confronted with the steps in https://github.com/dotnet/tye/blob/main/docs/tutorials/hello-tye/00_run_locally.md#certificate-is-invalid-exception-on-linux. These steps describe a workaround and not a real solution. I've made a suggestion to improve the situation for tye here: dotnet/tye#1025.
For general ASP.NET I think it would be an interesting option to not perform the https bind when there is no developer cert. This now gives an error, which could become a warning. That makes it possible for the individual developer to opt-out. So when he checks out some repo that used the template with the default https option, that code runs for him too. |
That is not the case, you are complaining about the certificate not being trusted and what happens is:
dotnet uses the machine trust root for determining whether a certificate is trusted or not. On windows we use the machine store, on Mac OS we use the system keychain. On Linux there is no centralized trust root and the different trust roots change across distros and browsers. Browsers on Mac and Windows rely on the system trust root for determining if the certificate is valid, Firefox does too if you provide the appropriate setting (which we cover in docs). On Linux we provide instructions on how to trust the certificate in all three major browsers (Edge, Chrome, Firefox). In all cases unless the user runs When they run the command we provide all that is necessary on all OS to work. On Windows and Mac we rely on the system trust roots, on Linux we point you to the steps you need to follow to achieve the same result. In all cases we tell you the manual steps to achieve a trusted dev-cert across the different browsers and OSs. The only difference is that some of the steps are automatic on Windows and Mac and require a few steps that you perform as a one time gesture on Linux to make it happen. While I understand your frustration about the experience on Linux, I don't think that's a good enough reason to change the default experience. The experience is not as good as on Windows or Mac because the HTTPS experience is in general "harder" on Linux, however even in the Linux case you can be up and running in 5 minutes by following the instructions we provide. As for this issue on tye, this is caused by a "long standing bug" that was fixed on Open SSL at least 6 month ago. Recent versions of most popular distros contain newer versions of Open SSL that contain this fix. HTTPS by default is a fundamental development default for several years and is not something we have plans to change. While I understand it might make things more difficult in some more advanced scenarios and require manual steps to solve, that doesn't mean we should switch defaults at the cost of offering a different experience across OS flavors. If the issue is with the certificate not being trusted in the container image, we can instead change how we build the image to add the certificate to the trust store for the distro as part of building the image, we don't have to change the experience for everyone not using containers on Linux. If we ever see a convergence on Linux to provide a common system trust root, we will likely consider adopting that within the dev-certs tool itself. However, given that it is not the case today we provide docs for how you can do it yourself. Our "limit" with the I hope this helps bring some clarity about our reasoning. |
We want to avoid special cases like that in the runtime, they're hard to distinguish from a real issue that you need to know about. In this case the app is explicitly telling us to bind to https but we can't and tell you accordingly. It works as expected when you remove the https address or the launchsettings file, right? |
Are you suggesting there is a better way to do this now?
I spent more than 5 minutes and I wasn't able to make it work. I use Fedora, not Ubuntu:
Yes. Though I don't think I should push that as a change in version control if the HTTPS dev cert works fine for other collaborators. HTTPS on localhost isn't providing any real security. Instead of causing me issues, I wish there was a way to opt-out. |
As @javiercn points out there is an opt-out, --no-https when create the project or unticking the box in Visual Studio HTTPS on localhost is not there for security, it's there to emulate what a real deployment looks like, to allow you to use oauth and other federated identities, to enable samesite cookies to work in the same way they would in production. Removing it, even if it's "just" on linux makes that experience impossible, and gives you surprises on deployment, something we want to avoid. |
launchsettings.json is the opt-in/out mechanism. If your team has opted in for a project then the only way for you to opt out is by providing a higher priority config like at the command line: |
blowdart
added
the
✔️ Resolution: Answered
|
Having the HTTPS has some advantages. Having no method to trust the certificate brings disadvantages. Because the certificate trusting is an issue for Linux devs, the request here is to opt-out at the machine developer level. Based on your feedback, I'll close the issue. |
|
@blowdart @Tratcher @javiercn I investigated further what is needed to trust the ASP.NET developer certificate on Fedora. Fedora does not want to accept a self-signed certificate in the system store. It needs a CA certificate. I wrote a console app that generates a self-signed CA certificate, and uses that to sign the ASP.NET HTTPS development certificate: https://github.com/tmds/linux-dev-certs/. I'll look into adding this to the |
|
@tmds thanks for the research. Unfortunately that's not something we are willing to accept. Creating a CA certificate on the user machine opens up the machine to way more liability and is not something that we want to manage and maintain. Can you post the fedora version you are using as well as the openssl version with |
There is no need to store the CA key, so I think that mitigates the security issues?
I'm on Fedora 33. Fedora releases are only supported for 1 year, so I'll be moving to F34 in the next half year or so. |
|
@tmds what's the openssl version |
|
|
@javiercn @blowdart you are not a fan of using CA certs. I would prefer to do without also. If you can make it work without them, that would be great. If you cannot, consider using them instead of having something broken. I can make a PR to add the using a CA cert, or try out some things if you have suggestions. |
|
@tmds I'm not sure what's happening in fedora, this works on Ubuntu following the instructions in the docs. I want to clarify some things:
I understand this is not what you are expecting to hear, however the change that you are suggesting increases the complexity for us in either having to support a more complex setup across all OSses or having to support a special setup for Linux machines. In addition to that, we don't have the resources to take on the challenge of trusting certs across al Linux distributions and to keep it working over time given the large variability in this space. We have provided fairly generic instructions on how to do this in popular distros like Debian/Ubuntu, it should be possible to adapt those instructions for other distros. If our instructions are not enough, we suggest looking for help on the specific distro channels about how to diagnose what is not working correctly and how it can be made to work, we simply don't have the capacity nor the expertise for such task. We believe that the cert we generate is completely valid and should be usable in all the platforms (in fact, it works fine on Ubuntu as per the image below) and that if it can't be successfully trusted in a given platform it means there is a bug on the platform.
|
Have you considered it may not be a bug, but a decision to not accept non-CA certificates in the system CA store?
It doesn't work for the popular Fedora distribution. I just tried to add the self-signed certificate to the system CA store on the Microsoft Alpine image, and it did not work either. |
|
@tmds I downloaded the latest Fedora and followed instructions similar to the ones that we provide. I was able to make it work in all scenarios (as per the image below). I essentially created two APIs, front end and backend and used HttpClient to call from frontend to backend and accessed frontend from all browsers.
|
|
It required a bit of trial and error and adjustment to fedora specific stuff, but in the end it worked. I don't have a "validated" script or set of steps, however its clearly possible to setup, and the instructions are mostly the same. For chrome/edge I did the exact same thing as for Ubuntu. Export the certificate and use the commands below to trust it. certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n localhost -i ./localhost.crt
certutil -d sql:$HOME/.pki/nssdb -A -t "C,," -n localhost -i ./localhost.crtThis has not always work correctly and I ended up removing $HOME/.pki/nssdb before I got it to work (AFAIK thats chrome cert store and it will recreate it when its not there, so do it at your own risk). For Firefox I had to follow the steps described here https://shivering-isles.com/Manage-Firefox-on-Fedora to setup the policy because this is distro specific for Fedora. For open SSL I copied the cert to the locations described here
And ran I closed the console by accident so I don't know the exact steps I took, It might have taken some trial and error before it. I think its possible to do this the |
|
@javiercn awesome you got it to work! I'm interested in particular in how you make the dotnet-to-dotnet work. This is what happens for me (on my Fedora 32 desktop): (edit: added missing Run a Create a Program.cs: And now run: Also: |
|
@tmds I believe your issue is either with the certificate format or with the openssl version. I believe you need to use PEM format (look at the updated command below). You shouldn't need to clean the certificate either (that will force a new one to be created) dotnet dev-certs https --clean
dotnet dev-certs https -ep /tmp/aspnet.crt --format PEM
sudo cp /tmp/aspnet.crt /etc/pki/ca-trust/
sudo cp /tmp/aspnet.crt /usr/share/pki/ca-trust-source/
sudo update-ca-trust In addition to that, if those changes don't work I would try the There is a long-standing issue in openssl that was solved on 1.1.1h and onwards and our docs reflect that. I suspect you might be running into that, however its not fully clear since you mentioned you are using 1.1.1k, so you might have updated manually. Other than that, I can only think of the openssl installation not having the certificate in the CA bundle. I want to be very clear that I'm not an expert on this area (Linux tooling around certs) and that it is incredibly frustrating because it is trial-and-error every time I have to do it. That said, the things that have worked for me to troubleshoot things are:
|
I tried again, adding the missing
This didn't have an effect either.
My Fedora 32 machine doesn't meet this version requirement.
On my Fedora 33 machine: The error message is different:
For some reason I have two: I tried these steps in a Fedora 34 container image, this is what I get there: @javiercn can you try and figure out again how you made it work? |
|
@omajid can you please try the steps outlined in #32361 (comment)? In particular the ones that configure OpenSSL. Those should make You need a recent version of Fedora (33+) so you have openssl 1.1.1h+. I am not able to make it work, but maybe pebkac, or maybe I messed up my machine. |
|
I just found the missing piece and put everything in script form echo 'pref("general.config.filename", "firefox.cfg");
pref("general.config.obscure_value", 0);' > ./autoconfig.js
echo '//Enable policies.json
lockPref("browser.policies.perUserDir", false);' > firefox.cfg
echo "{
\"policies\": {
\"Certificates\": {
\"Install\": [
\"aspnetcore-localhost-https.crt\"
]
}
}
}" > policies.json
dotnet dev-certs https -ep "${HOME}/work/linuxhttps/localhost.crt" --format PEM
sudo mv autoconfig.js /usr/lib64/firefox/
sudo mv firefox.cfg /usr/lib64/firefox/
sudo mv policies.json /usr/lib64/firefox/distribution/
mkdir -p ~/.mozilla/certificates
cp ./localhost.crt ~/.mozilla/certificates/aspnetcore-localhost-https.crt
# Trust Edge/Chrome
certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n localhost -i ./localhost.crt
certutil -d sql:$HOME/.pki/nssdb -A -t "C,," -n localhost -i ./localhost.crt
# Trust dotnet-to-dotnet (.pem extension is important here)
sudo cp localhost.crt /etc/pki/tls/certs/localhost.pem
sudo update-ca-trust
# Cleanup
rm localhost.crtThe trick was that the cert needs to go into the openssl folder and not on the system ones. |
@javiercn great! I'll give these instructions a try. |
|
@tmds this time around I've done it on a VM with checkpoints, so I just ran the script. It should at the very least, work for you on a clean VM using fedora 34 There is also https://github.com/javiercn/linuxhttps which I've used to test it, run the two projects and go to |
|
The steps work for me on Fedora. The system tools (like Some suggestions to improve the script:
I've applied these suggestions in the script below. Feel free to take the bits you like. Besides updating the docs, it would be even better if we can make the RHEL7 and RHEL8 don't meet the OpenSSL version requirement. Do you know what bug was fixed in OpenSSL 1.1.1h+? dnf list installed nss-tools >/dev/null 2>&1 ||
(echo "Installing dependencies." && \
sudo dnf install -y nss-tools)
echo "Exporting developer certificate."
DEV_CERT="$HOME/aspnet-$USER.pem"
dotnet dev-certs https -ep "$DEV_CERT" --format PEM
CERT_DB=$(echo "$HOME/.mozilla/firefox/*.default-release")
[ -d "$CERT_DB" ] && echo "Adding certificate to Firefox default profile certificates." && \
certutil -d "$CERT_DB" -A -t "C,," -n localhost -i "$DEV_CERT"
CERT_DB="$HOME/.pki/nssdb"
[ -d "$CERT_DB" ] && echo "Adding certificate to Edge/Chrome certificates." && \
certutil -d "$CERT_DB" -A -t "C,," -n localhost -i "$DEV_CERT"
echo "Adding certificate to System certificates."
sudo cp "$DEV_CERT" /etc/pki/tls/certs
sudo update-ca-trust
rm "$DEV_CERT" |
|
@tmds I'm glad that we sorted this out The openssl issue that was fixed is this one: WRT to Firefox the problem with not setting it up at the system level is that it might impact automation tools. We could do it at the user profile level, however it will have to be "smarter" than looking at the
This is a good idea, though not sure how many devs use the same machine with multiple users for development.
This might be the case, we would need to test this out, since I read something about a bug requiring this when I did this the first time. With regards to supporting this on .NET Core I think we would be open to it, however I want to set expectations here, since it's not just about making In addition to that, we wouldn't support
If we add support for this, we would need to try and cover as many distros as possible. We would need to make a list of the following per distro:
Implementation wise, we could potentially divide this into 3 areas:
I would avoid us having distro specific code where possible. It would be neat if we could embed distro/version specific scripts/metadata to drive the process so that there's only one path available for Linux that is largely driven by distro specific resources. That way, when something changes we just need to update a resource for a specific distro/version which greatly limits the chances of regressing other distros (and creates an implicit boundary of the scenarios to revalidate). All this is to say that this is a LOT of work and nothing that is planned for 6.0. We would be open to getting this done in 6.0 provided that this goes through the right feature design/process if someone wants to tackle it, however its not something we will ourselves tackle. The first thing we would require would be manual steps to setup and validate HTTPS on all distros and once we have that we could look at supporting it in There is a big chance that this will bring in tons of new issues and maintenance work to the repo, and if we can't ensure that we have a low cost story for validating that it works on each specifically supported distro and that we can make changes easily, we won't want to sign up for owning this. @tmds If this is something you are interested in contributing we would be open to accept it provided is based on the expectations set above. Let us know if this is the case and we can open a new issue for it. Given the size of it, there might be some things that could be "parallelized" or split between folks to enable other people from the community to contribute. I wouldn't expect you to tackle on all this work, since as you can tell this is a really large effort in terms of cost and maintenance for us. |
|
I hope I'll find some time for this in June. note: most of the distros in the list won't meet the OpenSSL version requirement. |
|
@tmds great! I have filed an issue with more details here and I'll see if I can rally folks to do this I tried several distros with |
ghost
locked as resolved and limited conversation to collaborators
amcasey
added
area-networking
By default kestrel exposes an HTTPS endpoint for localhost using a development certificate.
On Linux, the
dotnet dev-certs https --trustdoesn't work well leading to a bricked development experience.It would be nice if there was a global way to opt-out of the HTTPS localhost bind.
One option may be to not bind localhost HTTPS when there is no development certificate.
I think the reason for having the HTTPS endpoint is to be secure by default, though not having it on
localhostdoes not make things insecure by default.@Tratcher @halter73 @jkotalik what do you think?
The text was updated successfully, but these errors were encountered: