Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xrdp fails without IPv6 #714

Closed
MichaelSweden opened this issue Mar 28, 2017 · 18 comments
Closed

xrdp fails without IPv6 #714

MichaelSweden opened this issue Mar 28, 2017 · 18 comments
Labels

Comments

@MichaelSweden
Copy link
Contributor

History
The xrdp service don't start on one of my machines, if nic uses static ip address. I have seen the fault on another machine (different hardware) also, however it depends on how Ubuntu Server 16.10 is installed. I skip all details but I can say that I found out that IPv6 isn't always available during start of services at boot. This let me find out that xrdp can only work if an IPv6 network (IPv6 address on the nic) is available. For me right now I have solved by start the xrdp a second time a few seconds later. However, if a user want to run without IPv6 it is problem.

Platform
Ubuntu Server 16.10 amd64 (also Xubuntu 16.10 and Kubuntu 16.10)

Version
xrdp 0.9.0~20160601+git703fedd-3, included in Ubuntu Server 16.10
(the proposal is done for the latest github code)

Fault
The error messages at boot of system:
xrdp-sesman [ERROR] bind error on port '3350': 22 (Invalid argument)
xrdp [ERROR] xrdp_listen_main_loop: listen error, possible port already in use

How to reproduce
Boot the machine and check that xrdp-sesman and xrdp services is runnig (sudo lsof -i). Also look in the syslog.
sudo systemctl stop xrdp
Disable IPv6 on the machine.
sudo systemctl start xrdp-sesman
-> xrdp service will not start.

Details
In function g_tcp_bind_flags, getaddrinfo() will find two addresses. An IPv4 and an IPv6. The IPv4 will failed to bind and the IPv6 will be ok to bind. However if the IPv6 isn't available only IPv4 will be found and it will fail to bind.
Source code file: common/os_calls.c

Disable IPv6
It is possible to disable IPv6 in several ways. Here are the two ways I have used:
d6A) /etc/sysctl.conf:
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
sudo sysctl -p
d6B) grub:
Ubuntu: /etc/default/grub: GRUB_CMDLINE_LINUX_DEFAULT "ipv6.disable=1" ; sudo update-grub
CentOS: /etc/default/grub: GRUB_CMDLINE_LINUX "ipv6.disable=1" ; sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Progress
I believe both IPv4 and IPv6 should be supported, It is an automatic mapping available. The socket should be AF_INET6 and use flag AI_V4MAPPED. Need to remove flag AI_ADDRCONFIG. This work, however if the IPv6 loopback interface are disabled, it fails on the loopback port. I needed to make a special handling of this. By making a bind to "::FFFF:127.0.0.1". I had a proposal, but... Now I discover something in the latest source code (github); Fixes #432, commit 849a807 "If IPv6 not supported, fall back to IPv4". This doesn't solve the issue I have seen. After reading more, I found that this was for CentOS. I install a virtual machine and tested. Yes - difference. However this was because IPv6 was disabled in difference ways. If I disable IPv6 in the same way (via grub) in Ubuntu, I got the same behaviour. I have done a lot of testing with getaddrinfo and bind. This with small test applications. I'm not satisfied with how getaddrinfo works. For example using AI_V4MAPPED | AI_ADDRCONFIG | AI_PASSIVE and AF_UNSPEC, will return IPv4 before IPv6 net. I realise that it seems to be best to have special handling of loopback and any. This without getaddrinfo. Check the string and if loopback, try bind in6addr_loopback (AF_INET6) and if fail bind to INADDR_LOOPBACK (AF_INET). When the same for any. Check the string and if any, try bind in6addr_any (AF_INET6) and if fail bind to INADDR_ANY (AF_INET). Now we need to take care of other addresses. I believe the purpose of this is to be able to let xrdp only listen on a specific interface/network (nic). Today this is handle by inet_pton (AF_INET) and inet_pton (AF_INET6). However it cannot handle IPv6 addresses correctly. Nowadays the scope id needs to be specified in an IPv6 address, inet_pton cannot handle it (so bind will fail). Here I see little strange in the code, getaddrinfo is used to find networks and when the address is created in different ways. I realize that this functionality is what getaddrinfo is aimed to do. It can also handle scope id (interface name), added to the end of an IPv6 address with a percent sign. It is actually possible to use getaddrinfo for both IPv4 and IPv6 address strings. It works as supposed to, at least the combinations I haved tested. However I haven't found a nice way to implement it, because it is a problem with the today design. The call to socket is done in g_tcp_socket, which is called before call to g_tcp_bind and g_tcp_bind_address (which uses getaddrinfo). However getaddrinfo needs to be called first and when socket, since it's only after getaddrinfo we know for which family the socket should be created. I see no other nice option than to force a change outside of this source file. Now I realize these functions are used from many places. Okay, so here I start to think more about if it would be possible to solve without making changes outside the os_calls.c file. I come up with a proposal! So here I present a proposal that keep the interface to the os_calls intact. During first verification I found out that g_tcp_connect also needed to be modified. First I did a small change, if failed a call to getaddrinfo("::FFFF:127.0.0.1"). This was ok for d6A, however not for d6B. Here it goes strange, well I was confused and tested a lot. Let short it to my end result. We need be able to do 3 different connect calls. The connect is done non-blocking, so connect can return EINPROGRESS. It isn't specified how this function shall do, however at error it is called again and again. So it probably shouldn't wait. After an in-progress the next call to connect will be ok, this no matter of arguments to the connect call. Now I also guess that the reason for the EISCONN thing in the code, is for this second call (on some OS). At final verification it shows that Xubuntu 16.04 behave differently, so I needed to change the connect call and have all 3 alternatives in the new function.

Proposal
When bind to loopback or any, try IPv6, IPv4 and IPv4 mapped (this also apply to the connect call).
For other addresses, use getaddrinfo to create the socket address and bind to it.
API functions changed: g_tcp_bind g_tcp_bind_address g_tcp_connect
Local functions removed: address_match g_tcp_bind_flags
Local functions added: bind_loopback getaddrinfo_bind connect_loopback

Extra comments
a) The list from getaddrinfo wasn't freed.
b) In function g_tcp_socket I think it is some strange code. The IPV6_V6ONLY option is read out and only if it isn't zero, it is written. So for example if define XRDP_ENABLE_IPV6ONLY the option is only written to 1 if it is already 1. I think this isn't the wanted behavior.
c) It was a bug in g_tcp_connect that could result in infinite loop, the row "rp = h" within the for loop.
d) Please remove trailing white space in source code files.
e) g_tcp_connect is only fixed for the loopback address.
f) While do IPv6 connection from rdesktop, the local interface (nic name) needs to be given in the address, i.e. x::x:x:x:x%<interface>.

My build-A
$ sudo apt install devscripts dpkg-dev
$ sudo apt build-dep xrdp
$ apt source xrdp
$ cd
Edit os_calls.c (merged my new stuff and fix432 into the old file)
$ debuild -us -uc -b

My build-B
$ sudo apt install gcc make autoconf automake libtool pkgconf nasm git
$ sudo apt install xserver-xorg-dev libssl-dev libpam0g-dev libxfixes-dev libxrandr-dev
$ git clone --recursive https://github.com/neutrinolabs/xrdp
$ cd xrdp
$ ./bootstrap
$ ./configure --enable-ipv6
$ patch -p1 -i MyPatch.patch
$ make
$ sudo make install
$ cd ..
$ git clone --recursive https://github.com/neutrinolabs/xorgxrdp
$ cd xorgxrdp
$ ./bootstrap
$ ./configure
$ make
$ sudo make install
$ sudo ln -s /usr/local/sbin/xrdp /usr/sbin
$ sudo ln -s /usr/local/sbin/xrdp-sesman /usr/sbin
$ sudo systemctl enable xrdp
$ reboot

Test plan
Check points for each configuration:

  • Execute "sudo lsof -i" and check that ports are active (3389 and 3350).
  • Check log file (/var/log/syslog)
  • Do a rdp connection from a client computer and login.

Configurations to test:

  • Normal (default) installation of Ubuntu 16.10 (please check that you also can connect with the IPv6 address)
  • Disable IPv6 the d6A way
  • Disable IPv6 the d6B way
  • Back to normal (with IPv6). Specify an "address" in the xrdp.ini file. Use the nic's IPv4 address. Check so connect with IPv4 address works, but not with IPv6.
  • Check so connection with IPv4 address works with disabled IPv6, both d6A and d6B. After this enable IPv6.
  • Change the "address" to the nic's IPv6 address (don't forget to end with %<interface>).
  • Remove "address" value. Add a second nic (in my case, an USB connected network card). Check that it is possible to connect via both networks.
  • Specify the IPv4 address to the second nic in xrdp.ini. Check that it is possible to connect via that network, but not via the first nic's network.

Verification

  • I have tested on a Ubuntu Server 16.10 (with xfce4) machine. I installed xrdp via apt, i.e. the version via Ubuntu. Then I replaced the libcommon.so.0.0.0 file with the one I have built (see build-A). I execute according to Test plan above. All 8+ configurations worked OK. During this I had extra debug log messages in the code, to track my new stuff.
  • I also made a new installation of Xubuntu 16.04 (without xrdp). Build and install xrdp, see my build-B above. Executed the check points in the test plan. This also for d6A and d6B. Result OK
  • Fresh Xubuntu and build-B, but without "--enable-ipv6", instead "--disable-ipv6". This I tested with normal network configuration in Xubuntu, i.e. IPv6 available and also with d6A and d6B. Result OK

Disclaimer
I haven't tested other build configurations or operating systems.

Patch file
xrdp20170327proposal.patch.txt

Comments are welcome!

PS. If this is to satisfaction, I can push to git. However I'm new here on github, so please bare with me if I ask details on how I should do.

Best regards
Michael

@metalefty
Copy link
Member

I appriciate your detailed explanation and spending your time to solve the issue. I'll read it later.

Either if this is to satisfaction or not, can you try to send your patch as a pull request? We review patches on github. If you patch submitted as a pull request, we can comment on each line of diffs.

These documents will help you.
https://gist.github.com/Chaser324/ce0505fbed06b947d962
https://help.github.com/categories/collaborating-with-issues-and-pull-requests/

@Natureshadow
Copy link
Contributor

Natureshadow commented Mar 29, 2017 via email

@MichaelSweden
Copy link
Contributor Author

Thanks for links to information, I need to read!

However I'm confused about git. Things can be done in so many ways. Is it "master" or "devel" branch I should begin with?

Can I do like this:
git checkout devel
git fetch https://github.com/neutrinolabs/xrdp
git pull --rebase --autostash
git checkout -b support-disabled-ipv6
git add -u
git commit -m"Fix to handle OS disabled IPv6, issue #714.
- Changes made only in the os_calls.c file.
- Exported functions changed: g_tcp_bind g_tcp_bind_address g_tcp_connect
- Support three network configurations:
1) Normal network, with IPv6
2) Partly disabled IPv6 via sysctl.conf
3) Total disabled IPv6 via grub"
git push origin support-disabled-ipv6:support-disabled-ipv6

@speidy
Copy link
Member

speidy commented Mar 29, 2017 via email

@MichaelSweden
Copy link
Contributor Author

The "If IPv6 not supported, fall back to IPv4" Fixes #432: This only changed the socket call. The socket can fallback and open an IPv4 socket. Later on (doing connect) asking getaddrinfo for an IPv6 address, we probably will get an IPv6 address. That will fail. Maybe some OS version have given an IPv4 address back (in some configurations) and it could succeed.

For bind we asked for AF_UNSPEC, so it would work for IPv4 address. However depending on how IPv6 is disabled (and other cituations in the system) the socket can succeed to open as IPv6, but only IPv4 is available. In that case we have an IPv6 socket, but get an IPv4 address. This will fail.

This is how it is today with support for IPv6:
table

I think I don't have access to push, remote: Permission to neutrinolabs/xrdp.git denied to MichaelSweden.

PS. Here is a patch file with the extra debug text I had during part of the verification:
xrdp20170327extradebug.patch.txt

@speidy
Copy link
Member

speidy commented Mar 30, 2017

@MichaelSweden thanks for the detailed explanation and research.
You can submit your patch via Pull Request (you need to "fork" xrdp repository into your account, not directly push into neutrinolabs account)
Here is a nice tutorial: https://gist.github.com/Chaser324/ce0505fbed06b947d962

MichaelSweden added a commit to MichaelSweden/xrdp that referenced this issue Mar 30, 2017
- Changes made only in the os_calls.c file.
- Exported functions changed: g_tcp_bind g_tcp_bind_address g_tcp_connect
- Support three network configurations:
  1) Normal network, with IPv6
  2) Partly disabled IPv6 via sysctl.conf
  3) Total disabled IPv6 via grub
@MichaelSweden
Copy link
Contributor Author

Okay, now it is visible above that I managed to create a pull request. My first!

I would like to point out that my proposal keeps the libcommon interface intact. Another solution would be to change the interface and use two functions like: g_tcp_socket_bind (if argument address is null, bind to any) and g_tcp_socket_connect (if argument address is null, bind to loopback). However that will be a bigger change.

MichaelSweden added a commit to MichaelSweden/xrdp that referenced this issue Apr 22, 2017
metalefty pushed a commit that referenced this issue Apr 25, 2017
- Changes made only in the os_calls.c file.
- Exported functions changed: g_tcp_bind g_tcp_bind_address g_tcp_connect
- Support three network configurations:
  1) Normal network, with IPv6
  2) Partly disabled IPv6 via sysctl.conf
  3) Total disabled IPv6 via grub
@metalefty metalefty added the IPv6 label May 8, 2017
@metalefty
Copy link
Member

Solved by #714.

@metalefty
Copy link
Member

No, solved by #717.

@Suncatcher
Copy link

Suncatcher commented May 11, 2017

In what build this problem was fixed? I have 0.9.1-7build1 and the problem still here.
Why at all should I disable ipv6? How I can have ipv4 and ipv6 ports listened simultaneously?
I didn't find such setting at all.

@MichaelSweden
Copy link
Contributor Author

I have 0.9.1-7build1 and the problem still here.
This fix was merged after v0.9.2.

How I can have ipv4 and ipv6 ports listened simultaneously?
Normally the xrdp service will listen to both IPv4 and IPv6. If the xrdp is built with "--enable-ipv6".

@dachshund-digital
Copy link

This issue is not fixed in debian 9.1 (stretch) using the repo apt-get package for xrdp. I disabled ipv6 via the grub command line method as noted above, and xrdp will never accept connection. I enable ipv6 again, and xrdp accepts connection.

Basically what I encountered, now, is pretty much identical to the debian bug reported in the past...
Debian Bug report logs - #864230

Guys, please fix this, this is a very visible and frustrating issue to keep bouncing around.

@MichaelSweden
Copy link
Contributor Author

I would like to share some information. Sorry for this very late information. I found this out a couple of month ago.

I thought my original problem was two, xrdp and maybe some generic network problem in OS. After release of Ubuntu 17.04 I took a closer look again. One problem was that the two machines I run on before behaved different. The one with more problem, the HP, is in use and cannot be used for more testing. However I found out a way to reproduce the problem. This I think is independed of hardware. I always can reproduce it by doing this:
Ubuntu 17.04 + static IPv4 address + no network cable attached during boot

So doing this will result in broken xrdp (will not be able to start listen on socket). However I notice ssh server is working, so no major fault. The fault seems to be in xrdp only. So now I removed the normal xrdp (via apt) and instead took the source code, build and install (this include the fix I have done, i.e. this issue). Now it works!

I didn't think about, nor test, this while I did this fix. So we got lucky, this was also fixed - great!

PS. Still Ubuntu 17.04 using old version of xrdp, without this fix.

@Natureshadow
Copy link
Contributor

Natureshadow commented Sep 2, 2017 via email

@tidux
Copy link

tidux commented May 23, 2018

Attempting to bind to an IPv4 address in xrdp.ini results in no ports being bound at all and xrdp failing to start on Debian 9.4. IPv6 does work on this machine and is configured with SLAAC.

@metalefty
Copy link
Member

Raise your own issue if your issue is in a different context.

@tidux
Copy link

tidux commented Jul 26, 2018

This is the same issue. Attempting to set any listen address other than :: (global, all addresses, IPv6+4) results in an error with xrdp trying to bind to AF_INET6 :: port 0 per journalctl output, failing, and crashing.

@thedemonium
Copy link

thedemonium commented Aug 1, 2018

 journalctl -xe
xrdp-sesman[20120]: (20120)(140016739456000)[ERROR] b
xrdp-sesman[20120]: (20120)(140016739456000)[ERROR] F
systemd[1]: xrdp-sesman.service: Control process exit
xrdp-sesman[20131]: (20131)(140016739456000)[INFO ] s
xrdp-sesman[20131]: (20131)(140016739456000)[DEBUG] C
systemd[1]: xrdp-sesman.service: Failed with result '
systemd[1]: Failed to start xrdp session manager.
-- Subject: Unit xrdp-sesman.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit xrdp-sesman.service has failed.
--
-- The result is RESULT.
авг 01 20:17:28 wks-efimov systemd[1]: Dependency failed for xrdp daemon.
-- Subject: Unit xrdp.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit xrdp.service has failed.
--
-- The result is RESULT.
systemd[1]: xrdp.service: Job xrdp.service/start fail
lines 1141-1163/1163 (END)
xrdp-sesman[20120]: (20120)(140016739456000)[ERROR] bind_loopback(5, 3350) failed; IPv6 ::1 (errno=97), IPv4 127.0.0.1 (errno=98) and IPv6 ::FFFF:127.0.0.1 (errno=97).
xrdp-sesman[20120]: (20120)(140016739456000)[ERROR] Failed to start xrdp-sesman daemon, possibly address already in use.
systemd[1]: xrdp-sesman.service: Control process exited, code=exited status=1
авг 01 20:17:28 wks-efimov xrdp-sesman[20131]: (20131)(140016739456000)[INFO ] shutting down sesman 1
xrdp-sesman[20131]: (20131)(140016739456000)[DEBUG] Closed socket 7 (AF_INET 127.0.0.1:3350)
systemd[1]: xrdp-sesman.service: Failed with result 'exit-code'.
systemd[1]: Failed to start xrdp session manager.
-- Subject: Unit xrdp-sesman.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit xrdp-sesman.service has failed.
--
-- The result is RESULT.

OS: Ubuntu 18.04 bionic
Kernel: x86_64 Linux 4.15.0-29-generic
xrdp --version

xrdp: A Remote Desktop Protocol server.
Copyright (C) Jay Sorg 2004-2014
See http://www.xrdp.org for more information.
Version 0.9.5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

8 participants