Stop using qubes-rpc-multiplexer #141

DemiMarie · 2024-04-10T18:41:40Z

Instead, directly execute the command from C.

The command’s environment inherits all environment variables from the
calling process except for those that start with QREXEC. Environment
variables that start with QREXEC are not inherited, except for
QREXEC_SERVICE_PATH and QREXEC_AGENT_PID, but the following are set
explicitly:

QREXEC_SERVICE_FULL_NAME
QREXEC_REMOTE_DOMAIN
QREXEC_SERVICE_ARGUMENT if the service argument is present and non-empty.
QREXEC_AGENT_PID is set for calls to a VM that do not use MSG_JUST_EXEC.
Its handling in other cases is inconsistent, so such services should avoid it.

When the request is made to dom0, the following are also set:

QREXEC_REQUESTED_TARGET_TYPE is set to name if the requested target does not start with
QREXEC_REQUESTED_TARGET for services in dom0 if the target does not start with @.
QREXEC_REQUESTED_TARGET_KEYWORD for services in dom0 if the target does start with @.
The leading @ is removed, but subsequent @ are left unchanged.

This is a backwards-incompatible change to libqrexec. Therefore, it cannot be backported to R4.2.

Fixes: QubesOS/qubes-issues#9062

codecov · 2024-05-04T00:37:31Z

Codecov Report

Attention: Patch coverage is 70.81967% with 89 lines in your changes missing coverage. Please review.

Project coverage is 78.85%. Comparing base (b40d3da) to head (68a427f).
Report is 13 commits behind head on main.

Files with missing lines	Patch %	Lines
libqrexec/exec.c	58.09%	44 Missing ⚠️
daemon/qrexec-client.c	60.00%	16 Missing ⚠️
libqrexec/open_logger.c	66.66%	7 Missing ⚠️
libqrexec/buffer.c	0.00%	5 Missing ⚠️
agent/qrexec-agent-data.c	87.50%	3 Missing ⚠️
agent/qrexec-fork-server.c	0.00%	3 Missing ⚠️
daemon/qrexec-daemon.c	0.00%	3 Missing ⚠️
qrexec/client.py	40.00%	3 Missing ⚠️
agent/qrexec-agent.c	60.00%	2 Missing ⚠️
qrexec/tests/socket/agent.py	94.59%	2 Missing ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #141      +/-   ##
==========================================
- Coverage   79.17%   78.85%   -0.32%     
==========================================
  Files          54       55       +1     
  Lines        9953    10145     +192     
==========================================
+ Hits         7880     8000     +120     
- Misses       2073     2145      +72

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

qubesos-bot · 2024-06-28T18:00:38Z

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025020121-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2024111705-4.3&flavor=update

system_tests_qrexec
- TC_00_Qrexec_whonix-workstation-17: test_083_qrexec_service_argument_specific_implementation (error)
  subprocess.CalledProcessError: Command '/usr/lib/qubes/qrexec-clien...
system_tests_dispvm
- TC_20_DispVM_fedora-41-xfce: test_100_open_in_dispvm (failure)
  AssertionError: './open-file test.txt' failed with ./open-file test...
system_tests_kde_gui_interactive
- gui_keyboard_layout: wait_serial (wait serial expected)
  # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...
system_tests_guivm_vnc_gui_interactive
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(qvm-run --nogui -p sys-gui-vnc 'ls e1...
- gui_keyboard_layout: unnamed test (unknown)
system_tests_qrexec_perf@hw1
- TC_00_QrexecPerf_debian-12-xfce: test_110_simple_data_duplex (failure)
  AssertionError: '/usr/lib/qubes/tests/qrexec_perf.py --vm1=test-ins...
- TC_00_QrexecPerf_whonix-workstation-17: test_110_simple_data_duplex (failure)
  AssertionError: '/usr/lib/qubes/tests/qrexec_perf.py --vm1=test-ins...

Failed tests

8 failures

system_tests_qrexec
- TC_00_Qrexec_whonix-workstation-17: test_083_qrexec_service_argument_specific_implementation (error)
  subprocess.CalledProcessError: Command '/usr/lib/qubes/qrexec-clien...
system_tests_dispvm
- TC_20_DispVM_fedora-41-xfce: test_100_open_in_dispvm (failure)
  AssertionError: './open-file test.txt' failed with ./open-file test...
system_tests_kde_gui_interactive
- gui_keyboard_layout: wait_serial (wait serial expected)
  # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...
system_tests_guivm_vnc_gui_interactive
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(qvm-run --nogui -p sys-gui-vnc 'ls e1...
- gui_keyboard_layout: unnamed test (unknown)
system_tests_qrexec_perf@hw1
- TC_00_QrexecPerf_debian-12-xfce: test_110_simple_data_duplex (failure)
  AssertionError: '/usr/lib/qubes/tests/qrexec_perf.py --vm1=test-ins...
- TC_00_QrexecPerf_whonix-workstation-17: test_110_simple_data_duplex (failure)
  AssertionError: '/usr/lib/qubes/tests/qrexec_perf.py --vm1=test-ins...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/119126#dependencies

3 fixed

system_tests_extra
- TC_00_QVCTest_whonix-gateway-17: test_010_screenshare (failure)
  ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0
system_tests_basic_vm_qrexec_gui_zfs
- switch_pool: Failed (test died)
  # Test died: command 'dnf install -y ./zfs-release.rpm' failed at /...
system_tests_audio@hw1
- TC_20_AudioVM_PipeWire_whonix-workstation-17: test_260_audio_mic_enabled_switch_audiovm (failure)
  AssertionError: too short audio, expected 10s, got 0.00013605442176...

Unstable tests

marmarek · 2024-06-28T21:21:33Z

Ugh, upgrade will require a bit more care. Right now, just after upgrading dom0, qrexec calls (to dom0) fails for currently running VMs:

2024-06-28 13:11:55.919 qrexec-daemon[26518]: exec.c:85:exec_qubes_rpc_if_requested: exec qubes-rpc-multiplexer: No such file or directory

Release upgrade script will need to handle this case.

marmarek · 2024-06-28T21:24:37Z

Maybe keep the script in repo, and remove only later (R4.4?) ?
I noticed the problem in dom0, but it probably affect domUs too.

DemiMarie · 2025-01-25T23:46:08Z

Policy prompt in sys-gui doesn't work. This is what I find in the log there:

Exact same error is still there with the most recent version of this PR

I fixed the bug in #187 but git rebase added it back in the form of an unwanted commit. I fixed this by removing the unwanted commit.

marmarek · 2025-01-25T23:51:59Z

Now indeed sys-gui tests are green :)

* origin/pr/647: tests: add qrexec performance tests Pull request description: Add simple connection latency, and throughput tests. Run them with different type of services (scripts, socket, via fork-server or not). They print a test run time for comparison - the lower the better. QubesOS/qubes-issues#5740 This will be especially useful to measure impact of QubesOS/qubes-core-qrexec#141 and similar changes.

marmarek · 2025-01-26T23:39:24Z

All variables with names beginning with QREXEC are stripped from the
environment, except for QREXEC_SERVICE_PATH. This is a change in
behavior compared to the current code.

Why actually doing it? Is it intended to remove some specific variable(s)?

DemiMarie · 2025-01-28T02:27:35Z

All variables with names beginning with QREXEC are stripped from the
environment, except for QREXEC_SERVICE_PATH. This is a change in
behavior compared to the current code.

Why actually doing it? Is it intended to remove some specific variable(s)?

The intent is to ensure that with the exception of QREXEC_SERVICE_PATH, all QREXEC* variables are unset if qrexec did not set them explicitly, even if the user has them set in the environment somewhere. This is especially useful for testing, as it ensures that QREXEC* variables from the user’s environment don’t leak into the test environment.

marmarek · 2025-01-28T02:41:01Z

The way you described it may suggest various QREXEC_* variables like QREXEC_REMOTE_DOMAIN stopped being supported, which would be bad.

For testing, you don't need to modify release behavior to avoid those variables - simply exclude them in tests when calling various qrexec tools.

Anyway, I'm not strongly against this change, as it feels harmless (if not useful), but it needs better description. But also, would it be feasible to add some logging when excluding a variable that isn't going to be set a moment later (so, a variable that would be "leaked" to the service)? That would help finding unintended effects of this change.

libqrexec/exec.c

Instead, directly execute the command from C. Environment variables with names beginning with QREXEC are stripped from the environment, except for QREXEC_SERVICE_PATH and QREXEC_AGENT_PID. This stripping happens before qrexec-specific environment variables are set, so the following variables are still set as before: - QREXEC_SERVICE_FULL_NAME - QREXEC_REMOTE_DOMAIN - QREXEC_SERVICE_ARGUMENT - QREXEC_REQUESTED_TARGET_TYPE - QREXEC_REQUESTED_TARGET (dom0 only) - QREXEC_REQUESTED_TARGET_KEYWORD (dom0 only) This is a backwards-incompatible change to exec_qubes_rpc_if_requested(), which now takes an extra argument. Therefore, it cannot be backported to R4.2. It also requires changing the SELinux policy so that the labels on /etc/qubes-rpc/ and /usr/local/etc/qubes-rpc/ (and their contents) are correct. qubes-rpc-multiplexer is still present because it has legacy uses in Python code and for compatibility. Fixes: QubesOS/qubes-issues#9062

Instead, just use qrexec-client, as with any other service call.

It carries no information, and various parts of the code must strip it. Just omit it from the command entirely. Whether a command is an RPC command should be determined by the service descriptor being non-NULL. Review with "git diff --ignore-space-change".

The previous two changes were ABI breaks.

Upstream: QubesOS/qubes-core-qrexec#141

DemiMarie force-pushed the no-multiplexer branch 4 times, most recently from 189d832 to b4a2de9 Compare April 11, 2024 22:14

DemiMarie mentioned this pull request Apr 14, 2024

Support socket services with MSG_JUST_EXEC #142

Merged

DemiMarie force-pushed the no-multiplexer branch 3 times, most recently from 3de2f4b to 39f909f Compare April 16, 2024 23:58

DemiMarie marked this pull request as ready for review April 17, 2024 00:03

DemiMarie force-pushed the no-multiplexer branch from 39f909f to 2291280 Compare April 17, 2024 00:34

DemiMarie force-pushed the no-multiplexer branch 3 times, most recently from 9f69888 to a4de39f Compare May 4, 2024 00:10

DemiMarie changed the title ~~Draft: Rip out qubes-rpc-multiplexer~~ Rip out qubes-rpc-multiplexer May 4, 2024

DemiMarie force-pushed the no-multiplexer branch from a4de39f to b7502bc Compare May 9, 2024 23:29

marmarek mentioned this pull request May 18, 2024

Documentation #148

Merged

DemiMarie force-pushed the no-multiplexer branch from b7502bc to b4d7ed6 Compare June 27, 2024 15:49

marmarek added openqa-pending and removed openqa-pending labels Jun 28, 2024

marmarek mentioned this pull request Jun 28, 2024

Prepare R4.2 -> R4.3 upgrade tool QubesOS/qubes-issues#9317

Open

7 tasks

marmarek removed the openqa-pending label Jun 28, 2024

DemiMarie force-pushed the no-multiplexer branch from 51460e6 to 62f2de4 Compare January 24, 2025 20:58

marmarek added openqa-pending and removed openqa-pending labels Jan 25, 2025

This was referenced Jan 25, 2025

ZFS 2.3.0 packaging QubesOS/qubes-zfs-dkms#1

Merged

Pull in required Pipewire dependencies for Archlinux QubesOS/qubes-gui-agent-linux#224

Merged

Add initial packaging QubesOS/qubes-zfs#1

Merged

DemiMarie force-pushed the no-multiplexer branch 3 times, most recently from 17be9de to 7316fdc Compare January 29, 2025 01:07

marmarek added the openqa-pending label Jan 29, 2025

DemiMarie force-pushed the no-multiplexer branch from 7316fdc to 72689b1 Compare January 29, 2025 03:28

qubesos-bot mentioned this pull request Jan 29, 2025

tests: add simple storage performance tests QubesOS/qubes-core-admin#649

Merged

DemiMarie force-pushed the no-multiplexer branch from 72689b1 to b760f57 Compare January 31, 2025 15:48

marmarek reviewed Jan 31, 2025

View reviewed changes

libqrexec/exec.c Outdated Show resolved Hide resolved

DemiMarie added 4 commits January 31, 2025 18:54

Avoid qubes-rpc-multiplexer for dom0 -> dom0 calls

1adc290

Instead, just use qrexec-client, as with any other service call.

libqrexec SONAME bump

68a427f

The previous two changes were ABI breaks.

DemiMarie force-pushed the no-multiplexer branch from b760f57 to 68a427f Compare February 1, 2025 00:00

DemiMarie requested a review from marmarek February 1, 2025 00:32

marmarek approved these changes Feb 4, 2025

View reviewed changes

marmarek merged commit 68a427f into QubesOS:main Feb 4, 2025
3 of 5 checks passed

DemiMarie deleted the no-multiplexer branch February 4, 2025 02:17

CertainLach added a commit to CertainLach/nixos-qubes that referenced this pull request Feb 15, 2025

qubes-core-qrexec: self-rpc-multiplexer dependency was removed upstream

51ae978

Upstream: QubesOS/qubes-core-qrexec#141

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stop using qubes-rpc-multiplexer #141

Stop using qubes-rpc-multiplexer #141

DemiMarie commented Apr 10, 2024 •

edited

Loading

codecov bot commented May 4, 2024 •

edited

Loading

qubesos-bot commented Jun 28, 2024 •

edited

Loading

marmarek commented Jun 28, 2024

marmarek commented Jun 28, 2024

DemiMarie commented Jan 25, 2025

marmarek commented Jan 25, 2025

marmarek commented Jan 26, 2025

DemiMarie commented Jan 28, 2025

marmarek commented Jan 28, 2025

Stop using qubes-rpc-multiplexer #141

Stop using qubes-rpc-multiplexer #141

Conversation

DemiMarie commented Apr 10, 2024 • edited Loading

codecov bot commented May 4, 2024 • edited Loading

Codecov Report

qubesos-bot commented Jun 28, 2024 • edited Loading

OpenQA test summary

New failures, excluding unstable

Failed tests

Fixed failures

Unstable tests

marmarek commented Jun 28, 2024

marmarek commented Jun 28, 2024

DemiMarie commented Jan 25, 2025

marmarek commented Jan 25, 2025

marmarek commented Jan 26, 2025

DemiMarie commented Jan 28, 2025

marmarek commented Jan 28, 2025

DemiMarie commented Apr 10, 2024 •

edited

Loading

codecov bot commented May 4, 2024 •

edited

Loading

qubesos-bot commented Jun 28, 2024 •

edited

Loading