In the last article of this series
KVM/Qemu VMs with a multi-screen Spice console – II – local access with remote-viewer via a network port
KVM/Qemu VMs with a multi-screen Spice console – I – Overview over local and remote access methods
I discussed “remote-viewer” – a tool to access the graphical Spice console of a virtual machine [VM] on a KVM/Qemu host. Although remote-viewer is thought to be used over network connections, you can also use it locally on the virtualization host itself, e.g. in a desktop virtualization scenario.
To get an overview we first had a brief look at some central libvirt and qemu configuration files which control the interaction of Spice clients with the libvirtd daemonand/or the Qemu-emulator. We then defined a network port (e.g. 20001) in the Spice related entry of the VM’s XML configuration file. This file is evaluated when you start a Qemu VM with virsh or virt-manager. Afterward we could access the Spice console of the VM by entering
spice://localhost:20001
on a terminal in a graphical desktop session on the KVM host. If the host had an IP like 192.168.2.22
spice://192.168.2.22:20001
would in principle work, too. But a successful outcome may, of course, depend on local firewall settings for your network devices.
We verified that remote-viewer supports a multi-screen presentation of any major graphical Linux desktop started on the VM. Present versions of KDE and Gnome on a Kali/Debian/Opensuse guest automatically adapt to changes of the user’s Spice client windows (on the host’s desktop). XFCE, however, requires a manual configuration of the virtual screens with tools of the guest OS.
I also demonstrated that a Spice console session corresponds to a fragile “one seat” situation: Other local or remote users can kick us out of our Spice session at any time. We could at least protect us against unauthorized session switches by defining a password for the Spice console. So far, so good.
But: Local access via a network port is certainly not the fastest method for local user interaction with a VM: Each transaction triggers operations of a TCP network stack. Can we get around this?
Yes, we can – by working with a Unix socket on the VM host. This is the subject of this article. While we prepare a Spice socket configuration for remote-viewer we also answer the question which sockets the libvirtd daemon offers for its client.
We use our test VM “debianx” again, which has a Kali OS installed together with multiple desktops to choose from (Gnome, KDE, XFCE). The KVM/Qemu host is a Opensuse Leap 15.2 system with a KDE desktop.
Documentation? Not really …
I did not find any hints in the Opensuse documentation on virtualization of how to use remote-viewer with a Unix socket. Neither the man pages, nor other user-friendly descriptions of remote-viewer gave me a clue. I got a socket based approach to work after an accidental view into two discussions on the Internet plus a bit of trial and error. These are links to the discussions I referred to:
https://www.linuxquestions.org/questions/linux-virtualization-and-cloud-90/remote-viewer-spice-qemu-and-unix-domain-socket-how-to-connect-4175571914/
https://bugzilla.redhat.com/show_bug.cgi?id=1335832
You may find a description of socket based configurations for Spice also in articles on Virgl3D based rendering – a local socket based configuration is essential there.
Spice access via a socket
The first article of this series contains a schematic drawing which shows various access methods to the Spice console of a VM. However, aside TCP ports I had not displayed any Unix sockets there. The following picture gives you an idea how such a local alternative could look like:
You should not take the direct connections to the Spice console too literally. Of course, the real interaction occurs between remote-viewer and the Qemu-emulator.
In the drawing I also indicated that libvirt-based tools (as virt-manager, virt-viewer), which do not directly interact with the qemu-emulator, use a specific socket which is automatically created at a standard location on Opensuse Leap 15.x systems.
Off topic: What sockets are used by virt-manager and virt-viewer?
Actually, libvirtd exposes multiple sockets to libvirt clients – each for a specific purpose; see
https://libvirt.org/manpages/libvirtd.html
and
https://libvirt.org/daemons.html
for details.
Virt-manager has its own graphical console window. Select a running VM and simply choose the menu-point “Open” to open a Spice client window on your host.
Note: In contrast to remote-viewer and also virt-viewer, virt-manager does not allow you to open multiple “screen”windows.
In a terminal window we ask “netstat” to inform us about relevant active sockets; just for interest we also look into related directories:
MySRV:~ # netstat | grep libv unix 3 [ ] STREAM CONNECTED 554391 /run/libvirt/libvirt-sock unix 3 [ ] STREAM CONNECTED 513499 /run/libvirt/libvirt-sock unix 3 [ ] STREAM CONNECTED 554599 /run/libvirt/libvirt-sock unix 3 [ ] STREAM CONNECTED 515826 /var/lib/libvirt/qemu/domain-1-debianx/monitor.sock unix 3 [ ] STREAM CONNECTED 513581 /run/libvirt/libvirt-sock MySRV:~ # MySRV:~ # la /run/libvirt/ | grep sock srw------- 1 root root 0 Mar 9 17:48 libvirt-admin-sock srw-rw-rw- 1 root root 0 Mar 9 17:48 libvirt-sock srw-rw-rw- 1 root root 0 Mar 9 17:48 libvirt-sock-ro srw------- 1 root root 0 Mar 9 17:48 virtlockd-sock srw------- 1 root root 0 Mar 9 17:48 virtlogd-admin-sock srw------- 1 root root 0 Mar 9 17:48 virtlogd-sock MySRV:~ # MySRV:~ # la /var/lib/libvirt/qemu/ total 44 drwxr-x--- 10 qemu qemu 4096 Mar 9 17:48 . drwxr-xr-x 11 root root 4096 Oct 26 11:01 .. drwxr-xr-x 3 qemu qemu 4096 Apr 2 2017 channel drwxr-xr-x 2 qemu qemu 4096 Nov 16 17:56 checkpoint drwxr-x--- 2 qemu qemu 4096 Mar 9 17:48 domain-1-debianx drwxr-xr-x 2 qemu qemu 4096 Apr 2 2017 dump drwxr-xr-x 2 qemu qemu 4096 Apr 2 2017 nvram drwxr-xr-x 3 qemu qemu 4096 Sep 10 2018 ram drwxr-xr-x 2 qemu qemu 4096 Apr 2 2017 save drwxr-xr-x 2 qemu qemu 4096 Apr 2 2017 snapshot MySRV:~ # MySRV:~ # la /var/lib/libvirt/qemu/domain-1-debianx/ total 20 drwxr-x--- 2 qemu qemu 4096 Mar 9 17:48 . ndrwxr-x--- 10 qemu qemu 4096 Mar 9 17:48 .. -rw------- 1 qemu qemu 32 Mar 9 17:48 master-key.aes srwxrwxr-x 1 root root 0 Mar 9 17:48 monitor.sock MySRV:~ #
By the way: A nice tool to list sockets is “ss“; try “ss -axeo | grep libv” or
ss -a –unix -p | grep virt
for instance.
Some sockets obviously have very restrictive rights settings. However, the rights situation seems to be totally relaxed for two of the sockets in “/run/libvirt”. As we shall see in a minute these sockets are the relevant ones for Spice clients using libvirt.
Readers of the last article may have guessed that we configure access rights to these sockets in the file “/etc/libvirt/libvirtd.conf”. Sorry, but on a standard Opensuse Leap system this is wrong. Instead the sockets are created by systemd during the start of the libvirtd.service. See the respective files libvirtd.service, libvirtd.socket, libvirtd-ro.socket in the folder “/usr/lib/systemd/system”:
MySRV:/usr/lib/systemd/system # cat libvirtd.service [Unit] Description=Virtualization daemon Requires=virtlogd.socket Requires=virtlockd.socket # Use Wants instead of Requires so that users # can disable these three .socket units to revert # to a traditional non-activation deployment setup Wants=libvirtd.socket Wants=libvirtd-ro.socket Wants=libvirtd-admin.socket Wants=systemd-machined.service .....
and e.g.
MySRV:/usr/lib/systemd/system # cat libvirtd-ro.socket [Unit] Description=Libvirt local read-only socket Before=libvirtd.service BindsTo=libvirtd.socket After=libvirtd.socket [Socket] # The directory must match the /etc/libvirt/libvirtd.conf unix_sock_dir setting # when using systemd version < 227 ListenStream=/run/libvirt/libvirt-sock-ro Service=libvirtd.service SocketMode=0666 [Install] WantedBy=sockets.target
A standard rights setting of “666” is used. You could change this in a special local file under “/etc/systemd/”. We shall return to respective settings in another article.
Virsh and virt-manager require a (rw) socket which allows for VM re-configuration, i.e. writing operations to the Qemu-emulator configuration data for a VM. This socket is “/run/libvirt/libvirt-sock“.
Regarding access to a Spice console, however, a so called “ro”-socket, namely “/run/libvirt/libvirt-sock-ro“, is sufficient. You can verify this by trying
virt-viewer -c qemu:///system?socket=/run/libvirt/libvirt-sock-ro
The “ro” refers to a denial of configuration writing and changes of Qemu-process parameters controlling the VM. It is NOT directly related to the Linux access rights of the socket itself. Actually, the socket is a “stream” socket; this already implies that the user needs write access to it. Which is given in an Opensuse Leap system. See the generous “rw-rw-rw-“-setting in the listings above!
This raises the question:
What restricts the access to these libvirt standard sockets at all on an Opensuse Leap system?
Answer:
Its a policy setting; see the Opensuse documentation in “Virtualization guide – Connecting and authorizing“. More on this topic in another article.
How would we specify an URI, which points to a Unix-socket, for remote-viewer?
To force remote-viewer to use a Unix socket we first have to find out how we specify an URI pointing to such a socket. Well, the first of the named articles above together with the man-pages for remote-viewer gives us an idea. The right form for a local access is:
remote-viewer -v spice+unix://Path-To-Socket
Note that a path down from the root ”
/” of the Linux directory tree will lead to three “///” !
You cannot use the libvirtd-sockets with remote-viewer!
Can we use the sockets provided by libvirtd with remote-viewer? The answer is: No. More precisely: I do not know how, if it should work against my expectations. We use the above recipe and try:
MySRV:~ # remote-viewer -v spice+unix:///var/lib/libvirt/qemu/domain-1-debianx/monitor.sock Guest (null) has a spice display Opening connection to display at spice+unix:///var/lib/libvirt/qemu/domain-1-debianx/monitor.sock
With the result:
Something equally negative happens with
MySRV:~ # remote-viewer -v spice+unix:///run/libvirt/libvirt-sock Guest (null) has a spice display Opening connection to display at spice+unix:///run/libvirt/libvirt-sock (remote-viewer:22055): GSpice-WARNING **: 19:29:01.691: incomplete link header (-104/16)
We are stuck. Well, it would have been too simple. And it would have been strange, too, because remote-viewer was made to directly access Qemu. So, the next logical question is:
How can we tell Qemu to provide a suitable Unix socket for the Spice console?
Required settings in the XML domain file for the VM => specify a socket for the Spice item
We can use the information provided in the Red Hat bug tracker mentioned above. We guess that we can change the Spice settings according to the following pattern in the VMs’ domain definition file “/etc/libvirt/qemu/debianx.xml”:
<graphics type='spice' autoport='no' keymap='de' defaultMode='insecure'>
<listen type='socket' socket='/tmp/spice.socket'/>
<image compression='off'/>
<gl enable='no'/>
</graphics>
You see that I choose the “/tmp”-directory to create a socket. The directory must of course be writeable. For whom? And which access rights will our aspired socket get?
It is reasonable to assume that we will see the rights of the specific system user, which is used to run Qemu processes for VMs. We already saw in the last article that this is the user named “qemu” (single member of group “qemu”) on a standard Opensuse system.
OK, let us see whether we can start our “debianx” test VM with this new Spice item in its XML configuration.
No problem! To check whether the VM really is operational we start virt-manager’s integrated Spice console window and log into Kali’s KDE desktop:
No problem there either.
Note the settings for the Spice window – the last menu point must be checked to guarantee an automatic adaption of the VM’s desktop to the dimensions of the Spice console window.
If you now would use
virt-viewer -c qemu:///system?socket=/run/libvirt/libvirt-sock-ro
again, virt-manager’s Spice window would be closed (one seat!) and a virt-viewer window would open (NOT a remote-viewer window; see the top bar of the window below).
But all of this is no proof that a new socket has been created. virt-manager and virt-viewer use the sockets “/run/libvirt/libvirt-sock” and “/run/libvirt/libvirt-sock-ro” (to connect to libvirtd) – as we already know. But a look into the “/tmp”-folder
MySRV:~ # ls -lisa /tmp | grep spice 1452545 0 srwxrwxr-x 1 qemu qemu 0 Mar 13 14:31 spice.socket MySRV:~ #
actually shows that we indeed have a new socket. We get an additional confirmation by netstat:
MySRV:~ # netstat -ax | grep spice unix 2 [ ACC ] STREAM LISTENING 137989 /tmp/spice.socket MySRV:~ #
Remote-viewer with a local Unix socket
Let us use the prepared socket as user “uvma”. Unfortunately, a first trial with
remote-viewer spice+unix:///tmp/spice.socket
fails due to insufficient access rights. Obviously, the user “uvma” needs write rights on the specified “stream” socket.
The most simple solution for the moment is to add the user “uvma”, which we already put into a privileged group “libvirt” in the last article, to the group “qemu”, too. (This is probably not the best solution. We come back to security aspects later.) But for the time being we enter (as root)
usermod -a -G qemu uvma
As user “uvma” we afterward log out and in again to our desktop session on the host. In a terminal window we then enter (as user “uvma”)
remote-viewer spice+unix:///tmp/spice.socket
It works! A new Spice client window for remote-viewer is opened. I added one more “screen” and also started some applications to see, if we can really interact with the VM – which is indeed the case. Even sound works (if pulseaudio is used on the host):
Great!
Security considerations
Besides an improved performance, working with a Unix socket avoids opening a network port and related potential security issues. But, as we saw, we need some special rights for such a scenario. Which also has security implications ….
Regarding this point I do not think that putting a user into the “qemu”-group is a good idea. Well, you could argue that our user “uvma” is already privileged regarding VMs. Yes, true enough. But we may change this in the future and separate users which can use virt-manager from others which use remote-viewer for a specific VM: The user who gets the right to open the Spice console may be different from the special user which is allowed to start the VM.
Another problem is that a user being member of the qemu group has almost the same rights as the “qemu”-user itself. But, in contrast to the special user “qemu” on an Opensuse Leap system, a normal user as “uvma” has a login-shell:
MySRV:/etc/libvirt/qemu # cat /etc/passwd | grep qemu qemu:x:484:483:qemu user::/sbin/nologin MySRV:/etc/libvirt/qemu # cat /etc/passwd | grep uvma uvma:x:1025:100:uvma:/home/uvma:/bin/bash
“uvma” needs a login-shell as
he/she shall work with remote-viewer in his/her desktop session on the host. So, if this user gets hacked (e.g. via a browser bug) this would have consequences for all VMs on the host.
We can at least confine a selected user’s impact to a specific VM. Let us take a user “uvmb”. We want to allow this user to access our test-VM “debianx” locally via a socket – but only this VM!
For this purpose we create a special group “spicex” and add both “qemu”, “uvma” and “uvmb” to this group.
usermod -a -G spicex uvmb usermod -a -G spicex uvma gpasswd -d uvma qemu
We also remove user “uvma” from the “qemu”-group.
Then we create a special directory “/var/spice/spicex“. You can, of course, choose a different path. We set the group of this folder to “spicex”. Then we set the “s”-flag on the group and use ACLs to enforce a default group for the user and the group with certain rights on an files created in this folder.
chown qemu.spicex /var/spice/spicex chmod g+s /var/spice/spicex setfacl -dm u:qemu:rwx,g:spicex:rwx,other::--- /var/spice/spicex
In addition we stop our VM and change its XML configuration to
<graphics type='spice' autoport='no' keymap='de' defaultMode='insecure'>
<listen type='socket' socket='/var/spice/spicex/spice.socket'/>
<image compression='off'/>
<gl enable='no'/>
</graphics>
Afterward we restart the libvirtd-daemon. That’s it. Te next time, you start your VM by virt-manager as “uvma” you get something like
MySRV:/var/spice/spicex # la total 12 drwxrws---+ 2 qemu spicex 4096 Mar 17 19:26 . drwxr-xr-x 9 root root 4096 Mar 15 16:32 .. srwxrwx---+ 1 qemu spicex 0 Mar 17 19:26 spice.socket
Now, you change to user “uvmb” by logging out and starting a new X-/KDE-session on the host or by just entering “su – uvmb” in a terminal. We login as “uvmb” and try
uvmb@MySRV:~> remote-viewer -v spice+unix:///var/spicex/spice.socket
This should work; if you stayed in the KDE session of user “uvma” you probably get some error messages regarding audio and a denied pulseaudio access. But the Spice graphics should be OK. Similar precaution measures should be taken for other VMs. ACLs give you enough flexibility to control VM-specific access. Now, you could remove “uvma” from group spicex – and get some kind of tool access segregation. It would not make much sense however as long as uvma still is privileged by being a member of the libvirt group; uvma still can open the Spice console at any time via libvirtd and its special sockets. Regard “uvma” as a VM administrator.
Summary:
The configuration of a VM with a Unix socket has the advantage that we can control the Spice session access, and thereby a potential session switch to another user, a bit better. We restrict the access to libvirtd to selected members of an administrative group and the access to the socket to a selected user of the VM. Though it does not hurt to have a password in place in addition …
Off topic hint 1: Adjustment of Spice screen positions relative to each other on the guest system
When you open a second Spice windows with remote-viewer it is not guaranteed that the guest’s desktop system initially places these virtual Spice screens seamlessly side-a-side. But both KDE, Gnome and XFCE provide graphical tools to adjust the geometrical relation of multiple screens. With the help of these tools you can specify how relative corner positions of the virtual screens are handled. You only have to do this once, afterwards you can move your Spice windows around on the host’s desktop as you like :
Relative positioning of two virtual Spice screens with the monitor tool of the guest’s KDE desktop
Independent positions of the Spice windows on the KDE desktop of the KVM/Qemu host
Even resizing of the Spice windows won’t disturb the relative position definition of the virtual screen corners in the guest.
Off topic hint 2: Workaround for faulty Clipboard interaction between the KDE sessions on the host and in the Qemu guest VM
I noticed a strange dis-coupling between the functionality of the clipboard of KDE main session on the host and the clipboard of guest VM’s KDE session. Due to the spice-vdagent they should work seamlessly together. It looked OK in the beginning for the exchange of data between the host and the VM. However, when I tried to copy between two terminal windows on the host I got error message in the terminal from which I started the guest VM:
Spice-CRITICAL **: 14:46:03.424: clipboard_request: assertion 's->clipboard_by_guest[selection] == FALSE' failed
And no data were copied! This disappeared as soon as I activated the “system” tray in the control bar of the guest’s KDE desktop, activated the clipboard entry there and adjusted the clipboard settings there to those of the KDE session on the host.
Conclusion
Remote-viewer works very well with a local Unix socket instead of a TCP socket as the interface to the Qemu-emulator process of the VM. A socket based local access scenario to a Spice console can easily be configured via respective settings in the XML-definition file of a VM. However, to improve security you should avoid adding a standard user who wants to use remote-viewer to the “qemu” group. Involving ACL rights will help you to confine users to access the Spice console of specific VMs via placing a socket into a specific directory and controlling access to it and its contents.
Note by the way that using Unix sockets locally on the host also allows for a new remote-access scenario via SSH. Such a scenario could even be more efficient than a TLS encrypted standard connection over a network port in a LAN. Much to discover! Stay tuned. In the next article
I will show you how to use remote-viewer via SSH.
Links
The ss-command to list up sockets
https://www.networkworld.com/article/3327557/using-the-linux-ss-command-to-examine-network-and-socket-connections.html
lbvirt and authentication on Opensuse
https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha-libvirt-connect.html