KVM/Qemu VMs with a multi-screen Spice console – VII – remote-viewer, qemu and SASL authentication

I continue with my exploration of remote access methods to the Spice console of a VM run on a KVM/Qemu-server with Opensuse Leap 15.2 as OS.

KVM/Qemu VMs with a multi-screen Spice console – VI – remote access with remote-viewer and TLS encryption
KVM/Qemu VMs with a multi-screen Spice console – V – remote access via remote-viewer, a network port and a SSH-tunnel
KVM/Qemu VMs with a multi-screen Spice console – IV – remote access via SSH, remote-viewer and a Unix socket
KVM/Qemu VMs with a multi-screen Spice console – III – local access with remote-viewer via a Unix socket
KVM/Qemu VMs with a multi-screen Spice console – II – local access with remote-viewer via a network port
KVM/Qemu VMs with a multi-screen Spice console – I – Overview over local and remote access methods

The topic of my last post was the combination of remote-viewer with "TLS" encryption. We configured libvirt such that virt-manager or virsh started the qemu-process for our test-VM with support for TLS/SSL. But: Remote-viewer itself does not use any libvirt-layer; regarding connections to a VM-specific network port, it is the qemu-emulator-process of the VM which does the encryption business for us on the server's side - not libvirt.

In comparison to SSH-based methods discussed earlier a major disadvantage of the TLS-solution was the lack of any authentication on the server. The TLS-port for Spice on the server was open for anybody in our Intranet - not a nice situation for a "one seat" tool as Spice. We could, of course, have restricted access to certain client-systems by a firewall and by setting a general or a VM-specific password for the Spice console(s); but still there would not be any user-specific restrictions in place.

As a first improvement we would like to establish some server-side authentication as a condition for access to a Spice console. With SSH this was a peace of cake - and we could in addition set user-specific options for the SSH daemon to combine authentication with a user-specific access to the Spice console of a selected VM, only. Something similar is much harder to achieve with TLS - as long as we cling to libvirt-tools for starting VMs and remote-viewer as a client. I have not succeeded with it by simple means. Actually, I will need another article to describe some measures for real user specific restrictions - on the remote-client and, unfortunately, without reaching the same clear association of a user's authentication on the server with the access to a specific VM. But one step after the other.

In this article I am going to discuss the combination of

remote-viewer, Qemu, TLS and SASL.

We first take a look at a most simple SASL authentication mechanisms based on a sasldb2-file which contains the credentials. One could also directly involve a Kerberos system by just changing some settings in prepared configuration files. But Kerberos is beyond the scope of this series. Instead I want to show you how to configure the invocation of the saslauthd daemon as a general bridge to whatever authentication method you prefer in your network or on the server for special services. For reasons of simplicity we pick the local PAM machinery on the server as an example, but MySQL, LDAP or LDAP combined with Kerberos would be other options.

We are going to see that these two simple approaches to SASL based authentication - though working - come with a big disadvantage regarding Spice: The authentication is unfortunately global for all VMs. We won't get a restriction to the Spice console of a specific VM with remote-viewer and SASL. And we do not really restrict the access to the console of a specific VM to a defined UID on the server.

But, let us first focus on the problem of getting SASL active in combination with remote-viewer. We work again with our test-VM "debianx" with a Kali system on it, a laptop MyLAP as a remote client-system with OS Leap 15.2 and a KVM/Qemu server, also with Opensuse Leap 15.2 as OS (see the previous articles of this series).

Schematic overview

The following drawing gives you an overview about the involved components on the KVM/Qemu server and on the remote client-system.

This is more or less the graphics from my last article with additional elements. I have indicated that both remote-viewer and the Qemu-hypervisor must be configured to use SASL in addition to OpenSSL. This in turn means that these components must support SASL and its challenge/response mechanisms. On the server-side we need a SASL compatible backend for keeping credential data and performing authentication against them. Some Qemu-configuration options allow for a direct and seamless access of certain backends - as sasldb2 or Kerberos. However, the "saslauthd" daemon with its manifold of connectors to different internal or external backends can be used as a mediator, too.

Note that we would create severe security holes if SASL related credentials or other authentication data were transferred between the client-system and the server without encryption. SASL is basically independent of OpenSSL-libraries. It provides its own methods ("mechanisms") for data exchange - some with encryption, but some without. And saslauthd works with clear text data, only.

Therefore, any client-application using SASL is well advised to check that a TLS-tunnel really is in place if the used or available SASL mechanisms imply plain text transfer of challenge/response data. It is the sole responsibility of the applications to evaluate the circumstances. And remote-viewer indeed seems to take care ....

TLS was not really enforced by our settings ... so far

I am lazy and I use libvirt tools to start qemu-based VMs. The TLS configuration is, therefore, in a way a global one for all VMs. It activates (potential) TLS support for the Spice console whether the VM in the end really offers a special TLS-port or not. In this case the global configuration has no consequences in the sense of TLS enforcement; we must still define a VM-specific TLS-port to really enable TLS for a chosen VM. And as long as a standard network port is defined for the VM and opened on the KVM/Qemu-server the usage of TLS is not enforced. A good reason for remote-viewer to be careful with regard to SASL.

SASL RPMs

We have to install packages for SASL on the server and also on the client. Opensuse provides SASL in form of "cyrus-sasl". Required packages for our tests are:
cyrus-sasl, cyrus-sasl-digestmd5, cyrus-sasl-plain, cyrus-sasl-saslauthd, cyrus-sasl-scram, libgsasl-lang, libgsasl7, libsasl2-3, libsas2-3-32bit, perl-Authen-SASL.

Some of these packages provide specific SASL "mechanisms" for the exchange of challenge/response data.

Basic preparations for Qemu and SASL on the KVM/Qemu-server

In previous articles we learned that remote-viewer accesses the Spice console of a VM directly via Qemu (and not libvirt). So we must prepare the start of a qemu-process for our VM such that parameters enable the use of SASL. As with TLS we use the file "/etc/libvirt/qemu.conf" for the configuration. For our present purposes we activate the following options:

spice_tls = 1
...
spice_tls_x509_cert_dir = "/etc/pki/libvirt-spice"
...
spice_sasl = 1
...

You are already familiar with the TLS-related options. The only new point is the activation of SASL as the authentication mechanism for the Spice console. Folks who start qemu processes manually or with the help of scripts should read
https://www.spice-space.org/spice-user-manual.html
for the relevant options to add to the "/usr/bin/qemu-system-x86_64"-command. Spice related parameters may look similar to what I get in may case, namely

/usr/bin/qemu-system-x86_64 ..... -spice tls-port=20002,addr=0.0.0.0,sasl,disable-ticketing,x509-dir=/etc/pki/libvirt-spice,tls-channel=default,image-compression=auto_glz,seamless-migration=on ......

Note the location of the x509-directory; we got already acquainted with it it in the last article: It is used by qemu to read in TLS/SSL-certificates and key information which we must place there in advance, e.g for the CA and the server. If we started VMs directly with "qemu"-comands on a CLI we could change this directory individually for VMs. Using virt-manager or virsh forces us to define one x509 directory for all VMs, instead. Keep this in mind.

The commenting text in the file "/etc/libvirt/qemu.conf" tells you already that additional configuration steps are required in a file just below "/etc/sasl2/". This directory gathers SASL configuration files for various services that make use of SASL.

On a Leap 15.2 system no special default configuration file for "qemu" is provided at "/etc/sasl2/", but for "libvirt". For our present purposes we just copy the existing file "/etc/sasl2/libvirt.conf" to a new file "/etc/sasl2/qemu.conf". Why "qemu.conf"? Well, this is the instruction given in Opensuse's virtualization documentation - and it seems completely logical. This may lead you to the assumption that the "service" in question always has to be qualified as "qemu" in relation to SASL. We shall later see that this assumption is wrong when we invoke the "saslauthd"-daemon ...

For our first tests we activate the following two options, only, in the file "/etc/sasl2/qemu.conf" by un-commenting respective lines, out-commenting others and a minimum of editing:

mech_list:  plain scram-sha-256 
# mech_list:  plain scram-sha-256 digest-md5
..
sasldb_path: /var/spice/sasl/passwd.db
...

With the "mech_list" we obviously define a list of SASL mechanisms. We shall see how we specify which of these mechanism remote-viewer should use in a minute.

The mechanisms "plain" and "scram-sha-256" lead to an exchange of (salted) challenge/response data in clear text. They offer no real security for man-in-the-middle attacks in our Intranet. So, these mechanisms must be used in combination with a TLS tunnel, only. For possible attack-vectors see e.g.
https://tools.ietf.org/html/rfc7677
and
https://tools.ietf.org/html/rfc5802

The good message is: Remote-viewer will block the access to the Spice console of a VM if the connection is not secured by TLS/SSL!

Note the out-commented line with a crypto-mechanism specified in addition, namely "digest-md5". We shall use this line for tests of situations where a standard connection without TLS is established or might be established as a fallback. We shall see that remote-viewer will pick "digest-md5" automatically if TLS is not guaranteed and if the mechanism is available.

However, "digest-md5" must itself be regarded as insecure due to its weak cryptography. In a productive environment you should simply forget it - and best eliminate it from your configuration! Let remote-viewer check for sufficient TLS conditions - and let it refuse access to the target VM if necessary.

The parameter sasldb_path specifies the path to sasldb2-files which should contain the credentials. In my settings the reader recognizes the directory below which I placed a folder for a pure Unix socket in local access scenarios (the previous articles). You may, however, specify whatever directory you like on your system. But be aware of required access rights; see below.

How to fill sasldb2 with credentials? What access rights are required?

Some basic information about the usage of sasldb2-files is given e.g. at
https://blog.sys4.de/cyrus-sasl-sasldb-konfigurieren-de.html.

According to the test settings above we place our sasldb-files into a folder "/var/spice/sasl/" (see below). Before you blindly experiment with adding users, consider some hints regarding the entries:

SASL distinguishes between users of the same name for different systems or application environments. Users have to be qualified with a user-name and a realm in the form

username@realm

Regarding the "user":
It does not matter what usernames we choose; we can use fictitious users - they need not exist as valid users on the KVM/Qemu server (or the client-system) at all. However, as remote-viewer normally "suggests" the present username in a dialog for authentication (see below) it may be clever to add users with these suggested names - just to save some time with typing. In my test environment I work as user "myself" on the client "MyLAP", but sometimes also as user "uvmb" on the server "MySRV". So, I should have valid entries for both users.

Regarding the "realm":
For a simple sasldb you can basically define as a realm whatever you want - but the user who wants to authenticate has to know what to provide. So he/she should be able to associate it with some meaning, e.g. the VM's name. The "realm" is, unfortunately, NOT set or defined automatically later by the remote-viewer client and is in no way related to the contents of TLS certificates or target domain names presented during the authentication process. It can be freely set by the user in an SASL authentication dialog presented by remote-viewer: The user there can fill in anything after "username@...". To become authenticated, however, what he/she fills in must match an entry in the sasldb-file.

Thus, we could define a fantasy user "tom@waits" with a password "the-piano-has-been-drinking"; it would work - as long as the remote-viewer user has all this information and provides it properly in the authentication dialog.

There is, however, one more rule to keep in mind during your experiments:

If the user provides just a username without any realm in the SASL authentication dialog then the hostname will be added automatically on the server-side. And the hostname is the one in "/etc/hostname" on the server.

(The qemu-end of the SASL authentication does not seem to differentiate in cases where we use multiple IPs and different FQDN entries at a DNS server or in /etc/hosts for the host.)

In my test case the hostname is "myserv.anraconc.de". But be careful: You may have the short-name ("mysrv") standing in "/etc/hostname" on your Opensuse Leap 15.2 system! You should have a proper entry "myself@hostname" in your sasldb. If in doubt enter entries with both the short form and the FQDN of your KVM-host.

As root we now prepare a folder "/var/spice/sasl/" on the server (we have a folder "/var/spice/" already from previous efforts) and add some users with the help of the "saslpasswd2"-command :

mysrv:/var/spice # mkdir /var/spice/sasl
mysrv:/var/spice # chown root.qemu /var/spice/sasl
mysrv:/var/spice # chmod 750 /var/spice/sasl
mysrv:/var/spice # saslpasswd2  -a qemu myself@mysrv.anraconc.de -f /var/spice/sasl/passwd.db
Password: 
Again (for verification): 
mysrv:/var/spice # saslpasswd2  -a qemu myself -f /var/spice/sasl/passwd.db
...
mysrv:/var/spice # saslpasswd2  -a qemu uvmb@mysrv.anraconc.de -f /var/spice/sasl/passwd.db
...
mysrv:/var/spice # saslpasswd2  -a qemu uvmb -f /var/spice/sasl/passwd.db
...
mysrv:/var/spice # saslpasswd2  -a qemu xen -f /var/spice/sasl/passwd.db
...
mysrv:/var/spice # saslpasswd2  -a qemu xen@xensrv -f /var/spice/sasl/passwd.db
...

Afterward, you will find two new files in "/var/spice/sasl":

mysrv:/var/spice/sasl # la 
mysrv:/var/spice/sasl # la
total 28
drwxr-x--- 2 root qemu  4096 Apr 18 18:09 .
drwxr-xr-x 3 root root  4096 Apr 18 18:07 ..
-rw------- 1 root root    16 Apr 18 18:09 passwd.db.dir
-rw------- 1 root root 23384 Apr 18 18:09 passwd.db.pag
mysrv:/var/spice/sasl # 

Note the rights: The passwd-files can only be read by root. Why? Well, such a SASL file contains the credentials in clear text! It must be protected!

But, with these settings we would again run into a problem, which we have already seen in previous articles: The user of the qemu-processes is "qemu" on Leap 15.2-systems - at least when qemu-processes are started by libvirt-tools! This user "qemu" must be allowed to read critical files, in our present situation the "passwd.db"-files. Therefore, we change the group to "qemu" and the access rights to "640":

mysrv:/var/spice/sasl # chown root.qemu pass*
mysrv:/var/spice/sasl # chmod 640 pass*
mysrv:/var/spice/sasl # la
total 28
drwxr-x--- 2 root qemu  4096 Apr 18 18:09 .
drwxr-xr-x 3 root root  4096 Apr 18 18:07 ..
-rw-r----- 1 root qemu    16 Apr 18 18:09 passwd.db.dir
-rw-r----- 1 root qemu 16384 Apr 18 18:09 passwd.db.pag
mysrv:/var/spice/sasl # 

Warning: Please check that the qemu-group contains the user "qemu", ONLY !

To check which users are enlisted in our backend we use the command "sasldblistusers2".

mysrv:/var/spice/sasl # sasldblistusers2 -f /var/spice/sasl/passwd.db 
xen@xensrv: userPassword
xen@mysrv: userPassword
uvmb@mysrv: userPassword
uvmb@mysrv.anraconc.de: userPassword
myself@mysrv: userPassword
myself@mysrv.anraconc.de: userPassword
mysrv:/var/spice/sasl # 

You see that users provided without a system realm automatically got the present system's name attached!

If we wanted to delete a user we would use:

mysrv:/var/spice/sasl # saslpasswd2  -d xen@xen -f /var/spice/sasl/passwd.db
mysrv:/var/spice/sasl # sasldblistusers2 -f /var/spice/sasl/passwd.db 
xen@mysrv: userPassword
uvmb@mysrv: userPassword
uvmb@mysrv.anraconc.de: userPassword
myself@mysrv: userPassword
myself@mysrv.anraconc.de: userPassword
mysrv:/var/spice/sasl # 

For tests you should vary the passwords of usernames defined for different systems.

Indications of basic security problems

As long as a standard TCP port is defined for Spice fallback-situations may occur without any announcement. But remote-viewer checks whether the tunnel really is TLS secured or not. If not, it will automatically try to choose a SASL mechanism with encryption - as e.g. digest-md5 - if such a mechanism is available to the qemu-process on the server. If such a mechanism is not available remote-viewer will block the access to the Spice console:

A typical indication that something is wrong is a decline to enter a username in the dialog window for authentication opened by remote-viewer.

Problematic SASL mechanisms of course are PLAIN and all SCRAM-mechanisms. A noteworthy fact about remote-viewer is:

remote-viewer will try to use a SASL mechanism with encryption even if you have explicitly specified something else!

Problematic situation arise for a remote-viewer command (executed locally or on a real remote client) which contains the specification of a standard non-TLS port and a TLS-port - e.g.:

remote-viewer spice://myserv.anraconc.de:20001?tls-port=20002

for a VM with a Spice configuration that includes a standard non-TLS port and "defaultMode" set to "insecure" or "any". The same is true for:

remote-viewer spice://myserv.anraconc.de:20001?tls-port=20002&sasl=plain

If remote-viewer in its background dialog with the server does not find a "worthy" mechanism it will give you an error hint:

Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found)

Spice configuration to be used with plain text SASL mechanisms

To be on the safe side we configure Spice in the XML-definition file for a VM as follows:

   <graphics type='spice' tlsPort='20002' autoport='no' listen='0.0.0.0' keymap='de' defaultMode='secure' >
      <listen type='address' address='0.0.0.0'/>
      <image compression='auto_glz'/>
      <gl enable='no'/>
    </graphics>

The difference in comparison to specifications in previous articles is that we do not specify a standard "port" at all - but still have "autoport='no'" in place AND set "defaultMode='secure'".

How do we specify a SASL mechanism when starting remote-viewer?

The general form is as simple as this

remote-viewer spice://FQDN??tls-port=NR_OF_TLS_PORT&sasl=MECHANISM

where "FQDN" refers to the address of the server, "NR_OF_TLS_PORT" to the tlsPort settings for Spice in the VM's configuration and "MECHANISM" to one of the SASL mechanisms in the list of the "/etc/sasl2/qemu.config"-file.

Remote test

We are confident and test our present settings. We restart libvirtd on the server. A privileged user "uvma" starts the test VM there for us. Then we login into the remote system - in my case as user "myself" into "MyLAP". On a terminal we enter:

myself@mylap:~> remote-viewer spice://mysrv.anraconc.de?tls-port=20002&sasl=plain

We then get an authentication dialog:

We enter the password defined in the credentials file - in my case for "myself@mysrv.anraconc.de" and not for "myself@mysrv" (!!) - and there we go:

After a login we can work with the desktop of the VM as described in previous articles and open up more screens.

Tests for realm dependence of the authentication

We are now able to test the dependency of the authentication on the "realm"; we just type in the "username@realm" which we want to test and provide the related password. Try also cases without providing a "realm". I leave it to you to perform such tests.

Afterward it should be clear that the FQDN shown in the authentication dialog is important for the validity of the TLS server certificate - but it has nothing to do with the SASL realms!

A dis-functionality: Freezing Spice windows if the Spice session is taken by another user

Now you could try to do the same shortly after and locally on the server "mysrv" - i.e. you try to steal the "seat" in front of the Spice console. In previous articles we have seen that the Spice windows of the user, from which the seat was taken, were just closed without any warning. This was not the case on my systems with TLS and SASL. Instead and unfortunately the Spice window of the original user froze; he/she could not enter anything anymore. One could not even close it explicitly without killing the job.

Otherwise the handling of an active Spice session is as we are used to.

What do we get for situations with TLS not fully enforced?

Just for testing purposes we change the security situation by adding a standard port to the Spice configuration and set the defaultMode in the VM's XML-configuration to 'any':

   <graphics type='spice' port='20001' tlsPort='20002' autoport='no' listen='0.0.0.0' keymap='de' defaultMode='any' >
      ...
    </graphics>

Then a dubious situation as provoked by

myself@mylap:~> remote-viewer spice://mysrv.anraconc.de:20001?tls-port=20002&sasl=plain

The TLS-option is ignored in this case and this in turn leads to a decline to enter a user@realm in the first field of the dialog:

accompanied by a warning on the terminal window :

(remote-viewer:21318): GSpice-CRITICAL **: 17:33:36.750: Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found)

However, if you now change the "mech_list" for SASL in "/etc/sasl2/qemu.config" such that it includes "digest-md5", then

myself@mylap:~> remote-viewer spice://mysrv.anraconc.de:20001?tls-port=20002

would work. And also - somewhat unexpected - the following works:

myself@mylap:~> remote-viewer spice://mysrv.anraconc.de:20001?tls-port=20002&sasl=plain

Wireshark, however, reveals that the specified sasl-mechanism is ignored and digest-md5 is taken in this case:

Well, this is a bit confusing as remote-viewer does not inform the user about the measures taken. My advice is to prepare for clarity: If you want to work with SASL then disable the standard non-TLS port for Spice and do not include weak SASL crypto-mechanisms in the mech-list at all. It is better that remote-viewer stops the conversation ....

Failure for local access to the Spice console from virt-manager

There is a problem with our qemu-setting that enforces SASL and virt-manager. Afterwards we can no longer access the Spice console locally on the KVM/Qemu-server with the built-in Spice client of virt-manager. We get a screen that wants a password from us - but no user and realm:

You can fill in whatever you want of known passwords for root, qemu and other users which you may have filled in into your sasldb. (You could have added an entry for "root@mysrv.anraconc.de" with some password, because virt-manager runs with user root). It is not going to work.

There seems to be a basic problem; we get a log message as

  
2021-04-30T17:05:20.745776Z qemu-system-x86_64: warning: Spice: red-stream.c:1039:red_sasl_start_auth: sasl context setup failed -7 (invalid parameter supplied)

The funny thing is that another libvirt-dependent tool, namely "virt-viewer" works correctly with SASL. You can test it locally on the KVM/Qemu-server, e.g. for the following situation:

  • Allow for a standard Spice port without TLS (in our test case 20001; in our series we have not yet come to the point where we use libvirt-tools remotely together with TLS 🙂 ).
  • Let Spice listen on all on all network devices (including 127.0.0.1).
  • Activate SASL for qemu (via settings in "/etc/libvirt/qemu.conf"; see above).
  • Allow for a SASL mechanism with encryption.

If you then enter e.g. the command

mysrv:~ # virt-viewer -c qemu:///system

(e.g. as user root) an authentication dialog opens, where we can fill in a username ("root" is suggested) plus a realm, if we want, and a valid password for the combination.

Enter a valid password for some entry (e.g. for root@mysrv.anraconc.de") in the sasldb and the Spice console comes up as usual. So - there is some dis-functionality in virt-viewer, when SASL is activated for qemu (and not a libvirt port or local libvirt Unix socket).

Switching to the saslauthd daemon

"/etc/sasl2/qemu.config" was copied from a libvirt-oriented file. We can, of course, specify other SASL rules for the service "qemu". In particular we can invoke a "saslauthd" daemon. We achieve this by changing the settings in "/etc/sasl2/qemu.config" to:

mech_list:  plain scram-sha-256
pwcheck_method: saslauthd

Please out-comment other lines. Why did I only specify clear text mechanisms? Well, saslauthd simply does not work with crypto-mechanisms! It is not designed for it. It is invoked locally on our KVM server where it listens on a Unix socket after having been started:

mysrv:~ # netstat -ax | grep sasl
unix  2      [ ACC ]     STREAM     LISTENING     603842   /run/sasl2//mux

To get that far there are a few more things to do, of course: We must configure saslauthd and start the related service. The basic configuration of "saslauthd" on a Leap 15.2 system is done via settings in the file "/etc/sysconfig/saslauthd".

For a simple scenario, which uses the standard PAM mechanisms on the server, we can leave the default settings there as they are:

SASLAUTHD_AUTHMECH=pam
SASLAUTHD_THREADS=5
SASLAUTHD_PARAMS=""

What is the expected effect? Well, afterward, we should be able to authenticate ourselves with credentials (usernames and passwords) already defined for valid accounts on the server (e.g. in "/etc/passwd" and "/etc/shadow"). Note:

No login shell is required for our special Spice users!

So, we can create some (harmless) user accounts on the server to test the Spice/SASL authentication (with some preset complicated passwords).

Before testing we should not forget to start the daemon by "systemctl start saslauthd" or "rcsaslauthd start". But before we do this there is one more step required ....

PAM configuration to support authentication for Spice by saslauthd

PAM works for services - and what PAM should do for a service has to be defined. The configuration files are placed in "/etc/pam.d/". Naively, I had expected that we would need a "qemu" configuration file there. Well this is wrong. I could see it from resulting error messages in "/var/log/messages"

... saslauthd[25571]:                 : auth failure: [user=myself] [service=spice] [realm=] [mech=pam] [reason=PAM auth error]

The "service" to be configured is "spice" ! Ok, the machine gets what it wants. I created a simple file "/etc/pam.d/spice" with the following standard contents:

#%PAM-1.0
auth     requisite      pam_nologin.so
auth     include        common-auth
account  include        common-account
password include        common-password
session  required       pam_loginuid.so
session  include        common-session

This is enough to support our SASL operations; some lines as e.g. for the password could even have been omitted for our present objectives. Note hat a "realm" does not make much sense if we work with a local PAM for Linux password files on a Linux host, only, and no external backend.

Test of salsauthd

Stop your VM if it were running. Restart libvirtd, start saslauthd, start the VM.
You can pre-test your saslauthd configuration for the service "spice" with existing user accounts on the server with the command

mysrv:~ # testsaslauthd  -s spice -u uvmb -p ******** 
0: OK "Success."

You must of course replace "uvmb" without a username defined on your server and the "******" with the related password. .

Now you can try it for a remote-viewer session. The same authentication dialog we got before for our sasldb-experiment should appear. You then only enter a username (no realm) into the filed for the username. Should work without any problems.

Major deficits of the SASL-authentication for remote-viewer

During his/her tests of the authentication with sasldb2 the reader has certainly realized the following point:

Any valid combination of a username, a related realm and a password authorizes us to get access to the Spice console of our test-VM.

There is no unique relation between a UID on the remote-client or on the server and the "username@realm" combination. And there is no real relation of a username/user with a specific VM. Therefore, the situation is actually worse:

A user providing a valid "username@realm" (existing in sasldb2) has access to the Spice consoles of all VMs running on the server!

Activating SASL in the general way we did it for the startup of all qemu-processes establishes an authentication and authorization mechanism for the Spice consoles of all VMs on a KVM server - if the user knows the relevant TLS-ports or just checks out ports systematically.

Obviously, the logic for our present setup is that we authenticate a general Spice user via SASL on the server. We cannot establish a relation between the Spice related SASL "username@realm" and a specific VM. The present version of remote-viewer does not provide other data (as an rhost) which we could e.g. use in some cleverly crafted PAM rules for special modules. (You can see the information provided from log-entries for your VM and wrong authentications)

In my opinion an authorization should be specific for a user (or a group of selected users) and a specific target service. And it should be such that the server alone can control the service access. For a bunch of VMs on the KVM/Qemu-server we would assume that we could establish a specific user authentication for the Spice console of a specific VM and no other VMs.

So, it is very unfortunate that "realm" delivered by remote-viewer can freely be set by the user and that it is independent of the TLS parameters and/or data of the remote-host. Thus, there is no way to force the user on the client-system to enter a VM-specific user@realm combination. And any other user sharing one valid username/password combination known to him opens up access to all Spice consoles for other users - if no additional measures were taken.

If you wonder how a coupling of certificate data to the realm would have helped: We could have achieved a relation between a realm and a VM, if we had coupled the VM to a sub-domain for a wildcard certificate on the server and to a specific IP on the server - and if the realm were taken from the "common name" (i.e. the FQDN) in the certificate. Then we had just needed a specific username@ realm entry in the sasldb - and we would have gotten a VM-specific authentication. In the case of saslauthd the delivery of information about the remote host would have helped. I leave it to the user to figure that out. Its fruitless thinking anyway, ...

Actually, setting a password int VM's XML file would have given us a VM-specific authentication. Unfortunately, we are not allowed to combine the setting of a VM-specific password in the XML-file in addition to SASL-authentication. The VM can not be started by virt-manager or virsh then. So, it is a bit of a mess with SASL and remote-viewer ...

Are there valid counter arguments?
Well, you could argue that we could hinder access to specific VMs by blocking ports on the server for certain client-systems. Well, and what for situations where multiple users are allowed to login to any client-machine in an office?
And you could argue: But the user has certainly to login into the VM as well? Yeah, but there is the problem with Spice - if someone legitimately had already opened a a session on the VM via a Spice console this session could be taken from him/her in the present open status by an other user authenticated via SASL for some other entry in the sasldb. Security is something else ...

Conclusion

To activate SASL for some general authentication to get access to the Spice console of a chosen VM is simple. However, the SASL authentication does not allow us

  • to restrict the access to the Spice console of a specific VM to a specific username@realm combination.
  • to restrict the access to the Spice console to a specific UID on either the remote client or the KVM-server.

So, TLS in combination with SASL is to a certain degree a disappointment. A VM-specific password has more advantages. Note, that for present versions of "qemu" it is not displayed anymore in the output of "ps aux | grep qemu".

The combination of TLS and SASL with remote-viewer is not really useful for situations where we want Intranet users to work with VMs via Spice sessions - each of them with a (user-) specific VM or some (user-) specific VMs, only.

If you wanted to make Spice sessions available to a selected user only for specific VMs we thus end up with the question: How can we become more restrictive?

In the next article

KVM/Qemu VMs with a multi-screen Spice console – VIII – VM and user specific restrictions for remote-viewer connections – iptables and sudo

I will therefore discuss the application of user specific iptables- and sudo-rules on remote client-systems. Not ideal - but a step forward.

KVM/Qemu VMs with a multi-screen Spice console – V – remote access via remote-viewer, a network port and a SSH-tunnel

I continue my article series about methods to access the Spice console of a KVM/Qemu based VM. Spice clients - as e.g. remote-viewer and virt-viewer - enable local or a remote users to work on the graphical desktops of a VM.

KVM/Qemu VMs with a multi-screen Spice console – IV – remote access via SSH, remote-viewer and a Unix socket
KVM/Qemu VMs with a multi-screen Spice console – III – local access with remote-viewer via a Unix socket
KVM/Qemu VMs with a multi-screen Spice console – II – local access with remote-viewer via a network port
KVM/Qemu VMs with a multi-screen Spice console – I – Overview over local and remote access methods

In my last article I used a local socket based connection of remote-viewer to the Spice console of a VM on the KVM/Qemu host. I just added SSH to achieve a kind of elementary network capability: I transferred the graphical output of remote-viewer running on the KVM host via SSH to the X11-service of a remote client-system located somewhere in a LAN. With some additional SSH-tricks for audio data I got a well working, secure and surprisingly responsive solution for a remote desktop of a VM.

However, without data compression, such an approach consumes a considerable part of the LAN-bandwidth as soon as reactions of the VM's window manager to fast window movements are requested on the remote SSH client: For a "virtio" video device of the VM we measured up to 45 MiB/s. SSH data compression helped to bring data transfer rates below 9 MiB/s. Unfortunately, the gzip based compression of SSH led to some reduction in responsiveness when I moved windows fast and irregularly across the VM's desktop surface.

But when I worked with figures and operations of graphical applications (like Libreoffice Draw) within the "ssh -X" based display of the VM's Spice console screens on my remote client, I found much, much smaller data transfer rates. These rates were even significantly smaller than the rates required for a direct "ssh -X" connection to the VM for the display of the chosen applications's graphical output on our remote X11-service. So, even my simple scenario for the remote display of the VM's whole desktop via "local Spice" and "ssh -X" offers a remarkable advantage in comparison to a pure "ssh -X" access to graphical applications of a VM.

With this article we now turn to "real" remote configurations. "Real" refers to the fact that both remote-viewer and virt-viewer are parts of a client/server architecture for the Spice protocol: This time we are going to run remote-viewer on the remote system.

This in turn means that we have to establish a network connection from the remote client-system to a (dedicated) TCP network port on the KVM-server to access the Qemu-hypervisor process for a specific VM. Graphical desktop data are then transferred between the Qemu-hypervisor on the KVM-server and the remote client via the Spice TCP protocol. This is much closer to the original intentions of the Spice developers than what I did in my last article.

But: As SSH allows for port-forwarding we can, of course, easily combine this kind of real remote client/server approach for Spice with SSH encryption. Thus we have two subjects to cover in this article:

  • Method A: We access the Spice console of a VM by using remote-viewer on a remote client-system and interact with the VM via an unencrypted TCP connection to a specific network port on the KVM/Qemu-server.
  • Method B: We access the Spice console of a VM by using remote-viewer on the remote system but transfer encrypted data between the client and the KVM-server through a SSH-tunnel with port-forwarding.

Note that both approaches should already cover the transfer of audio data along with the video data without any additional measures - provided that Pulseaudio is running on the client-system. Remember what we saw already in the 2nd article of this series: remote-viewer opens multiple data transfer channels - one of it is intended for audio data.

As in the last article we will measure data transfer rates and consider the felt responsiveness of the solutions. In addition we are now able to apply (native) data compression methods within the Spice protocol and compare the results with the gzip compression offered by SSH.

Schematic drawing

Both methods for our remote scenario are displayed in the following drawing:

The main difference in comparison to the sketch in my last article is: The remote-viewer application is now started on the remote client-system and not on the KVM-server. The remote system is in my case a laptop "MyLAP"; I also speak of it as the (remote) client.

Spice configuration

In one of the previous articles I have already discussed the Spice configuration settings in a (libvirt) XML definition file for our test-VM "debianx":

     <graphics type='spice' port='20001' autoport='no' listen='0.0.0.0' keymap='de' defaultMode='insecure'>
      <listen type='address' address='0.0.0.0'/>
      <image compression='off'/>
      <gl enable='no'/>
    </graphics>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>i
    <video>
      <model type='virtio' heads='2' primary='yes'>
        <acceleration accel3d='yes'/>
      </model>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

We define a specific TCP port (20001) to be used for our test-VM. (Another additional VM on the KVM host would require the definition of another port). The "defaultMode", by which we control whether TLS security measures are required to start the VM, is set to "insecure"; i.e. we neglect TLS encryption for the time being. Note also that I use a "virtio" video device. We made good experiences with it during our last experiments. If the virtio device should not work on your systems replace it by some reasonable QXL configuration.

The HW graphics "acceleration" can be set to "yes" for the "virtio" device. HW acceleration will, however, not be used as long as Spice has the setting "<gl enable=no>". [You may try change this - good luck then with Nvidia cards and their proprietary drivers (it won't work). I will not cover HW acceleration of the virtual graphics in this series.]

Note that I did not set a password to block other users from accessing the Spice console. So, anybody in our Intranet can take over an already opened Spice session - without any authentication. That is one of the reasons why we shall invoke SSH again in a few minutes.

A first test

We open local and router based firewalls in our (segmented) LAN for the communication of the client-system with the virtualization server over port 20001. On the KVM-server "MySRV" a privileged user "uvma" starts our already familiar test-VM "debianx" (in my case with a Kali OS on it) via virt-manager. Just for control purposes user "uvma" opens the Spice console on the server with a local remote-viewer instance (with 2 screens), logs in into the VM and starts a VM desktop session:

Then I log myself in into the Linux client "MyLAP" as user "myself" and start remote-viewer there:

myself@mylap:~> remote-viewer spice://mysrv:20001 & 

The expected result is that the Spice console session is closed on the (KDE) desktop session of "uvma" on MySRV. Instead Spice console windows are opened in the (KDE) desktop session of "myself" on "MyLAP" - displaying again the desktop of the VM:

(I adjusted the Spice window positions a bit).

The necessity of some security measures is obvious: The opened network port on the KVM server could in principle be accessed by anybody in the LAN. Even if we restricted access by some firewall rules to the MyLAP-client any of its users could hijack the Spice console session. Furthermore: No authentication is required; the person having accessed the Spice console can work freely on the VM with the rights of the user who was actively using the desktop on the VM before. And the data exchange between MyLAP and MySRV occurs unencrypted.

Before we take care of encryption and other security measures, let us compare the present data transfer rates with the rates seen in the experiment of my last blog post.

Data transfer rates without compression

In my last article we put some pressure on the window-manager of the VM by moving a window bigger than 800x800 px with complex content fast and irregularly across the surface of the VM's desktop. We got data exchange rates between the remote client and the KVM-server beyond 80 MiB/s for a QXL device; for a "virtio" video device the pak rates were close to 45 MiB/s.

In my present remote scenario I observe the following rates for data sent from MySRV to MyLAP:

Hmmm ..., rates of up to 70 MiB/s are not at all convincing. The responsiveness is excellent again: Application windows follow the erratic mouse movements across the desktop very quickly and with a completely negligible delay.

So far the client/server approach with remote-viewer does not offer us any advantage over our primitive "ssh -X" scenario for the transfer of graphical data from a local Spice client on the KVM-server to the X11-server on a remote system.

Transfer rates with activated Spice data compression

The Spice settings include an option <image compression='....'/>.
In the experiments of last posts we could not use this option; virt-manager (libvirtd) would not start a VM that provides a Spice console via a local pure Unix socket in combination with data compression. Obviously, such a combination was and is regarded useless by the Spice developers.

But now, as we work with a network port, we are allowed to define a method for the compression of Spice data. You find valid values for compression methods at https://libvirt.org/formatdomain.html#video-devices.

For our next test we use

 <graphics type='spice' port='20001' autoport='no' listen='0.0.0.0' keymap='de' defaultMode='insecure'>
      <listen type='address' address='0.0.0.0'/>
      <image compression='auto_glz'/>
      <gl enable='no'/>
    </graphics>
   

We get

The resulting data transfer rates are below 9 MiB/s. (Your rates may deviate a bit due to a different number of Spice screens, different Spice screen dimensions, different window dimensions and window content. With 2 Spice screens of 1920x1200 px and a FF window of 1200x800 px I get up to 12 MiB/s).
These values are comparable with the rates found for SSH data compression in the last article's experiment. However, there is a major difference:

The responsiveness remains really excellent - despite data compression!

This is a first good argument for using remote-viewer in the manner it was originally designed for - namely as a remote tool!
But we have no encryption, yet ....

What about sound?

Remote-viewer should support the local Pulseaudio [PA] server on the client system without any further measures . And it indeed does so:

The picture above shows that PA (on MyLAP) has recognized the locally started remote-viewer there as a valid audio source - and we can play any sound with any player of the VM. So, getting sound with remote-viewer running on the remote client is considerably easier than fiddling around with the PA and SSH tricks we had to apply in the basic "ssh -X" scenario.

Data transfer rates for LO Draw (with Spice data compression)

As in my last article we also open a LO Draw sheet to test data transfer rates for a graphical application used within the VM's desktop:

The measured data transfer rates whilst moving colored transparent figures fast across the LO sheet of LO Draw are excellent - namely on average below 500 KiB/s:

The rate depends a bit on the size of a moved element. Regarding the felt responsiveness: It is really comparable to working on a local application - you do not feel that you are working on a VM on some server over a LAN connection.

Add encryption via SHH and port forwarding

Now, let us add SSH encryption. On the remote client-system "MyLAP" we open a terminal window and enter:

myself@mylap:~> ssh -N -f -L 31001:localhost:20001 -i ~/.ssh/id_rsa_x uvmb@mysrv
Enter passphrase for key '/home/myself/.ssh/id_rsa_x': 
myself@mylap:~> remote-viewer spice://localhost:31001 &

By "-N" we just signal that we do not want to get an interactive shell, and by "-f" we fork the SSH process into the background. In addition we redirect data traffic targeted for the arbitrarily chosen port 31001 on MyLAP to port 20001 on MySRV through an encryption tunnel. (Control question: What system does "localhost" in the SSH statement refer to?)

And there we go again:

And the data transfer rates as well as the responsiveness remain excellent as before; here the rates for moving a FF window quickly across a Spice screen of 1920x1200 px:

And here for quickly moving figures around a full screen LO Draw sheet:

Hints for improving security

We have established a SSH tunnel for encrypted data transfer between our systems. But this is not enough regarding security as anybody having SSH access to the KVM host can still hijack an open Spice console session in an uncontrolled way. What can we do to improve security? In particular, we have to restrict the access to the VM's Spice console to specific users. The first measure to achieve this is to close the network port defined for Spice again on the KVM-host for remote access. We do not need it to be accessible from external locations as we use a SSH tunnel anyway. And then there is a cascade of additional things you can do with SSH:

  1. You create a special user on the KVM-server - but set his login-shell to "/bin/false" or "/sbin/nologin". Thus he/she cannot work interactively with a shell on the KVM-server. But port-forwarding would still be possible for him/her ...
  2. You allow SSH connections for this user from a special IP address, only, and via public key authentication, only. (You must configure the SSH service on the KVM host accordingly). Then you create the required key pair for this user and place the public key into the file "~/.ssh/authorized_keys" on the KVM/Qemu server.
  3. You confine the actions of this special user even more by adding port-forwarding restrictions to his/her file for public keys "~.ssh/authorized_keys".
  4. You restrict the allowed actions of this special user in addition by a "Match user"-section in the sshd_config-file on the KVM-server.

The required restriction would be the dis-allowance for any port-forwarding with the exception of the defined Spice port for the VM. In addition you also disable X11-forwarding and SSH Gateway ports. Then you block port-forwarding to the VM's Spice port for all other SSH users (with the exception of an administrator account, maybe).
For more information see the SSH documentation of your distribution and e.g. :
https://askubuntu.com/questions/48129/how-to-create-a-restricted-ssh-user-for-port-forwarding
https://blog.tinned-software.net/restrict-ssh-access-to-port-forwarding-to-one-specific-port/

And last - but not least - you, of course, set a password in the Spice configuration. We have covered this topic already in a previous article.

Conclusion

Remote-viewer used on a remote client-system without any data compression requires almost the same data transfer rates as a solution based on the transfer of graphical data via "ssh -X" from remote-viewer running on the KVM-server to a remote X11-service. However, remote-viewer run as a real Spice client tool on a remote system has one big advantage over a scenario based on a pure "ssh -X" solution:
The compression methods integrated with the Spice protocol have almost no negative impact on the responsiveness of the VM's desktop displayed in the remote Spice windows!
For a standard compression setting we get a substantial reduction of data transfer rates combined with an almost optimal responsiveness of the remote desktop. We can work with graphical applications within the desktop of a VM on remote Spice windows as if we were using a local application. Required rates for LO Draw are in the range of 0.5 MiB/s, only.
To get a secure configuration we can at any time sent the Spice data through an encryption tunnel established via SSH and port-forwarding. This had no negative effect on responsiveness. Additional restrictive user-specific configurations of SSH for port-forwarding offer a solid basis for the creation of a reasonably secure Spice solution.

Conclusion: The combination of remote-viewer on remote systems with Spice data compression and a SSH-tunnel offers users who can live with a "one seat" remote scenario an almost optimal solution for working with graphical applications of a VM within a remote desktop. Regarding data transfer rates this is much better than using "ssh- X" directly for the application (without the desktop environment).

In the next article we prepare our systems for a TLS encrypted connection instead of a SSH-tunnel.

DSGVO, Freelancer, E-Mails und Umzug KVM-virtualisierter Linux-E-Mail-Server auf verschlüsselte Platten/Partitionen – III

Das Thema der vorhergehenden Artikel dieser Serie

DSGVO, Freelancer, E-Mails und Umzug KVM-virtualisierter Linux-E-Mail-Server auf verschlüsselte Platten/Partitionen – I
DSGVO, Freelancer, E-Mails und Umzug KVM-virtualisierter Linux-E-Mail-Server auf verschlüsselte Platten/Partitionen – II

war eine allgemeine Diskussion darüber, warum man sich spätestens nach der DSGVO als Freelancer um Schutzmaßnahmen für E-Mails und um entsprechende vertragliche Vereinbarungen kümmern sollte. Eine reine Verschlüsselung von Transportwegen ist meiner Meinung nach nicht hinreichend; eine Lagerung von Mails in verschlüsselten Dateicontainern ist mit zu vielen Gefahrenpunkten verbunden. An einer Verschlüsselung von Volumes oder Partitionen der mail- und datei-verarbeitenden Systeme führt bei der Aufbewahrung von Mails und Anhängen kein Weg vorbei.

Als Freelancer steht man also womöglich vor der Aufgabe, sowohl den eigenen Mail-Server im LAN als auch Clients zur Mail- und Auftragsbearbeitung auf einen Unterbau aus verschlüsselte Partitionen umzustellen. Als Linuxer, die auf effiziente Ressourcennutzung bedacht sind, greifen wir zur Lösung auf virtualisierte Systeme zurück. Sprich: Meine Reaktion auf die von etlichen Kunden an mich gestellten pauschalen DSGVO-Anforderungen ist, die Auftragsverarbeitung für Kunden nur noch über besonders geschützte virtualisierte Systeme auf verschlüsselten Volumes durchzuführen. Der aufmerksame Leser hat sicher bemerkt, dass dieser Gedanke über die Mail-Verarbeitung hinaus weist.

Ich gehe kurz auf die Grenzen eines solchen Vorgehens ein; anschließend kümmern wir uns (endlich) um den Umzug eines bereits virtualisierten Mailservers auf eine verschlüsselte HD/SSD-Plattform.

Ist Volume- und Partitionsverschlüsselung einer Systeminstallation für Datensicherheit hinreichend?

Der Grundgedanke einer Partitions- oder Volume-Verschlüsselung ist: Nur verschlüsselte Daten sollen über verschiedene Zugriffsschichten Plattencontroller und Magnet- wie Flash-Speicher erreichen; eine vollständige Entschlüsselung von Dateien soll ausschließlich im RAM des Betriebssystems [OS] stattfinden. So schön das sein mag - es hilft nicht, wenn sich jemand bereits unbefugt auf dem System eingenistet hat und munter mitliest:

Die Verschlüsselung von Datenträgern, Partitionen oder Volumes bietet keinen Schutz auf gehackten Systemen. Sie bietet "nur" Schutz gegen unbefugten Zugriff im nicht-entschlüsseltem und/oder nicht-gemountetem Zustand der Volumes also z.B. im heruntergefahrenen Zustand des Systems.

Volume-Verschlüsselung ist also nur eine - wenn auch eine wichtige Maßnahme - zur Ausschaltung bestimmter Risiken. Auch das sollte aus meiner Sicht in einem Vertrag mit einem Auftraggeber festgehalten werden. Natürlich muss man sich u.a. auch um die Integrität der Virtualisierungshosts selbst kümmern - wir reden da aber über Systeme mit einer sehr begrenzten Anzahl an OS-Komponenten, die selbst keine direkte Verbindung zum Internet aufnehmen müssen.

Man erkennt, dass ein vernünftiger DSGVO-bezogener Vertrag zum Schutz personenbezogener und anderer geheim zu haltender Daten im Endeffekt wesentlich mehr beinhalten muss als nur Hinweise auf die Mailbehandlung und zugehörige Krypto-Verfahren für Transport und Lagerung. U.a. wird man eine Begrenzung ein- und aus-gehender Kommunikation von (virtualisierten) Servern und Clients, die bei der Auftragsbearbeitung eingesetzt werden, auf nur sehr wenige erlaubte Adressen im Internet umsetzen müssen. Z.B. gilt: Völlig offene HTTP(S)- / SMTP(S) - / IMAP(S)- oder gar UDP-basierte Kanäle nach außen zu beliebigen Zieladressen sind im Zeitalter von strafbewehrten DSGVO-Verträgen überhaupt keine gute Idee! Zumindest nicht auf denjenigen Server- und Client-Systemen, auf denen man mit geheim zu haltenden Informationen operiert.

Virtuelle KVM/QEMU-Systeme auf verschlüsselten Volumes

Will man aus anderen grundlegenden Risikoerwägungen heraus nicht gleich alle "Volumes" auf allen Systemen im Haus-/Firmen-Netz verschlüsseln, bleibt nur der Weg über virtuelle Maschinen. Das betrifft dann Server- wie Client-Systeme gleichermaßen.

Im Besonderen Linux-E-Mail-Server sind dabei recht komplexe Systeme; eine Neuinstallation der vielen Einzel-Komponenten auf verschlüsselten Platten/Partitionen mag man sich daher wirklich ersparen. Es geht somit um System-Migration. Nachfolgend betrachte ich deshalb den Umzug eines (virtuellen) (E-Mail-) Servers auf einem KVM/QEMU-Host von unverschlüsselten Partitionen/Volumes auf verschlüsselte Volumes.

Ich gebe einige wichtige Befehle am Beispiel eines KVM/QEMU-Hosts unter Opensuse an. Das ist insofern nützlich, weil man hierfür mit YaST allein nicht auskommt und auf die Kommandozeile runter muss. Für einen Umzug voll ausgestatteter virtueller Clients (KVM/QEMU-Linux-Gastsysteme) zur Mail- und Auftragsbearbeitung gelten die Ausführungen dann analog.

Übrigens: Durch voll-virtualisierte KVM/QEMU-Systeme (für Mailserver und Clients),

  • die netzwerktechnisch weitgehend abgeschottet sind,
  • über die kein freier Internetzugang mit Browsern erlaubt ist
  • und die auch nur spezielle Mail-Accounts bedienen

kann man auch Hackern das Eindringen ein wenig erschweren!

Umzug eines physikalischen Linux-E-Mail-Servers in eine Virtualisierungsumgebung?

Das erste Problem besteht für manchen Leidensgenossen ggf. darin, einen schon vorhandenen physikalischen Server in eine Virtualisierungsumgebung zu bringen. Ehrlich gesagt, habe ich das noch nie selbst gemacht - und leider auch keine Zeit, das mal testweise auszuprobieren. Server für spezielle Aufgaben sind in meinem begrenzten Netz schon seit langem aus guten Gründen virtualisiert 🙂 .

Ich verweise für diese Aufgabe unter Linux deshalb auf sog. "P2V-Tools" (physical to virtual) und entsprechende Literatur. Die ganz unten aufgeführten Links geben hierzu Hinweise und Anleitungen. Für die nachfolgenden Schritte setze ich voraus, dass es bereits einen unter KVM/QEMU virtualisierten Server gibt - allerdings auf unverschlüsselten Partitionen/Volumes.

Vorüberlegung A: Wie sieht es mit der Performance und der Flexibilität auf verschlüsselten Plattformen aus?

Kann man sich auf seinen Systemen eigentlich eine Verschlüsselungsschicht zusätzlich zur Virtualisierung leisten? Das ist eine gute Frage, die man pauschal schlecht beantworten kann. Unter Linux kommt meist eine Kombination aus dm-crypt und LUKS zum Einsatz. Für Systeme mit Prozessoren der letzten 5 Jahre sieht es dabei hinsichtlich der reinen Krypto-Performance recht gut aus. Bei Virtualisierungshosts im Heim-Bereich (also eher mit begrenzten Ressourcen) sind aber ein paar zusätzliche Punkte zu beachten:

Schichtung
Hat man als Plattenunterbau Raid-Systeme im Einsatz, so stellt sich etwa die Frage der Schichtung der verschiedenen Zugriffsverfahren. Aus meiner Sicht ist folgende Reihenfolge für virtualisierte Serversysteme sinnvoll:

  • Plattenpartitionen für Raid   >>  
  • Raid 10   >>  
  • LVM   >>  
  • Volumes für KVM-Host-OS   >>  
  • dm-crypt/Luks für ein LVM-Raw-Volume   >>  
  • KVM/QEMU- und Gastsystem mit Zugriff auf das Raw-Volume als virtuelle Platte   >>  
  • LVM /Volumes (oder Partitionen) im Gastsystem   >>  
  • ext4-Filesysteme im Gastsystem

Verschlüsselung vor Raid scheint mir aus naheliegenden Gründen wenig performance-optimierend zu sein. Bzgl. der Reihenfolge "LVM <> dm-crypt/Luks" kann man streiten. Für die eine oder andere Wahl ist aus meiner Sicht dabei nicht die Performance sondern die Flexibilität ausschlaggebend. Bei der von mir gewählten Vorgehensweise verschlüsseln wir LVM-Raw-Volumes und nicht die sie tragenden Platten-Partitionen. Das erlaubt den Einsatz unterschiedlicher Krypto-Algorithmen und individueller Passphrases für die zu verschlüsselnden Volumes. Ein Nachteil ist, dass Größenänderungen der Volumes und Filesysteme danach nicht im Online-Betrieb möglich sind. Das stört im semi-professionellen Umfeld aber weniger (s.u.).
Siehe auch:
https://superuser.com/questions/1193290/best-order-of-raid-lvm-and-luks

Einsatz von virtio-Treiber
Für optimale Performance setze ich auf dem KVM-Host für die Vermittlung des Zugriffs des KVM-Gast-Systems auf die als virtuelle HDs bereitgestellten Volumes den QEMU-eigenen "VirtIO"-Treiber ein. Das sieht im "virt-manager" dann etwa so aus:

Übernahme der Verschlüsselung durch den Virtualisierungs-Host!
Man erkennt an der oben angegebenen Schichtung, dass ich die Verschlüsselung vollkommen dem Virtualisierungs-Host überlasse. Das erscheint mir sinnvoll, da ihm normalerweise mehr CPU-Cores als dem Gastsystem zur Verfügung stehen.

Ich kann jedenfalls mit diesem Setup auf einem schon sehr betagten Host mit einem KVM-Gast-Mail-Server, der nur um die 10 Mails pro Minute verarbeiten muss, sehr gut leben. (Der Host beherbergt dabei noch mehr aktive virtualisierte Systeme!) Der Unterschied zur Situation ohne Verschlüsselung ist gefühlt klein. Soviel zum Mail-Server und Performance.

Vorüberlegung B: Verschlüsselung eines (LVM-) "Raw"-Devices? Verzicht auf "qcow2"?

Gem. der oben propagierten Schichtung möchte ich das Gastsystem für meinen E-Mail-Server offenbar auf einem "Raw-Volume" der LVM-Schicht betreiben - also direkt auf einem LVM-Volume, das vom Gast-System aus formatiert wurde, und nicht auf einem "qcow2"-Loopback-File eines übergeordneten Filesystems des Hosts. Was spricht gegen ein Vorgehen mit "qcow2"-Dateien auf einem KVM-Host für einen virtualisierten Server?

"qcow2"-Dateien - mit internem Filesystem für den Gast - sind zwar hochflexibel einsetzbar - der Zugriff des Gastes auf seine virtuelle Platte erfolgt dabei aber immer über die Filesystem-Schicht des KVM/QEMU-Hosts UND über die Filesystem-Schicht des KVM-Gastes; das kostet Performance!

Der entsprechende Overhead ist nicht unerheblich - vor allem dann nicht, wenn der Server ggf. von mehreren Clients gleichzeitig genutzt werden soll. Die QEMU-Leute haben deshalb schon immer vorgesorgt und erlauben den direkten Zugriff eines Gastes auch auf nicht gemountete Volumes oder Partitionen des Hosts.

Der zweite Grund entspringt ein wenig Sicherheitsüberlegungen.

Auf eine "qcow2"-Datei muss vom Gast-System über ein Mount-Verzeichnis des Host-Dateisystems aus zugegriffen werden. Die Datei liegt dann also bereits in einem unverschlüsselten Dateisystem, das auf einem Verzeichnis des Hosts gemountet ist, vor. Damit besteht aber auch - je nach Rechtesetzungen - die potentielle Gefahr, dass ihr Inhalt im laufenden Betrieb auch anderweitig ausgelesen werden kann (z.B. mit qemu-mount).

Die Situation ist im Fall von Raw-Devices komplexer. Das LVM-Volume wird nach einer Entsperrung des Verschlüsselungsverfahrens (s.u.) zwar wie eine Platte als Device unter "/dev/mapper" bereitgestellt. Aber dieses Device ist auch bei laufendem Gastsystem nicht direkt im Verzeichnisbaum des laufenden Host-Systems verankert. Ein evtl. durchgeführter Mountvorgang auf dem Host ist aber gar nicht so einfach zu verschleiern. Er wäre unnötig und würde bei entsprechender Protokollierung Alarmglocken auslösen.

Dennoch gilt:

Verschlüsselung von LVM-Volumes und Einsatz von qcow2 müssen bei hinreichenden Performance-Reserven kein Widerspruch sein.

Eine Schichtung

Raid 10   >>   LVM   >>   dm-crypt/Luks-Volume mit ext4   >>   Gemountetes LUKS/ext4-Volume mit qcow2-Datei auf dem KVM/QEMU-Host   >>   KVM/QEMU-Gast mit Zugriff auf qcow2-Loopback-Device   >>   LVM-Volumes des Gastsystems auf qcow2-Device   >>   ext4-Gast-Filesystem auf LVM-Volumes des Loop-Devices

funktioniert auf schnellen Hosts auch gut und bietet Verschlüsselung samt qcow2-Flexibilität. Ich nutze diese Variante u.a. auf Linux-Workstations für virtualisierte Clients.

Ich verfolge nachfolgend für unseren geplanten (Server-) Umzug aber die Variante ohne qcow2-Loopback-Device. Auch unser bisheriger virtualisierter (Mail-) Server mag bereits direkt auf einem unverschlüsselten Volume des Hosts verankert sein.

Vorüberlegung C: Größe des neuen kryptierten Raw-Volumes für den Umzug?

Wir können nun z.B. über den LVM-Volume-Manager von YaST oder über LVM-CLI-Kommandos (vgcreate, lvcreate) ein Raw-Volume kreieren, das nach seiner Verschlüsselung eine Kopie des alten Systems aufnehmen soll. Wie groß muss dieses logische Volume sein?

Naiverweise könnte man davon ausgehen, dass genau die Größe des alten Volumes (oder der alten Partition) in GiB hinreichend ist. Das wäre falsch. LUKS hinterlegt Informationen zum verschlüsselten Plattenplatz und zu 8 möglichen Passphrase-Keys in einem sog. "Verschlüsselungsheader". Dieser Header benötigt zusätzlichen Platz jenseits der verschlüsselten Nutzlast. Der Platzbedarf ist zwar nicht groß (ca. 2 MB) - aber er ist eben zu berücksichtigen. Man muss dem neuen Volume etwas mehr Platz als dem alten spendieren. Ich vergebe meist gleich 1 GB extra.

Verschlüsselung des Raw-Volumes

In unserem Beispiel heiße das neue logische LVM-Volume für das virtuelle Plattendevice des KVM-Gastes "lvimap_hd0" und sei Teil einer LVM-Volume-Group "volgrp4". Als Device erscheint dieses Volume dann unter "/dev/mapper" als "volgrp4-lvimap_hd0".

myserv:~ # la /dev/mapper 
...
lrwxrwxrwx  1 root root       8 Jun 10 09:40 volgrp4-lvimap_hd0 -> ../dm-14
...

Der zugehörige Link verweist im Beispiel dann ebenso wie "/dev/volgrp4/lvimap_hd0" auf "/dev/dm-14".

Wir setzen zur Kryptierung "dm-crypt" ein. "dm-crypt" nutzt LUKS über gut integrierte Module.

Die Aufgabe, ein (LVM-basiertes) "Raw-"-Device mit dmcrypt/LUKS zu verschlüsseln, lässt sich unter Opensuse Leap (42.2/42.3) entgegen jeder Erwartung mit YaST leider nicht erfolgreich durchführen. YaST wickelt die Anlage eines verschlüsselten Volumes nur dann korrekt ab, wenn das Filesystem bereits vorgegeben wird - also NICHT bei der Verschlüsselung für Raw-Devices innerhalb der Reihenfolge :

Partition   >>   LVM-Group   >>   Raw LVM-Volume   >>   dm-crypt/LUKS   >>   KVM-guest   >>   LVM im Gast   >>   Filesystem

Das ist bereits von anderen beklagt worden; siehe etwa:
https://forums.opensuse.org/showthread.php/528938-installation-with-LUKS-cryptsetup-installer-gives-error-code-3034

Man muss also auf der Kommandozeile arbeiten und das Kommando "cryptsetup" bemühen. Für unser Beispiel-Volume "lvimap_hd0" (erste virt. Platte des künftigen E-Mail-Servers) lautet ein mögliches Kommando zur Verschlüsselung dann:

myserv:~ # cryptsetup luksFormat --hash=sha512 --key-size=512 --cipher=aes-xts-plain64 --verify-passphrase /dev/volgrp4/lvimap_hd0

Für Details zu den verfügbaren Verschlüsselungs- und Hash-Verfahren werfe man einen Blick in die man-Seiten zum Kommando. Man muss die künftige Passphrase zweimal eingeben. Damit ist die Verschlüsselung aktiv. Ein anschließender Blick auf das Volume mit dem YaST-Partitioner zeigt entsprechend ein Verschlüsselungssymbol an.

Altes KVM-Gast-System mit "dd" kopieren oder mit "virt-manager" klonen?

Bzgl. des geplanten Umzugs unseres unter KVM/QEMU virtualisierten (Mail-) Servers auf das neue Krypto-Device stellt sich nun die Frage, wie wir das am besten bewerkstelligen:

Sollen wir "dd" einsetzen? Das ist zwar möglich, erfordert anschließend aber wegen kopierter UUIDs, MAC-Adressen etc. etliche manuelle Nacharbeiten im geklonten System und auf dem Host.

Besser ist deshalb aus meiner Sicht ein Cloning mit Hilfe von virt-manager! Dabei ist folgende Schrittfolge einzuhalten:

  • Schritt 1 : Öffnen des neuen dm-crypt-Devices unter einem verständlichen Namen
     
    myserv:~ # cryptsetup open /dev/mapper/volgrp4-lv_imap_hd0  cr_imap_encr

    Dabei muss natürlich die bereits gesetzte Passphrase für das Crypto-Device eingegeben werden. "cr_imap_encr" steht dann als ansprechbares Device unter "/dev/mapper/" bereit

  • Schritt 2 : virt-manager öffnen und in der Liste der Gäste mit der rechten Maustaste auf den zu klonenden Gast klicken. Dann die Option "Clone ..." wählen. Das führt uns dann zu einer Maske, die in etwa so aussieht und die zu klonende virtuelle Platte (vm2_hd1) des vorhandenen Gastes anbietet:
     

    Bzgl. der zu klonenden Disk öffnet man die Drop-Down Box und wählt "Details" :

  • Schritt 3 : In der sich neu öffnenden Maske trägt man den Pfad zum geöffneten "dm-crypt"-Decvice ein:
     

    Dann in der aktuellen Maske den "OK"-Button und anschließend in der Ausgangsmaske den Button "Clone" drücken.

Je nach Größe unseres virtuellen Servers und der Schnelligkeit der Plattensysteme wie des Prozessors dauert das etwas. Anschließend erhalten wir allerdings einen Clone, den man sofort booten kann. (Vorher den ursprünglichen Mail-Server natürlich runterfahren, um IP-Konflikte zu vermeiden. Das geklonte System hat immer noch die gleiche IP-Adresse wie das alte! )

Das war es im Wesentlichen schon. Wir haben erfolgreich ein virtualisiertes Server-System auf einen verschlüsselten LVM-Volume-Unterbau umgezogen!

Wie schließt man das verschlüsselte System manuell?

Unsere manuelle Vorgehensweise hat dazu geführt, dass das verschlüsselte Volume nicht in Systemdateien eingetragen wurde. Es steht daher nach einem Boot-Vorgang des Hosts nicht automatisch zu Verfügung. Auch das Passwort für die Entschlüsselung wird im Bootvorgang nicht automatsich abgefragt.

Ein aktivierter "libvirtd"-Service kann daher nach seinem Start im Zuge eines Bootens des Hosts auch nicht auf das Volume für den umgezogenen Gast zugreifen. Ein automatisches Hochfahren des Gastes über libvirt-Einstellungen ist somit ebenfalls nicht möglich. Das sind Probleme, um die wir uns in nachfolgenden Artikeln kümmern müssen.

An dieser Stelle möchte ich aber wenigstens den notwendigen manuellen Befehl für das Schließen des Crypto-Devices im Zuge eines Herunterfahrens angeben:

  • Schritt 1 - Herunterfahren des Gastsystems: Dies geschieht entweder über Standard-Befehle im Gast selbst oder z.B über virt-manager.
  • Schritt 2 - Schließen des Crypto-Devices :
     
    myserv:~ # cryptsetup close cr_imap_encr

Ausblick

In folgenden Artikeln möchte ich im Nachgang zu unserem "Umzug" ein wenig auf das Thema eingehen, wie man das Hochfahren des KVM-Hosts mit Gastsystemen auf verschlüsselten Volumes gestalten kann. Zudem wollen wir den "Verschlüsselungsheader" sichern und uns mit den Themen Backup und "fstrim" für SSDs als Basis der beschriebenen Schichtung befassen. Sinnvoll ist auch ein Blick auf (Mail-) Client-Systeme: Von irgendwo aus muss ja auch mal grafisch auf die virtualisierten Systeme zugegriffen werden. Dann werden z.B. Spice oder X2GO-Clients samt zugehörigen Protokollen zu einem Sicherheitsthema auf dem System, von dem aus wir auf virtualisierte Clients für die Auftragsbearbeitung zugreifen. Verschieben wir mit der Virtualisierung im Client-Umfeld also nur die Sicherheitsebene?

Links

P2V-Tools
http://manuel.kiessling.net/2013/03/19/converting-a-running-physical-machine-to-a-kvm-virtual-machine/
http://libguestfs.org/virt-p2v.1.html
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/v2v_guide/chap-v2v_guide-p2v_migration_converting_physical_machines_to_virtual_machines
http://events17.linuxfoundation.org/sites/events/files/slides/virt-v2v-rjones-backup-slides.pdf
http://qemu-buch.de/de/index.php?title=QEMU-KVM-Buch/_Speichermedien/_Physical-to-Virtualp2v_migration_converting_physical_machines_to_virtual_machines
https://access.redhat.com/articles/1351473

dm-crypt/Luks und die Reihenfolge von Zugriffsschichten
https://superuser.com/questions/1193290/best-order-of-raid-lvm-and-luks
https://blog.raptor2101.de/2009/09/23/verschlusselung-von-raids/
http://www.andreas-janssen.de/cryptodisk.html
http://linux-club.de/wiki/opensuse/Verschluesselung:_dm-crypt/luks_unter_openSUSE
https://wiki.archlinux.org/index.php/Dm-crypt/Specialties